Information processing method and related device

Abstract

Provided are an information processing method and a related device, which relate to the technical field of computers and communications. The method comprises: obtaining probabilistic text embedding vectors (S110); obtaining an image feature of an image, and obtaining the degrees of matching between the probabilistic text embedding vectors and the image feature (S120); using the probabilistic text embedding vector with the highest degree of matching as a target probabilistic text embedding vector (S130); and determining a target prompt template corresponding to the target probabilistic text embedding vector, and on the basis of target scenario prompt information corresponding to the target prompt template, determining a target scenario corresponding to the image (S140).

Description

Information processing methods and related equipment

[0001] Cross-reference to related applications

[0002] This disclosure claims priority to Chinese Patent Application No. 202410993678.8, filed on July 23, 2024, entitled "Information Processing Method and Related Equipment", the entire contents of which are incorporated herein by reference. Technical Field

[0003] This disclosure relates to the fields of computer and communication technology, and in particular to an information processing method, information processing apparatus, electronic device, computer-readable storage medium, and computer program product. Background Technology

[0004] In some applications, it is necessary to process and recognize images to determine the corresponding scenes. The solutions provided by related technologies need to further improve the accuracy and efficiency of image scene recognition. Summary of the Invention

[0005] This disclosure provides an information processing method, comprising: obtaining a probabilistic text embedding vector; obtaining image features of an image and obtaining a matching degree between the probabilistic text embedding vector and the image features; selecting the probabilistic text embedding vector with the highest matching degree as a target probabilistic text embedding vector; determining a target prompt template corresponding to the target probabilistic text embedding vector; and determining a target scene corresponding to the image based on target scene prompt information corresponding to the target prompt template. In an exemplary embodiment, the method may be executed by an electronic device (e.g., a terminal and / or a server), or the method may be executed by a device (e.g., a chip) configured in an electronic device.

[0006] This disclosure provides an information processing apparatus. In one design, the apparatus may include modules corresponding to the methods / operations / steps / actions described in any embodiment of this disclosure. These modules may be hardware circuits, software, or a combination of hardware circuits and software. In one design, the apparatus includes: a processing unit configured to obtain probabilistic text embedding vectors; the processing unit is further configured to obtain image features of an image and obtain a matching degree between the probabilistic text embedding vectors and the image features; the processing unit is further configured to use the probabilistic text embedding vector with the highest matching degree as a target probabilistic text embedding vector; the processing unit is further configured to determine a target prompt template corresponding to the target probabilistic text embedding vector, and determine the target scene corresponding to the image based on the target scene prompt information corresponding to the target prompt template.

[0007] This disclosure also provides a processor, including an input circuit, an output circuit, and a processing circuit. The processing circuit is used to receive signals through the input circuit and to transmit signals through the output circuit, causing the processor to execute the method in any embodiment of this disclosure.

[0008] In exemplary embodiments, the processor described above can be one or more chips, the input circuit can be an input pin, the output circuit can be an output pin, and the processing circuit can be a transistor, a gate circuit, a flip-flop, and various logic circuits, etc. The input signal received by the input circuit can be received and input by, for example, but not limited to, a receiver, and the signal output by the output circuit can be, for example, but not limited to, output to a transmitter and transmitted by the transmitter. Furthermore, the input circuit and the output circuit can be the same circuit, which is used as both the input circuit and the output circuit at different times. This disclosure does not limit the specific implementation of the processor and various circuits.

[0009] This disclosure provides an electronic device including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform an information processing method according to any embodiment of this disclosure by executing the executable instructions.

[0010] This disclosure provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the information processing method of any embodiment of this disclosure.

[0011] This disclosure provides a computer program product, including a computer program that, when executed by a processor, implements the information processing method of any embodiment of this disclosure. Attached Figure Description

[0012] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0013] Figure 1 shows a flowchart of an information processing method according to an embodiment of the present disclosure.

[0014] Figure 2 shows a schematic diagram of an information processing method according to an embodiment of the present disclosure.

[0015] Figure 3 shows a schematic diagram of a probability-based cue learning module in an embodiment of this disclosure.

[0016] Figure 4 shows a schematic diagram of another information processing method according to an embodiment of the present disclosure.

[0017] Figure 5 shows a structural block diagram of an information processing device according to an embodiment of the present disclosure.

[0018] Figure 6 shows a structural block diagram of an electronic device according to an embodiment of the present disclosure. Detailed Implementation

[0019] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, these exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, they are provided so that this disclosure will be more comprehensive and complete, and will fully convey the concept of the exemplary embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0020] Furthermore, the accompanying drawings are merely illustrative of this disclosure, and the same reference numerals in the drawings denote the same or similar parts, thus repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0021] The following explains some of the terms used in the embodiments of this disclosure.

[0022] General large model: The large model can also be called the Foundation Model or the Pre-Training Unit (PTU). The model extracts and learns knowledge from hundreds of millions of corpora or images, and then produces a large model with hundreds of millions of parameters.

[0023] Cue learning: Without significantly altering the pre-trained model structure, the model's parameters can be fine-tuned by adding "cues" to the input or changing the downstream task to a text generation task. For example, in the following illustration, K (K is a positive integer greater than or equal to 1, also referred to as the second quantity) cue templates are provided, and the model determines which of these K cue templates corresponds to which scene an image or sample image belongs to. When fine-tuning the model's parameters, a loss function can be calculated based on the predicted scene and the actual scene labels, and the model parameters can be adjusted based on this loss function.

[0024] Probabilistic learning: How individuals learn to predict whether an event will occur or not, within the bounds of probabilistic principles. It is understood that while the following examples use a Gaussian distribution, this disclosure is not limited to this. If other distributions are used, the formulas below can be modified accordingly, for example, by using a uniform distribution. Different distributions yield different results for different data sources; some Gaussian distributions are more effective, while others are more effective with uniform distributions. The appropriate distribution can be chosen based on the specific scenario.

[0025] Video Network: The video network is the fifth basic network, in addition to mobile networks, broadband networks, the Internet of Things, and satellite networks. It is a video backhaul and processing network.

[0026] The specific implementation methods of the embodiments of this disclosure will now be described in detail with reference to the accompanying drawings.

[0027] Figure 1 shows a flowchart of an information processing method according to an embodiment of this disclosure. The method provided in the embodiment of Figure 1 can be executed by any electronic device, such as a terminal and / or a server, which is not limited thereto. As shown in Figure 1, the method provided in the embodiment of this disclosure may include the following steps.

[0028] In S110, the probabilistic text embedding vector is obtained.

[0029] In this embodiment of the disclosure, the probabilistic text embedding vector refers to a vector used to characterize the sampling probability of each prompt template. If the sampling probability of a prompt template is higher in the probabilistic text embedding vector, then the prompt template is more likely to be selected for feature matching with image features / sample image features. In some embodiments, the dimension of the probabilistic text embedding vector can be the same as the number of prompt templates. For example, if there are K (also referred to as a second number) prompt templates, then each probabilistic text embedding vector has K dimensions, where the value of the k-th dimension indicates the probability of selecting the k prompt templates. Here, k is a positive integer greater than or equal to 1 and less than or equal to K.

[0030] In an exemplary embodiment, the method provided by this disclosure further includes: obtaining a second number of prompt templates; inputting the second number of prompt templates into a frozen and pre-trained text encoder to obtain a second number of text vectors.

[0031] In this embodiment, the prompt template refers to a preset template in which prompt information is embedded. This prompt information is used to assist or guide the model in making predictions, thereby improving the accuracy and speed of prediction. The preset template can be set according to the actual scenario, and this disclosure does not limit the specific form of the template. The prompt information can be determined according to the actual content of the prediction output. For example, if it is necessary to predict the scene of an image, the prompt information may include scene prompt information. Scene prompt information refers to prompt information used to assist or guide the model in predicting the scene corresponding to the image.

[0032] In this embodiment of the disclosure, a text encoder refers to a machine learning model or deep learning model that can encode text data. This embodiment of the disclosure does not limit the type or structure of the text encoder.

[0033] In this embodiment, the text encoder can be pre-trained using a text dataset. Before obtaining the probabilistic text embedding vectors, K cue templates can be input into the pre-trained text encoder. During the process of obtaining the K text vectors corresponding to the K cue templates, the text encoder is frozen; that is, the model parameters of the pre-trained text encoder are used, and the model parameters are not updated or adjusted. These K text vectors can then be used to generate the probabilistic text embedding vectors.

[0034] In an exemplary embodiment, the method includes a first number of probabilistic text embedding vectors, wherein the probability distribution embedding vectors have a second number of dimensions, each dimension indicating the sampling probability of the corresponding prompt template; each prompt template includes corresponding scene prompt information. Specifically, it includes C probabilistic text embedding vectors, wherein the probability distribution embedding vectors are K-dimensional, and the k-th dimension indicates the sampling probability of the k-th prompt template among the K prompt templates. Wherein, K and C are both positive integers greater than or equal to 1, K is greater than C, k is a positive integer greater than or equal to 1 and less than or equal to K, and the k-th prompt template includes the k-th scene prompt information.

[0035] In this embodiment, C probabilistic text embedding vectors are generated based on K prompt templates, and each probabilistic text embedding vector includes K dimensions, where the value of the k-th dimension indicates the sampling probability of the k-th prompt template. On one hand, since the number of prompt templates in real-world scenarios is usually large (e.g., K equals 50, 100, 200, etc.), directly matching the K text vectors of the K prompt templates with image features would require significant computational resources, and both matching speed and efficiency would decrease. This disclosure uses a smaller number of C probabilistic text embedding vectors to match image features, which reduces computational resource consumption and improves matching speed and efficiency. For example, C can be 5, 10, or 15. On the other hand, by keeping the dimension of the probabilistic text embedding vector consistent with the number of prompt templates, and by having the k-th dimension of the probabilistic text embedding vector indicate the sampling probability of the k-th prompt template, the complete K prompt templates can be represented using C probabilistic text embedding vectors.

[0036] In this embodiment of the disclosure, the kth prompt template includes the kth scene prompt information, and different prompt templates contain different scene prompt information.

[0037] In an exemplary embodiment, obtaining a probabilistic text embedding vector includes: obtaining a first number of candidate probabilistic text embedding vectors, wherein the candidate probability distribution embedding vectors have a second number of dimensions, each dimension indicating the candidate sampling probability of the corresponding prompt template (e.g., the k-th dimension indicates the candidate sampling probability of the k-th prompt template); obtaining sample image features of a sample image and scene label information of the sample image; obtaining the sample matching degree between each candidate probabilistic text embedding vector and the sample image features; taking the candidate probabilistic text embedding vector with the largest sample matching degree as the target candidate probabilistic text embedding vector; determining the prompt template corresponding to the target candidate probabilistic text embedding vector and its corresponding scene prompt information, thereby determining the scene prediction information corresponding to the sample image; obtaining the difference degree between the scene prediction information of the sample image and the scene label information; and adjusting the first number of candidate probabilistic text embedding vectors according to the difference degree to obtain the first number of probabilistic text embedding vectors.

[0038] In this embodiment of the disclosure, the candidate probabilistic text embedding vector refers to the probabilistic text embedding vector that is being adjusted using the sample image and its corresponding scene label information. When the candidate probabilistic text embedding vector is used to make the difference between the predicted scene prediction information and its scene label information of the sample image meet the preset conditions, the candidate probabilistic text embedding vector at this time can be used as the probabilistic text embedding vector. That is, the probabilistic text embedding vector is the candidate probabilistic text embedding vector that has been adjusted using the sample image and its corresponding scene label information and makes the difference meet the preset conditions.

[0039] In this embodiment of the disclosure, the scene label information of the sample image refers to the pre-labeled real scene of the sample image. The scene prediction information of the sample image refers to the scene of the sample image predicted by the model, which may be consistent with its real scene, may have a certain deviation, or may even be incorrect.

[0040] In this embodiment of the disclosure, the degree of difference refers to the degree of deviation or index between the scene prediction information of the sample image and its true scene label information. There are various ways to measure the degree of difference, such as the distance between the scene prediction information and the scene label information (e.g., Euclidean distance, but this disclosure is not limited to this), loss function (e.g., cross-entropy loss function), etc., and this disclosure does not limit it.

[0041] In this embodiment, the preset conditions can be set according to actual needs. For example, when the adjusted candidate probability text embedding vector is used for feature matching with the sample image features, and the predicted scene prediction information of the sample image completely matches the real scene label information (the difference is, for example, 0), then the difference is considered to meet the preset conditions. However, this disclosure is not limited to this.

[0042] If the difference does not meet the preset condition, you can go back and continue adjusting the values ​​of each dimension in the candidate probability text embedding vector. For example, you can instruct the selection of another C prompt templates from the remaining KC prompt templates. Here, the remaining KC prompt templates refer to the prompt templates other than the C prompt templates randomly selected in the first step. If the difference still does not meet the preset condition after adjustment, adjust the values ​​of each dimension in the candidate probability text embedding vector again. For example, you can instruct the selection of another C prompt templates from the remaining K-2C prompt templates. Repeat this process until the difference meets the preset condition.

[0043] In an exemplary embodiment, obtaining a first number of candidate probability text embedding vectors includes: obtaining a first number of initial probability text embedding vectors, wherein the initial probability distribution embedding vector has a second number of dimensions, each dimension indicating the initial probability of the corresponding prompt template (e.g., the k-th dimension indicates the initial probability of the k-th prompt template); using the initial probability text embedding vectors as the center vectors of the corresponding candidate probability text embedding vectors; obtaining the diagonal covariance matrix of the initial probability text embedding vectors; and determining the probability density function of the candidate probability text embedding vectors based on the initial probability text embedding vectors as a Gaussian probability distribution with the corresponding center vector and the diagonal covariance matrix, thereby obtaining the corresponding candidate probability text embedding vectors. For example, using the c-th initial probability text embedding vector as the center vector of the c-th candidate probability text embedding vector; obtaining the diagonal covariance matrix of the c-th initial probability text embedding vector; and determining the probability density function of the c-th candidate probability text embedding vector based on the c-th initial probability text embedding vector as a Gaussian probability distribution with the center vector and the diagonal covariance matrix, thereby obtaining the c-th candidate probability text embedding vector.

[0044] In this embodiment of the disclosure, candidate probability text embedding vectors can be initialized to obtain initial probability text embedding vectors. For example, during initialization, C prompt templates can be randomly selected from K prompt templates, and C initial probability text embedding vectors can be used to represent these C selected prompt templates respectively. For example, for a certain initial probability text embedding vector, assuming it is represented as {1,0,0,…,0}, it indicates that the first prompt template among the K prompt templates has been selected. As another example, if a certain initial probability text embedding vector is represented as {0,0,1,…,0}, it indicates that the third prompt template among the K prompt templates has been selected. That is, when a certain initial probability text embedding vector indicates that the k-th prompt template has been selected, the value of the k-th dimension of the initial probability text embedding vector is greater than the values ​​of the other dimensions.

[0045] In this embodiment, each of the C initial probability text embedding vectors is processed to satisfy a Gaussian distribution, thereby enabling the obtained C candidate probability text embedding vectors to represent the candidate sampling probabilities of K cue templates. Simultaneously, this makes the selection of C cue templates from the K cue templates for matching with image features or sample image features more systematic and predictable.

[0046] In an exemplary embodiment, obtaining a first number of candidate probability text embedding vectors further includes: resampling each candidate probability text embedding vector to make it conform to a normal distribution.

[0047] Optionally, after ensuring each candidate probability text embedding vector follows a Gaussian distribution, each candidate probability text embedding vector can be resampled to follow a normal distribution. This allows the normally distributed candidate probability text embedding vectors to be used for feature matching with the sample image features, thus obtaining the sample matching degree.

[0048] In S120, the image features of the image are obtained, and the matching degree between the probabilistic text embedding vector and the image features is obtained.

[0049] In this embodiment of the disclosure, the matching degree refers to the degree of similarity between the probabilistic text embedding vector and the image features. The higher the matching degree, the higher the similarity; the lower the matching degree, the lower the similarity.

[0050] In an exemplary embodiment, obtaining sample image features of a sample image includes: inputting the sample image into a frozen and pre-trained image encoder to obtain the sample image features.

[0051] The image encoder in this disclosure refers to a machine learning or deep learning model used to extract image features to encode information or data in an image. This disclosure does not limit the type or structure of the image encoder. In this disclosure, the image encoder is first pre-trained using an image dataset. When obtaining sample image features, the sample image is input into the pre-trained image encoder, and during this period, the model parameters of the image encoder are frozen; that is, the model parameters of the pre-trained image encoder are used.

[0052] In an exemplary embodiment, the image features include K dimensions, where K is a positive integer greater than or equal to 1. The cross-entropy between the probabilistic text embedding vector and the image features is obtained based on the following formula:

[0053] Where c is a positive integer greater than or equal to 1 and less than or equal to C, C is the number of probabilistic text embeddings, and C is a positive integer greater than or equal to 1; q (d,c) This represents the cross-entropy between the c-th probability text embedding vector and the image feature; The k-th dimension represents the image feature, where k is a positive integer greater than or equal to 1 and less than or equal to K; Let k represent the k-th dimension in the c-th probability text embedding vector.

[0054] In some embodiments, the matching degree between probabilistic text embedding vectors and image features can be measured by calculating the cross-entropy between each probabilistic text embedding vector and the image features. The matching degree calculated in this way is inversely correlated with the cross-entropy; that is, the larger the cross-entropy, the lower the matching degree, and the smaller the cross-entropy, the higher the matching degree.

[0055] It is understood that although the embodiments of this disclosure propose the above-described methods for measuring matching degree, this disclosure is not limited to this. For example, matching degree can also be measured by cosine similarity or Euclidean distance between probabilistic text embedding vectors and image features.

[0056] In some embodiments, the sample matching degree can be calculated with reference to formula (1). In this case, q in formula (1) (d,c) It can represent the cross-entropy between the c-th candidate probability text embedding vector and the sample image features; The k-th dimension represents the feature of the sample image; Let k represent the k-th dimension in the text embedding vector of the c-th candidate probability. However, this disclosure is not limited thereto.

[0057] In S130, the probability text embedding vector with the highest matching degree is used as the target probability text embedding vector.

[0058] For example, the probability text embedding vector with the highest matching degree can be selected from C probability text embedding vectors as the target probability text embedding vector.

[0059] In S140, the target prompt template corresponding to the target probability text embedding vector is determined, and the target scene corresponding to the image is determined based on the target scene prompt information corresponding to the target prompt template.

[0060] In this embodiment of the disclosure, based on the values ​​of each dimension in the K-dimensional target probability text embedding vector, the prompt template corresponding to the dimension with the largest value can be determined as the target prompt template. For example, assuming the target probability text embedding vector is {0.7, 0.001, 0.002, 0, ..., 0}, that is, the first dimension in the K-dimensional vector has the largest value, which is equal to 0.7. Since the first dimension corresponds to the first prompt template among the K prompt templates, the first prompt template is determined as the target prompt template.

[0061] In this embodiment of the disclosure, each of the K prompt templates includes corresponding scene prompt information. The scene prompt information in the target prompt template is taken as the target scene prompt information. For example, if the first prompt template is the target prompt template and the target scene prompt information contained in the first prompt template is "kitchen", then the target scene prompt information is determined to be "kitchen", thereby determining that the target scene corresponding to the image is "kitchen".

[0062] In an exemplary embodiment, the method provided in this disclosure further includes: acquiring the image from an image acquisition device; and determining the location information of the image acquisition device based on the target scene corresponding to the image.

[0063] In this embodiment of the disclosure, an image can be captured by an image acquisition device (such as a camera or surveillance camera, but this disclosure is not limited to these; examples are used below for illustration only), and the image can be sent to the electronic device. The electronic device can obtain the image and the identification information of the image acquisition device. This identification information is used to uniquely identify the image acquisition device, that is, to indicate that the image was captured by the image acquisition device. By processing the image using the above method, the target scene corresponding to the image can be determined, and thus the target scene can be used as the location information of the image acquisition device. That is, the location information here refers to the scene captured, located in, or monitored by the image acquisition device. This enables the automatic acquisition and automatic annotation of the location information of the image acquisition device.

[0064] The information processing method provided in this disclosure uses probabilistic text embedding vectors to perform feature matching with image features in order to identify the target scene corresponding to the image, thereby improving the efficiency and accuracy of image scene recognition.

[0065] The 5G (5th Generation Mobile Communication Technology) era has spurred a massive number of video-based applications, such as smart and safe cities, the internet, autonomous driving, and video surveillance, all based on image and video content, with a very wide range of application scenarios. Image and video-related applications will become one of the main sources of incremental traffic in the 5G and post-5G era.

[0066] Driven by new infrastructure and technologies such as big data, AI (Artificial Intelligence), 5G, cloud computing, and IoT (Internet of Things), and in line with the development trend of digital services and industries, the Video Network (also known as the Video Network) was launched.

[0067] With the rapid development of 5G, big data, and artificial intelligence, video has become ubiquitous in work and life. However, in video application scenarios, such as video surveillance, the location information (or position information) of surveillance cameras (which are a type of image acquisition device) is often missing. For example, if there are 50 cameras in a building, it's impossible to know their exact locations, such as which cameras are in the stairwells, corridors, meeting rooms, kitchens, etc.

[0068] The video surveillance system in the relevant technology cannot automatically acquire and label the location information of the cameras. It can only rely on human eyes to view the video and manually label the location information of the cameras. When the camera has historical tags, the manually labeled location information is manually compared with the historical tags of the camera to finally determine the location information of the camera.

[0069] The mainstream framework for acquiring camera location information in video networks in related technologies mainly includes the following modules:

[0070] (1) Image information input section, which is used to input the image information of the camera in the video network to the system and transmit it to the manual processing module section.

[0071] (2) Manual processing module, which manually processes the image information of the camera in the video network, and understands the scene information or scene label information in the camera image based on human eyes.

[0072] (3) Tag information comparison module compares the existing tag location information (i.e., historical tags or historical location information) of the camera with the scene tag information in the image as understood by the human eye. The historical location information was manually labeled before this.

[0073] (4) Obtain the label comparison results.

[0074] (5) Obtain the camera's location information / location information / location information based on the label comparison results.

[0075] This is because, in real-world scenarios, the location or scene of some cameras may change. For example, the place monitored by the camera might be renovated; previously a kitchen, it might become a barbershop. This leads to inconsistencies between the previously classified scene and the current classification. Due to the large number of cameras, manually updating the location information typically takes a long time, resulting in incorrect camera location information stored in the system, causing difficulties or errors in subsequent processing. However, the solution or large model provided in this disclosure can automatically and periodically scan the images captured by the cameras, perform image recognition processing, and update the camera location information in a timely manner. This improves the efficiency of updating camera location information and prevents errors in subsequent analysis and processing caused by incorrect location information.

[0076] The system modules in the related technologies have the following technical problems:

[0077] (1) The video network in related technologies is to understand the scene by viewing the camera images with the human eye. However, the video network has tens of millions of cameras, and it is operated manually by operators, which requires a huge investment of manpower and is labor-intensive and time-consuming.

[0078] (2) The video network in related technologies does not understand the content of the image / video itself, but compares the text information of the video tags. Some of the video tags are manually annotated in history, and due to the long time, there are certain errors.

[0079] (3) In the video surveillance system of related technologies, the tag information comparison module compares the existing tag location information with the scene tag information in the image as understood by the human eye. This is done manually by the operators, which is a huge workload.

[0080] As can be seen from the above, the video systems or video surveillance systems in the related technologies cannot automatically acquire the location information of each camera, resulting in the inability to automatically label the location information of each camera; nor can they understand the scene information of the cameras based on the keywords of the location information. In these related technologies, the only way is for humans to view the surveillance video and manually label the location information of each camera. The video network management platform in these technologies has approximately 30 million front-end cameras, making the viewing of video surveillance extremely time-consuming and labor-intensive. Furthermore, the existing artificial intelligence and deep learning algorithms in these technologies cannot meet the application requirements of accuracy, scene understanding, and abstract semantics for intelligent video analysis tasks.

[0081] In some scenarios, existing technologies cannot accurately locate camera positions based on keywords in location information. However, using the method provided in this disclosure, after obtaining the latitude and longitude (or geographical location information) and location information of the camera, the scene category carried by the latitude and longitude and location information can be combined to obtain more accurate camera positioning. For example, it can be distinguished that there are 4 cameras in the kitchen, 1 in the kitchen on the 6th floor, and 1 in the kitchen in the garden, thus locating the kitchens where the cameras are deployed in different locations.

[0082] The purpose of the method provided in this disclosure is to overcome the shortcomings of related technologies, and to achieve automatic acquisition of the location information of each camera; automatic annotation of the location information of each camera; and understanding of camera scene information based on keywords in the location information. Optionally, it can also perform high-accuracy camera location information positioning based on keywords in the location information.

[0083] The method provided in this disclosure, for video networks, especially surveillance video systems, designs a probability distribution-based prompting learning method for fine-tuning large model parameters, and designs a pixel text matching loss calculation method based on cross entropy (refer to the above formula (1)). When scene recognition is required for cameras deployed in a new location, such as a new city, it can improve the accuracy of scene category judgment; it can also realize the automatic acquisition and automatic labeling of the location information of each camera in the video network.

[0084] As shown in Figure 2, the prompt template 11 is input into the text encoder 12 to obtain the text vector of the prompt template. The text encoder 12 is connected to the probabilistic prompt learning module 13 so that the probabilistic prompt learning module 13 can generate candidate probability distribution embedding vectors and probabilistic text embedding vectors based on the prompt template.

[0085] Image 21 is input to image encoder 22. Here, image 21 can be a sample image during the adjustment of the candidate probability distribution embedding vector. Feature matching module 30 is connected to both probability cue learning module 13 and image encoder 22 to obtain the matching degree between the probability text embedding vector and image features. Then, based on the probability text embedding vector with the highest matching degree, output 40 is obtained. This output (also called Results) 40 can be the target scene corresponding to image 21. Alternatively, feature matching module 30 obtains the sample matching degree between the candidate probability text embedding vector and sample image features. Based on the candidate probability text embedding vector with the highest sample matching degree, scene prediction information corresponding to the sample image is obtained. In this case, output 40 can be the scene prediction information corresponding to the sample image.

[0086] The text encoder and image encoder in Figure 2 are both pre-trained and frozen. Furthermore, the text encoder and image encoder are aligned, which can be achieved, for example, through CLIP (Contrastive Language-Image Pre-training, a pre-trained model based on contrastive text-image pairs) or BLIP-2 (an efficient vision-language pre-training method).

[0087] Figure 3 shows a schematic diagram of the probabilistic cue learning module 13 in Figure 2. As shown in Figure 3, the probabilistic cue learning module 13 includes a deterministic embedding between class and prompts module 131 and a probabilistic embedding based on Gaussian distribution module 132. The input of the deterministic embedding between class and prompts module 131 is connected to the output of the text encoder 12, but can also be decoupled from the text encoder 12. The output of the deterministic embedding between class and prompts module 131 is connected to the input of the probabilistic embedding based on Gaussian distribution module 132.

[0088] In an exemplary embodiment, the probability cue learning module 13 may further include a resampling analysis module 133. The input of the resampling analysis module 133 is connected to the output of the Gaussian distribution-based probability embedding module 132. The output of the resampling analysis module 133 or the output of the Gaussian distribution-based probability embedding module 132 serves as the output of the probability cue learning module 13.

[0089] Figure 4 illustrates the structure of the point governance system based on Gaussian distribution probabilistic cue learning with large model cross-entropy for visual networks, as proposed in this embodiment. As shown in Figure 4, the feature matching module 30 may further include a cross-entropy calculation module 31 and a similarity calculation module 32. When the difference (e.g., loss) between the output 40 and the scene label information does not meet preset conditions, the system returns to the deterministic embedding module 131 of the probabilistic cue learning module 13 to readjust the values ​​of the C candidate probabilistic text embedding vectors, for example, by reselecting new C cue templates.

[0090] The methods and systems provided in the embodiments of this disclosure are illustrated below with reference to Figures 3 and 4.

[0091] As shown in Figure 3, assume that prompt template 11 includes K prompt templates. For example, assume that the first prompt template p1 is "This is a picture of a kitchen scene", the second prompt template p2 is "This is a picture of a street scene", the third prompt template p3 is "This is a picture of a garbage room scene", the fourth prompt template p4 is "This is a picture of an elevator scene", ..., and the Kth prompt template pK is "This is a picture of a construction site scene".

[0092] In this embodiment of the disclosure, multiple text representations can be used to construct a prompt template P, which is obtained through model learning and constitutes the prompt templates of the scene target category or K categories in the scene category to be judged, as shown below:

[0093] Where p 1 ,p 2 ,……p K The terms "etc." can be represented as follows:

[0094] In the formula above, L represents the set length of the prompt word (i.e., the length of the prompt template), and L is a positive integer greater than or equal to 1. It's understandable that different prompt templates may contain different numbers of characters, but for prompt templates, the scene prompt information in prompt templates such as (kitchen) and (garbage room) can be enclosed in parentheses so that the length L seen by the model is the same.

[0095] Here, the image is represented as g, and the image encoder is... The text encoder is This model structure includes an image encoder. and text encoder K prompt templates are input into text encoder 12 to obtain K text vectors. Image 21 is input into image encoder 22. As shown in Figure 4, it is assumed that the input image (i.e., image 21) to image encoder 22 is a scene from a garbage room surveillance camera. Text Encoder For example, it can be a transformer model. The image encoder can use any image encoding method.

[0096] In this embodiment of the disclosure, when the probabilistic cue learning module 13 adopts a Gaussian distribution, it can be referred to as a Gaussian distribution probabilistic cue learning module. This embodiment of the disclosure designs a Gaussian distribution probabilistic cue learning module based on a large model. Figure 3 shows the structure of the Gaussian distribution probabilistic cue learning module based on a large model designed in this embodiment of the disclosure. Through the Gaussian distribution probabilistic cue learning module based on a large model, probabilistic text embeddings or probabilistic text embedding vectors are effectively represented using Gaussian distribution probabilistic cue learning.

[0097] After the K types of prompt templates are processed by the text encoder, they will be processed by the deterministic embedding module 31, which calculates the relationship between the category and the prompt. Subsequent inputs are the probability embedding module 132 (or the probability embedding module based on the factor Gaussian distribution) and the resampling analysis module 133.

[0098] In this embodiment, in the deterministic embedding module 131 between categories and prompts, 15 prompt templates are randomly selected from 200 prompt templates. C represents the number of scene target categories to be judged, i.e., the number of prompt templates selected at one time when performing feature matching with image 21. K refers to the number of preset prompt templates in the database, which is usually relatively large (e.g., 200) and is frequently updated with new prompt templates added continuously. C is initially preset, for example, C = 10, 15, 20. C is less than K. The probabilistic prompt learning module 13 selects C prompt templates from K prompt templates to match with image 21, because image 21 is usually not related to K prompt templates, but only to a small portion of them. If 200 prompt templates are directly used for matching each time, the computational load is large. In this embodiment, C (e.g., 15) prompt templates are randomly selected, and then the difference (e.g., distance) between them and image 21 is calculated. If the distance between these 15 prompt templates and image 21 is relatively large, the process can return to selecting 15 new prompt templates from the 200 prompt templates. The value of C can be adjusted later during fine-tuning, i.e., training.

[0099] The deterministic embedding module 131 between categories and prompts can be based on a text encoder. The output K types of prompt templates are used to generate candidate probability text embedding vectors (also known as text embedding representations) w for each scene target category c to be judged. c , will w c The hint template representation of class C is a collection:

[0100] Where w1, w2, w3, ... w c These can be represented as follows:

[0101] Representation set w c This is used to define a probability distribution from which probability cues for scene category c are sampled. For example, initially, w1 is set to {1, 0, 0, ...}, which represents the first cue template in the Kth cue template. Subsequent probability sampling is performed to ensure that w... c The values ​​of each dimension change, for example, becoming w1 = {0.7, 0.001, 0, ...}, meaning that some dimensions' values ​​will be close to 0. This makes the extraction of prompt templates more predictable and, on the other hand, allows each w to... c Instead of simply representing a single prompt template, combining C prompt templates can represent K prompt templates, but only C vectors need to be calculated during the computation, thereby reducing the amount of data computation.

[0102] Deterministic embedding module 131 between categories and prompts, each wc Let w represent a K-dimensional row vector. The probabilistic embedding module 132, based on a Gaussian distribution, processes the above w1, w2, ... to w... each time. C We have a row vector in the array, and we process each row vector as follows. The following example uses the c-th row vector as an illustration.

[0103] In this embodiment of the disclosure, the probability density function p(z) c |w c ) is defined as having a central vector and diagonal covariance matrix (to w) c The factorization Gaussian probability distribution obtained by solving this row vector. The center vector... Set to:

[0104] That is, the vector w c The center vector is itself. Factorization of the Gaussian probability density function p(z) c |w c The calculation is as follows, ensuring that it satisfies a Gaussian distribution and follows a factorized Gaussian probability density function, thus modeling the distribution of the prompts to be classified as a Gaussian probability model:

[0105] The Gaussian probability model described above can be understood as the probability distribution of possible representations to be classified. c This indicates that a Gaussian distribution is used for w. c The vector obtained after probability sampling is the c-th candidate probability text embedding vector.

[0106] From p(z) c |w c In this process, C candidate probability text embedding vectors are obtained:

[0107] Where z1, z2, z3, ... z C The following are examples:

[0108] Then, based on the resampling analysis module 133, z c Reparameterization and resampling aim to make the data conform to a normal distribution and smoother. The calculation formula is as follows: the resampling analysis module 133 will calculate z... c Extracted using Gaussian probability partial sampling:

[0109] μ(w c ) is p(z c |w c The mean of ) σ(w) c ) is p(zc |w c The standard deviation of ) and ∈K follows a normal distribution:

[0110] The sample is then sent to the subsequent feature matching module 30 (which includes a similarity calculation module 32). As shown in Figure 4, this embodiment of the present disclosure designs a similarity calculation module 32 based on Gaussian distribution probability hint learning of a large model and cross-entropy.

[0111] After the resampling analysis module 133, the following is obtained: The Cross Entropy Calculation module 31 calculates the image encoder. The output features (e.g., image features or sample image features) and z c The cross entropy between them is then calculated by the similarity calculation module 32 to obtain the final scene result to be judged (output 40, i.e., Results), which can be the target scene or scene prediction information.

[0112] When an image 21 is input into the image encoder 22, the feature g of that image is obtained. d Then, the features g of this image d , respectively with each text feature z c Calculate the cross entropy; there are a total of C text features, where c ranges from 1 to C, and g... d The representation is as follows:

[0113] In order to solve equation (9), z c and g d The vector lengths remain consistent, while g d The vector length can be set using the image encoder.

[0114] Calculate the features g of the image d and each text feature z1, z2, z3, ... z C θ between (d,j) As shown in formula (1) above.

[0115] In the above formula (1), Image features g d The elements in the vector. Text features z c The elements in the vector. The image features g can be calculated. d and text features z c between q(d,c) The value is the Cross Entropy value.

[0116] After obtaining the Cross Entropy value, the class distance value is calculated. The smaller the Cross Entropy value, the closer the text feature vector is to the image feature vector. The probability that image g belongs to class c in text z is calculated as follows:

[0117] θ represents the distance value between the image and the scene category c to be understood. The category with the smallest distance value is the scene category to which the image belongs. (d,c) The smallest category is the scene category to which the image belongs in the judgment. (Calculated...) Given the probability value that image g belongs to text category c, output the location scene information or location information or scene information of the image captured by the video network camera.

[0118] This disclosure provides a point governance method and system based on Gaussian distribution cue learning with cross-entropy of a large model. It utilizes Gaussian distribution probabilistic cue learning to fine-tune the parameters of the large model and introduces a novel pixel-text matching loss calculation method to improve the accuracy of scene category judgment in visual networks. This probabilistic cue learning method allows the model to better understand and process the connection between vision and language, exhibiting high performance and generalization ability.

[0119] To achieve automatic acquisition of camera location information in a visual network, this disclosure presents a Gaussian probability-based prompt learning method based on a large model and a pixel-text matching loss calculation method based on cross-entropy. This enables intelligent understanding, automatic acquisition, and automatic annotation of camera location information in the visual network. The system categorizes the scene types of millions of cameras and pushes relevant resources to cameras belonging to the same scene type. This system solves the location management problem for millions of cameras in a visual network. In other words, this disclosure provides a location management method and system for visual networks based on Gaussian distribution prompt learning using cross-entropy from a large model. This can improve data processing efficiency in visual networks.

[0120] Addressing the requirements and functional architecture of a general large-scale model generation system in intelligent video surveillance, this paper solves the problem of location management for millions or tens of millions of cameras in a video network. By using a method based on the Bray Curtis Distance Gaussian distribution probability hint learning of a large model, the location information of each camera in the video network can be automatically acquired.

[0121] This disclosure belongs to the field of artificial intelligence, specifically the subfields of general large-scale models, cue learning, probabilistic learning, and visual networks. It focuses on the research and application of key technologies for multimodal large-scale models in the field of visual network operation and maintenance.

[0122] Based on the same inventive concept, this disclosure also provides an information processing apparatus, as described in the following embodiments. Since the principle by which this information processing apparatus solves the problem is similar to that of the method embodiments described above, the implementation of this information processing apparatus embodiment can refer to the implementation of the method embodiments described above, and repeated details will not be elaborated further.

[0123] Figure 5 shows a structural block diagram of an information processing apparatus according to an embodiment of the present disclosure. As shown in Figure 5, the information processing apparatus 500 includes a processing unit 510.

[0124] The processing unit 510 is used to obtain the probabilistic text embedding vector.

[0125] The processing unit 510 is also used to obtain image features of the image and to obtain the matching degree between the probabilistic text embedding vector and the image features.

[0126] Processing unit 510 is also used to use the probability text embedding vector with the highest matching degree as the target probability text embedding vector.

[0127] The processing unit 510 is further configured to determine the target prompt template corresponding to the target probability text embedding vector, and determine the target scene corresponding to the image based on the target scene prompt information corresponding to the target prompt template.

[0128] Other aspects of the embodiment shown in Figure 5 can be found in the above embodiments and will not be repeated here.

[0129] Based on the same inventive concept, this disclosure also provides an electronic device, as described in the following embodiments. Since the principle by which this electronic device solves the problem is similar to that of the above-described method embodiments, the implementation of this electronic device embodiment can refer to the implementation of the above-described method embodiments, and repeated details will not be elaborated further.

[0130] It should be noted that the above-mentioned modules / units, as part of a device, can be executed in a computer system such as a set of computer-executable instructions.

[0131] Those skilled in the art will understand that various aspects of this disclosure can be implemented as a system, method, or program product. Therefore, various aspects of this disclosure can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "system."

[0132] The electronic device 600 according to this embodiment of the present disclosure will now be described with reference to FIG6. The electronic device 600 shown in FIG6 is merely an example and should not be construed as limiting the functionality and scope of the embodiments of the present disclosure.

[0133] Electronic device 600 can be a terminal, also known as a terminal device, user equipment (UE), mobile station, mobile terminal, etc. Terminals can be widely used in various scenarios, such as device-to-device (D2D), vehicle-to-everything (V2X) communication, machine-type communication (MTC), Internet of Things (IoT), virtual reality, augmented reality, industrial control, autonomous driving, telemedicine, smart grids, smart furniture, smart offices, smart wearables, smart transportation, smart cities, etc. Terminals can be mobile phones, tablets, computers with wireless transceiver capabilities, wearable devices, vehicles, drones, helicopters, airplanes, ships, robots, robotic arms, smart home devices, etc. The embodiments of this disclosure do not limit the specific technologies or device forms used in the terminals.

[0134] As shown in Figure 6, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, and a bus 630 connecting different system components (including storage unit 620 and processing unit 610).

[0135] The storage unit 620 stores program code that can be executed by the processing unit 610, causing the processing unit 610 to perform the steps described in the "Exemplary Methods" section above according to various exemplary embodiments of this disclosure.

[0136] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0137] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0138] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0139] Electronic device 600 can also communicate with one or more external devices 640 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. As shown, network adapter 660 communicates with other modules of electronic device 600 via bus 630. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0140] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0141] In particular, according to embodiments of this disclosure, the process described above with reference to the flowchart can be implemented as a computer program product, which includes a computer program that, when executed by a processor, implements the above-described information processing method.

[0142] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, which may be a readable signal medium or a readable storage medium. The computer-readable storage medium stores a program product capable of implementing the methods described above. In some possible implementations, various aspects of this disclosure may also be implemented as a program product including program code that, when run on a terminal / network device, causes the terminal / network device to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of this disclosure.

[0143] More specific examples of computer-readable storage media in this disclosure may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] In this disclosure, a computer-readable storage medium may include a data signal propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of transmitting, propagating, or transmitting a program for use by or in connection with an instruction execution system, apparatus, or device.

[0145] Optionally, the program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0146] In practical implementation, program code for performing the operations of this disclosure can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0147] In this disclosure, at least one item can be described as one item or multiple items, and multiple items can be two, three, four, or more items, without limitation. " / " can indicate that the related objects are in an "or" relationship; for example, A / B can mean A or B. "And / or" can be used to describe three relationships between related objects; for example, A and / or B can represent: A alone, A and B simultaneously, and B alone, where A and B can be singular or plural. To facilitate the description of the technical solutions of this application, terms such as "first," "second," "A," or "B" can be used to distinguish technical features with the same or similar functions. These terms do not limit the quantity or execution order. Furthermore, the terms "first," "second," "A," or "B" are not necessarily different. The words “exemplary” or “for example” are used to indicate examples, illustrations, or explanations. Any design described as “exemplary” or “for example” should not be construed as being superior or more advantageous than other design options. The use of words such as “exemplary” or “for example” is intended to present the relevant concepts in a concrete manner to facilitate understanding.

[0148] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0149] Furthermore, although the steps of the method in this disclosure are described in a specific order in the accompanying drawings, this does not require or imply that the steps must be performed in that specific order, or that all the steps shown must be performed to achieve the desired result. Additional or alternative steps may be omitted, multiple steps may be combined into one step, and / or a step may be broken down into multiple steps.

[0150] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, mobile terminal, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0151] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This disclosure is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and examples are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the appended claims.

Claims

1. An information processing method characterized by comprising: The method comprises: obtaining a probability text embedding vector; obtaining an image feature of an image, and obtaining a matching degree between the probability text embedding vector and the image feature; taking the probability text embedding vector with the largest matching degree as a target probability text embedding vector; determining a target prompt template corresponding to the target probability text embedding vector, and determining a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template.

2. The method of claim 1, wherein, The probability distribution embedding vector comprises a first number of probability text embedding vectors, and has a second number of dimensions, each dimension indicating a sampling probability of a corresponding prompt template; each prompt template comprises corresponding scene prompt information.

3. The method of claim 2, wherein, The method comprises: obtaining a first number of candidate probability text embedding vectors, wherein the candidate probability distribution embedding vector has a second number of dimensions, each dimension indicating a candidate sampling probability of a corresponding prompt template; obtaining a sample image feature of a sample image and scene label information of the sample image; obtaining a sample matching degree between each candidate probability text embedding vector and the sample image feature respectively; taking the candidate probability text embedding vector with the largest sample matching degree as a target candidate probability text embedding vector; determining a prompt template corresponding to the target candidate probability text embedding vector and scene prompt information corresponding to the prompt template, to determine scene prediction information corresponding to the sample image; obtaining a difference degree between the scene prediction information of the sample image and the scene label information; adjusting the first number of candidate probability text embedding vectors according to the difference degree, to obtain the first number of probability text embedding vectors.

4. The method of claim 3, wherein, The method comprises: obtaining a first number of initial probability text embedding vectors, wherein the initial probability distribution embedding vector has a second number of dimensions, each dimension indicating an initial probability of a corresponding prompt template; taking the initial probability text embedding vector as a center vector of the corresponding candidate probability text embedding vector; obtaining a diagonal covariance matrix of the initial probability text embedding vector; determining a probability density function of the candidate probability text embedding vector based on the initial probability text embedding vector as a Gaussian probability distribution with the corresponding center vector and the diagonal covariance matrix, to obtain the corresponding candidate probability text embedding vector.

5. The method of claim 4, wherein, The method further comprises: resampling each candidate probability text embedding vector to satisfy a normal distribution.

6. The method of claim 3, wherein, The method comprises: inputting the sample image into a frozen and pre-trained image encoder to obtain the sample image feature.

7. The method according to any one of claims 1 to 6, characterized in that, The method further comprises: obtaining a second number of prompt templates; inputting the second number of prompt templates into a frozen and pre-trained text encoder to obtain a second number of text vectors.

8. The method of claim 1, wherein, The image feature comprises K dimensions, K being a positive integer greater than or equal to 1; wherein a cross-entropy between the probability text embedding vector and the image feature is obtained based on a formula as follows, comprising: where c is a positive integer greater than or equal to 1 and less than or equal to C, C is a number of the number of probability text embeddings, C is a positive integer greater than or equal to 1; q (d,c) represents a cross-entropy between the cth probability text embedding vector and the image feature; a kth dimension representing the image feature, k being a positive integer greater than or equal to 1 and less than or equal to K; the k-th dimension in the c-th probability text embedding vector is represented as the matching degree and the cross entropy are in an inverse correlation relationship.

9. The method of claim 1, wherein, The method further comprises: obtaining the image from an image acquisition device; determining point information of the image acquisition device according to the target scene corresponding to the image.

10. An information processing apparatus, characterized by comprising: The method comprises: a processing unit configured to obtain a probability text embedding vector; The processing unit is further configured to obtain an image feature of the image, and obtain a matching degree between the probability text embedding vector and the image feature. The processing unit is further configured to select a probability text embedding vector with the largest matching degree as a target probability text embedding vector. The processing unit is further configured to determine a target prompt template corresponding to the target probability text embedding vector, and determine a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template.

11. An electronic device, comprising: The processing unit is further configured to obtain an image feature of the image, and obtain a matching degree between the probability text embedding vector and the image feature. The processing unit is further configured to select a probability text embedding vector with the largest matching degree as a target probability text embedding vector. The processing unit is further configured to determine a target prompt template corresponding to the target probability text embedding vector, and determine a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template. The processing unit is further configured to obtain an image feature of the image, and obtain a matching degree between the probability text embedding vector and the image feature. The processing unit is further configured to select a probability text embedding vector with the largest matching degree as a target probability text embedding vector.

12. A computer readable storage medium having stored thereon a computer program, characterized in that, The processing unit is further configured to determine a target prompt template corresponding to the target probability text embedding vector, and determine a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template. The processing unit is further configured to obtain an image feature of the image, and obtain a matching degree between the probability text embedding vector and the image feature. The processing unit is further configured to select a probability text embedding vector with the largest matching degree as a target probability text embedding vector. The processing unit is further configured to determine a target prompt template corresponding to the target probability text embedding vector, and determine a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template. The processing unit is further configured to obtain an image feature of the image, and obtain a matching degree between the probability text embedding vector and the image feature. The processing unit is further configured to select a probability text embedding vector with the largest matching degree as a target probability text embedding vector. The processing unit is further configured to determine a target prompt template corresponding to the target probability text embedding vector, and determine a target scene corresponding to the image according to target scene prompt information corresponding to the target prompt template. The processing unit is further configured to obtain an image feature of

Images