Camera position determination method and apparatus, storage medium, and electronic device

Abstract

The present disclosure provides a camera position determination method and device, computer storage medium and electronic equipment, and relates to the technical field of computer video processing. The method comprises: obtaining a target scene image whose position information is to be determined, and determining an image feature vector of the target scene image based on an image encoder; inputting prompt templates of multiple scene categories into a text encoder to obtain text embedding feature vectors of target scene categories matched with the target scene image; determining a probability distribution between a target sub-vector and the image feature vector to obtain scene feature vectors corresponding to each target scene category; calculating a similarity between the scene feature vectors and the image feature vector; determining the scene feature vector with the largest similarity as a target scene feature vector, and determining position information of a camera corresponding to the target scene image according to a scene category corresponding to the target scene feature vector. This method can improve the efficiency and accuracy of camera position determination.

Description

Technical Field

[0001] This disclosure relates to the field of computer video processing technology, and in particular to a method for determining camera position, a device for determining camera position, a computer storage medium, and an electronic device. Background Technology

[0002] With the rapid development of 5G, big data, and artificial intelligence, video is increasingly becoming an integral part of people's daily work and study. Within video application scenarios, video surveillance is finding increasingly widespread applications. However, existing video surveillance systems lack camera location information, which affects the unified management and operational efficiency of these systems.

[0003] In related technical solutions, the location information of cameras / cameras is usually marked manually one by one. However, the above-mentioned manual marking method is time-consuming and labor-intensive, especially for large-scale video surveillance systems that contain a massive number of cameras. Manually marking each camera is not only inefficient and costly, but the manual confirmation of scene categories is also prone to errors, making it difficult to guarantee accuracy. Summary of the Invention

[0004] This disclosure provides a method for determining camera position, a device for determining camera position, a computer storage medium, and an electronic device, thereby improving the efficiency and accuracy of camera position determination.

[0005] In a first aspect, one embodiment of this disclosure provides a method for determining camera position. The method includes: acquiring a target scene image whose position information needs to be determined; determining an image feature vector for the target scene image based on an image encoder; inputting prompt templates for multiple scene categories into a text encoder to obtain a text embedding feature vector for the target scene category that matches the target scene image; determining a probability distribution between the target sub-vector and the image feature vector for each target sub-vector in the text embedding feature vector, and determining a scene feature vector corresponding to each target scene category based on the probability distribution; calculating the similarity between each scene feature vector and the image feature vector; determining the scene feature vector with the highest similarity as the target scene feature vector, and determining the position information of the camera corresponding to the target scene image according to the scene category corresponding to the target scene feature vector.

[0006] In an optional embodiment of this disclosure, the method further includes: acquiring multiple scene images of multiple scene categories as a training sample set; inputting the training sample set into a pre-trained general large model to obtain text embedding feature vectors of multiple scene images in each scene category; and constructing prompt templates for multiple scene categories based on the text embedding feature vectors of multiple scene images.

[0007] In one optional embodiment of this disclosure, a prompt template for multiple scene categories is constructed based on the text embedding feature vectors of multiple scene images, including: obtaining a preset prompt word length; and constructing prompt templates for multiple scene categories based on the preset prompt word length and the text embedding feature vectors of multiple scene images.

[0008] In an optional embodiment of this disclosure, the text encoder is a Transformer model.

[0009] In an optional embodiment of this disclosure, for each target sub-vector in the text embedding feature vector, the probability distribution between the target sub-vector and the image feature vector is determined to obtain the scene feature vector corresponding to each target scene category. This includes: for each target sub-vector, determining the probability distribution between the target sub-vector and the image feature vector to obtain the initial scene feature vector corresponding to each target scene category; determining the product between the image feature vector and the normal distribution based on the unit vector, and updating the parameters of the initial scene feature vector based on the product and the sum of each target sub-vector in the text embedding feature vector to obtain the scene feature vector corresponding to each target scene category.

[0010] In an optional embodiment of this disclosure, calculating the similarity between each scene feature vector and the image feature vector includes:

[0011] For each first feature sub-vector in the image feature vector, the difference is calculated with each second feature sub-vector in the scene feature vector, and the absolute value of the difference is processed to obtain multiple target values; the number of elements greater than a preset value among the multiple target values ​​is counted; based on the number of elements and the total number of elements, the similarity between each scene feature vector and the image feature vector is calculated; the total number of elements is the sum of the number of first feature sub-vectors in the image feature vector and the number of second feature sub-vectors in the scene feature vector.

[0012] In an optional embodiment of this disclosure, the probability distribution is a Gaussian probability distribution.

[0013] Secondly, one embodiment of this disclosure provides a camera position determination device, which includes: an image acquisition module, used to acquire a target scene image whose position information needs to be determined, and to determine an image feature vector for the target scene image based on an image encoder; a prompt template input module, used to input prompt templates of multiple scene categories into a text encoder to obtain a text embedding feature vector of the target scene category that matches the target scene image; a probability distribution determination module, used to determine the probability distribution between the target sub-vector and the image feature vector for each target sub-vector in the text embedding feature vector, and to determine the scene feature vector corresponding to each target scene category based on the probability distribution; a similarity determination module, used to calculate the similarity between each scene feature vector and the image feature vector; and a camera position determination module, used to determine the scene feature vector with the highest similarity as the target scene feature vector, and to determine the position information of the camera corresponding to the target scene image according to the scene category corresponding to the target scene feature vector.

[0014] In an optional embodiment of this disclosure, the device may further include a model training module and a template construction module. The model training module is used to acquire multiple scene images of multiple scene categories as a training sample set; input the training sample set into a pre-trained general large model to obtain text embedding feature vectors of multiple scene images in each scene category; the template construction module is used to construct prompt templates for multiple scene categories based on the text embedding feature vectors of multiple scene images.

[0015] In an optional embodiment of this disclosure, the template construction module is specifically used to obtain a preset prompt word length; and to construct prompt templates for multiple scene categories based on the preset prompt word length and the text embedding feature vectors of multiple scene images.

[0016] In an optional embodiment of this disclosure, the text encoder is a Transformer model.

[0017] In an optional embodiment of this disclosure, the probability distribution determination module is specifically used to determine the probability distribution between the target sub-vector and the image feature vector for each target sub-vector, thereby obtaining the initial scene feature vector corresponding to each target scene category; determine the product between the image feature vector and the normal distribution based on the unit vector, and update the parameters of the initial scene feature vector based on the product and the sum of each target sub-vector in the text embedding feature vector, thereby obtaining the scene feature vector corresponding to each target scene category.

[0018] In an optional embodiment of this disclosure, the similarity calculation module is specifically used to calculate the difference between each first feature sub-vector in the image feature vector and each second feature sub-vector in the scene feature vector, and to perform absolute value processing on the difference to obtain multiple target values; count the number of elements in the multiple target values ​​that are greater than a preset value; calculate the similarity between each scene feature vector and the image feature vector based on the number of elements and the total number of elements; the total number of elements is the sum of the number of first feature sub-vectors contained in the image feature vector and the number of second feature sub-vectors contained in the scene feature vector.

[0019] In an optional embodiment of this disclosure, the probability distribution is a Gaussian probability distribution.

[0020] Thirdly, one embodiment of this disclosure provides a computer-readable storage medium having a computer program stored thereon that, when executed by a processor, implements the above-described method for determining the camera position.

[0021] Fourthly, one embodiment of this disclosure provides an electronic device, including: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform the above-described method for determining the camera position by executing the executable instructions.

[0022] Fifthly, one embodiment of this disclosure provides a computer program product, including a computer program that is executed by a processor to implement the camera position determination method described above.

[0023] The technical solution disclosed herein has the following beneficial effects:

[0024] The aforementioned method for determining camera positions involves acquiring a target scene image whose position information needs to be determined, and then using an image encoder to determine the image feature vector for the target scene image. Multiple scene category prompt templates are input into a text encoder to obtain text embedding feature vectors for the target scene categories that match the target scene image. For each target sub-vector in the text embedding feature vector, the probability distribution between the target sub-vector and the image feature vector is determined, and the scene feature vector corresponding to each target scene category is determined based on the probability distribution. The similarity between each scene feature vector and the image feature vector is calculated. The scene feature vector with the highest similarity is determined as the target scene feature vector, and the position information of the camera corresponding to the target scene image is determined according to the scene category corresponding to the target scene feature vector. This method converts the image into a text embedding feature vector by learning pre-trained prompt templates and determines the probability distribution between the text embedding feature vector and the image feature vector of the target scene image. It can automatically acquire and label camera locations with unknown position information, thus overcoming the low accuracy and efficiency problems caused by manual labeling in related technical solutions. This improves the efficiency and accuracy of camera position determination and is therefore suitable for managing the position information of millions of cameras in a visual network. Furthermore, the probability distribution-based cue learning method employed in this disclosure enables the model to better understand and process the connection between vision and language, resulting in higher performance and generalization ability.

[0025] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this disclosure. Attached Figure Description

[0026] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this disclosure and, together with the description, serve to explain the principles of this disclosure. It is obvious that the drawings described below are merely some embodiments of this disclosure, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort.

[0027] Figure 1 This illustration schematically shows a flowchart of one of the methods for manually acquiring the location information of each surveillance camera in this exemplary embodiment.

[0028] Figure 2 This diagram illustrates the architecture of a camera position determination system according to one of the exemplary embodiments of the present invention.

[0029] Figure 3 A flowchart illustrating one method for determining the camera position according to this exemplary embodiment is shown;

[0030] Figure 4This schematically illustrates a flowchart of a method for determining the similarity between scene feature vectors and image feature vectors according to one of the exemplary embodiments of the present invention.

[0031] Figure 5 This diagram schematically illustrates one method for determining the position of the entire camera according to this exemplary embodiment.

[0032] Figure 6 This schematic diagram illustrates the structure of a camera position determination device according to one of the exemplary embodiments of the present invention.

[0033] Figure 7 The schematic diagram illustrates the structure of an electronic device according to one of the exemplary embodiments of this invention. Detailed Implementation

[0034] Exemplary embodiments will now be described more fully with reference to the accompanying drawings. However, exemplary embodiments can be implemented in many forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided to make this disclosure more comprehensive and complete, and to fully convey the concept of exemplary embodiments to those skilled in the art. The described features, structures, or characteristics can be combined in any suitable manner in one or more embodiments. In the following description, numerous specific details are provided to give a full understanding of embodiments of this disclosure. However, those skilled in the art will recognize that the technical solutions of this disclosure can be practiced with one or more specific details omitted, or other methods, components, apparatus, steps, etc., can be employed. In other instances, well-known technical solutions are not shown or described in detail to avoid obscuring various aspects of this disclosure.

[0035] Furthermore, the accompanying drawings are merely illustrative of this disclosure and are not necessarily drawn to scale. The same reference numerals in the drawings denote the same or similar parts, and therefore repeated descriptions of them will be omitted. Some block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities may be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] To help those skilled in the art better understand the technical solutions of this disclosure, the relevant content involved in the technical solutions of this disclosure will be introduced below.

[0038] 1) General-purpose large model: The large model, also known as the foundation model (FM), refers to a large-scale and powerful machine learning model. It usually far exceeds the conventional machine learning model in terms of training data, number of model parameters and computing resources. It extracts and learns knowledge from hundreds of millions of corpora or images, and then produces a large model with hundreds of millions of parameters.

[0039] 2) Prompt Learning: This is a method that assists learners in self-guidance and active learning by providing guided information or prompts. It mainly involves fine-tuning the model parameters by adding "prompt information" to the input parameters or changing the downstream task to a text generation task without significantly changing the structure of the pre-trained model, so as to improve the performance and accuracy of the model.

[0040] 3) Probability Learning: Under the premise of conforming to the principle of probability distribution, this study investigates how individuals learn to predict whether an event will occur or not.

[0041] In the context of relevant technologies, with the rapid development of 5G, big data, and artificial intelligence, video is gradually taking over people's daily work and study. Within video application scenarios, video surveillance is also becoming increasingly widespread. For example, 5G has spurred a series of video-based applications, such as smart and safe cities, the internet, autonomous driving, and video surveillance—all applications based on images and video.

[0042] However, in existing video surveillance systems, the location information of video surveillance equipment (such as surveillance cameras) is often missing, affecting the user's overall management of video surveillance and consequently impacting the related services mentioned above. For example, in the field of smart and safe city applications, when a serious incident occurs within the surveillance coverage area, the lack of location information for each camera in the video surveillance system makes it difficult to quickly determine the location of the incident, thus affecting processing efficiency. Similarly, in the field of autonomous driving applications, the missing camera locations prevent the central control processor from coordinating the control of the numerous cameras on the vehicle, thus affecting the normal operation of the vehicle during autonomous driving.

[0043] In relevant technical solutions, the following two methods are mainly used to determine the camera installation location:

[0044] The first method: manually acquiring and marking the camera's installation location information. This will be combined with... Figure 1An exemplary method for obtaining the location information of each surveillance camera in the current video network system through manual means is provided.

[0045] Figure 1 This schematic diagram illustrates an application scenario of a method for manually acquiring the location information of various surveillance cameras in this exemplary embodiment; see reference. Figure 1 As shown, firstly, scene images from cameras in the video network are acquired and input into the video surveillance system. The image information is then manually processed within the system, specifically by manually identifying the scene information within the images. Next, existing tag location information is manually compared with the scene tag information in the image as understood by the human eye, resulting in a tag comparison result. Finally, based on the tag comparison result, the location information of the corresponding camera can be obtained.

[0046] However, the aforementioned method of manually acquiring and marking the location information of surveillance cameras is time-consuming, labor-intensive, and inefficient, especially for current operator video network management platforms, which contain a massive number of front-end cameras (e.g., millions of cameras), requiring enormous manual labor costs. Furthermore, the manual annotation method involves comparing the location information of historically manually annotated tags with the scene tag information in the image as understood by the human eye. This inevitably leads to problems such as errors in the tag location information due to its age and inaccuracies in the scene tag information understood by the human eye, making it difficult to guarantee accuracy.

[0047] The second approach involves using artificial intelligence and deep learning algorithms. While this can address the inefficiency of the manual annotation methods mentioned above, it cannot meet the application requirements of video intelligent analysis tasks, which demand accuracy, scene understanding, low latency, and abstract semantics.

[0048] Therefore, existing methods for determining camera location cannot efficiently and accurately acquire and label the installation location information of each camera in a video surveillance system; nor can they understand camera scene information based on keywords in the installation location information, nor can they locate camera location information with high accuracy based on keywords in the installation location information.

[0049] This disclosure, in consideration of the aforementioned problems, proposes a method for determining camera positions. This method converts images into text embedding feature vectors by learning pre-trained cue templates through cue learning, and determines the probability distribution between the text embedding feature vectors and the image feature vectors of the target scene image. This allows for the automatic acquisition and labeling of camera locations with unknown position information, overcoming the low accuracy and efficiency issues caused by manual labeling in related technical solutions. This improves the efficiency and accuracy of camera position determination, making it suitable for managing the position information of millions of cameras in visual networks. Furthermore, the probability distribution-based cue learning method employed in this disclosure enables the model to better understand and process the connection between vision and language, exhibiting high performance and generalization ability.

[0050] The following will combine Figure 2 The method and apparatus for determining the position of a camera as disclosed herein are applied to... Figure 2 An embodiment of the system architecture in the exemplary application environment shown is illustrated.

[0051] like Figure 2 The diagram illustrates an application scenario provided by an embodiment of this application. In this scenario, a terminal device 201, a server 202, and a camera 203 may be included. Multiple cameras 203 are used to capture video sequences for each scene category, and the scene image is a frame from one of the video sequences.

[0052] Terminal device 201 can be, for example, any device involved in displaying scene images captured by camera 203, such as a mobile phone, tablet computer (PAD), laptop computer, desktop computer, smart TV, smart in-vehicle device, smart wearable device, or aircraft. Terminal device 201 may have a target application installed, which may have functions such as displaying scene images from video surveillance. The application involved in this embodiment can be a software client, or a webpage, mini-program, etc. Server 202 is the server corresponding to the software, webpage, mini-program, etc., and the specific type of client is not limited.

[0053] Server 202 can be a backend server for the target application, providing corresponding backend services, such as camera location determination. It can be a standalone physical server, a server cluster or distributed system composed of multiple physical servers, or a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communication, middleware services, domain name services, security services, i.e., Content Delivery Network (CDN), as well as big data and artificial intelligence platforms, but it is not limited to these.

[0054] It should be noted that the camera position determination method in this embodiment can be executed by either the terminal device 201 or the server 202 alone, or by both the server 202 and the terminal device 201. When executed by either the terminal device 201 or the server 202 alone, the model training and application processes can be implemented by either the terminal device 201 or the server 202 alone. For example, on the terminal device 201, a general large model can be trained using a cue learning method based on multiple scene images of multiple scene types. After adjusting the model parameters of the general large model, the terminal device 201 can use the trained general large model to generate cue templates for multiple scene categories. Alternatively, the server 202 can also execute the above process. When executed jointly by server 202 and terminal device 201, server 202 can train a general-purpose large model and then deploy the trained model to terminal device 201, whereby terminal device 201 determines the camera position. Alternatively, some parts of the model training or application process can be implemented by terminal device 201, while others can be implemented by server 202, with both collaborating to achieve the model training or application process. Specific configurations can be made according to the actual application, and this application does not impose specific limitations here.

[0055] Both server 202 and terminal device 201 may include one or more processors, memory, and I / O interfaces for interaction. Furthermore, server 202 may be configured with a database to store trained model parameters. The memory of server 202 and terminal device 201 may also store program instructions required for execution in the camera position determination method provided in this embodiment. These program instructions, when executed by the processor, can be used to implement the model training process or the camera position determination process provided in this embodiment.

[0056] It should be noted that when the camera position determination method provided in this application embodiment is executed by either server 202 or terminal device 201 alone, the above application scenario may also include only a single device, either server 202 or terminal device 201. Alternatively, server 202 and terminal device 201 may be considered as the same device. Of course, in practical applications, when the camera position determination method provided in this application embodiment is executed by both server 202 and terminal device 201, server 202 and terminal device 201 may also be the same device. That is, server 202 and terminal device 201 may be different functional modules of the same device, or a virtual device virtualized from the same physical device.

[0057] In this embodiment, the terminal device 201 and the server 202 can communicate directly or indirectly through one or more networks 204. The network 204 can be a wired network or a wireless network; for example, the wireless network can be a mobile cellular network or a Wireless-Fidelity (WIFI) network, and of course, it can be other possible networks. This embodiment does not limit the types of networks used. It should be noted that... Figure 2 The examples shown are merely illustrative; in reality, the number of terminal devices and servers is unlimited and is not specifically limited in the embodiments of this application.

[0058] For example, in one exemplary embodiment, server 202 acquires a target scene image whose location information needs to be determined, and determines an image feature vector for the target scene image based on an image encoder; inputs prompt templates for multiple scene categories into a text encoder to obtain a text embedding feature vector for the target scene category that matches the target scene image; for each target sub-vector in the text embedding feature vector, determines the probability distribution between the target sub-vector and the image feature vector, and determines the scene feature vector corresponding to each target scene category based on the probability distribution; calculates the similarity between each scene feature vector and the image feature vector; determines the scene feature vector with the highest similarity as the target scene feature vector, and determines the location information of the camera 203 corresponding to the target scene image according to the scene category corresponding to the target scene feature vector, and displays the location information of the camera 203 on the terminal device 201.

[0059] The following describes the method provided by exemplary embodiments of this application in conjunction with the application scenarios described above and with reference to the accompanying drawings. It should be noted that the above application scenarios are only shown to facilitate understanding of the spirit and principles of this application, and the embodiments of this application are not limited in any way. It should also be noted that the following method can be executed by the aforementioned terminal device or server, or by both the terminal device and the server. Here, the method is specifically shown as being executed by the terminal device or server.

[0060] Figure 3 A flowchart illustrating one method for determining the camera position according to this exemplary embodiment is shown; see also Figure 3 The diagram shown illustrates the training process of a target language model according to an embodiment of this disclosure. This method can be executed by a computer device, which may be... Figure 2 The terminal device 201 or server 202 shown in the figure, the specific implementation process of this method includes the following steps S301-S305:

[0061] Step S301: Obtain the target scene image whose location information is to be determined, and determine the image feature vector for the target scene image based on the image encoder.

[0062] Step S302: Input the prompt templates of multiple scene categories into the text encoder to obtain the text embedding feature vector of the target scene category that matches the target scene image.

[0063] Step S303: For each target sub-vector in the text embedding feature vector, determine the probability distribution between the target sub-vector and the image feature vector, and determine the scene feature vector corresponding to each target scene category based on the probability distribution.

[0064] Step S304: Calculate the similarity between the feature vectors of each scene and the feature vectors of the image.

[0065] Step S305: Determine the scene feature vector with the highest similarity as the target scene feature vector, and determine the position information of the camera corresponding to the target scene image based on the scene category corresponding to the target scene feature vector.

[0066] In some embodiments of this disclosure, the technical solutions involve acquiring a target scene image whose location information is to be determined, and determining an image feature vector for the target scene image based on an image encoder. Multiple scene category prompt templates are input into a text encoder to obtain a text embedding feature vector for the target scene category matching the target scene image. For each target sub-vector in the text embedding feature vector, a probability distribution between the target sub-vector and the image feature vector is determined, and a scene feature vector corresponding to each target scene category is determined based on the probability distribution. The similarity between each scene feature vector and the image feature vector is calculated. The scene feature vector with the highest similarity is determined as the target scene feature vector, and the location information of the camera corresponding to the target scene image is determined according to the scene category corresponding to the target scene feature vector. This method converts the image into a text embedding feature vector by learning pre-trained prompt templates and determines the probability distribution between the text embedding feature vector and the image feature vector of the target scene image. This allows for the automatic acquisition and labeling of camera locations with unknown location information, overcoming the low accuracy and efficiency problems caused by manual labeling in related technical solutions. This improves the efficiency and accuracy of camera location determination and is therefore suitable for managing the location information of millions of cameras in a visual network. Furthermore, the probability distribution-based cue learning method employed in this disclosure enables the model to better understand and process the connection between vision and language, resulting in higher performance and generalization ability.

[0067] The following will describe in conjunction with specific embodiments Figure 3 The specific implementation methods of each step in the illustrated embodiment are described in detail below:

[0068] In step S301, a target scene image with the location information to be determined is acquired, and an image feature vector for the target scene image is determined based on the image encoder.

[0069] Location information refers to the camera's position information, such as its installation location. The installation location is typically related to the application scenario associated with the video surveillance system. For example, in a school setting, camera installation locations usually include classrooms, playgrounds, garbage rooms, elevators, and corridors. For autonomous driving applications, camera installation locations on autonomous vehicles mainly include front-view cameras, surround-view cameras, rear-view cameras, and side-view cameras.

[0070] The target scene image can be one frame or a collection of multiple frames from a video sequence captured by one or more cameras with unknown location information in a video surveillance system. The video sequence frames can be video frames captured by the aforementioned cameras at historical moments or video frames captured by the cameras in real time; this disclosure does not impose any special limitations on this. The image encoder is a component used to process visual information, performing feature extraction on the target scene image to extract useful information from it.

[0071] For example, the target scene image can be input into an image encoder to extract the corresponding image feature vector from the target scene image.

[0072] It should be explained that the image encoder can be any model capable of extracting image feature vectors. For example, the image encoder can be a deep learning model based on a convolutional neural network (CNN), which extracts image features through multiple convolutional and pooling layers in the CNN deep learning model; it can also be an unsupervised learning model based on an autoencoder, which consists of an encoder and a decoder, where the encoder compresses the input image into a low-dimensional code, and the decoder restores the low-dimensional code to the reconstructed image for use in data dimensionality reduction, image feature extraction, etc. Alternatively, recurrent neural networks (RNNs), models based on ViT (Vision Transformer) and CLIP (Contrastive Language–Image Pre-training), NFNet (Normalizer-Free ResNets), etc., can also be used. This disclosure does not impose any special limitations on these models.

[0073] In step S302, prompt templates for multiple scene categories are input into the text encoder to obtain text embedding feature vectors of the target scene category that match the target scene image.

[0074] The scene category refers to different types of scenes under video surveillance. For example, an application scenario may include kitchen scenes, street scenes, garbage room scenes, elevator scenes, construction site scenes, and corridor scenes. The corresponding scene categories can then be divided into kitchen scene category, street scene category, garbage room scene category, elevator scene category, construction site scene category, and corridor scene category.

[0075] For example, multiple scene category prompt templates are used as input to a text encoder to obtain the text feature vector corresponding to each prompt template, thereby obtaining the text embedding feature vector of the target scene category that matches the target scene image. Each prompt template includes text prompt information corresponding to the image scene category.

[0076] In an optional embodiment of this disclosure, the text encoder is a Transformer model.

[0077] For example, the Transformer model introduces a self-attention mechanism, which allows the model to weigh the importance of different positions in the input sequence when generating output, thereby better handling long-distance dependencies.

[0078] Before step S302, which inputs prompt templates for multiple scene categories into the text encoder to obtain text embedding feature vectors for the target scene category that match the target scene image, it is necessary to pre-construct the prompt templates. In this embodiment, prompt templates can be constructed through large-scale model learning, which will be described in detail below with reference to specific embodiments.

[0079] In an optional embodiment of this disclosure, multiple scene images of multiple scene categories are obtained as a training sample set; the training sample set is input into a pre-trained general large model to obtain text embedding feature vectors of multiple scene images in each scene category; based on the text embedding feature vectors of multiple scene images, prompt templates for multiple scene categories are constructed.

[0080] Among them, the aforementioned general large model can be a cue learning model, which adds "cue learning" to the input, changes the image problem to a text generation task, and thus obtains text embedding feature vectors of multiple scene images in each scene category.

[0081] For example, a large number of scene images are acquired and classified according to the scene category of each scene image, thereby obtaining a sample set of multiple scene categories. Through a general large model, multiple scene images in each scene category can be represented by text, thereby obtaining prompt templates for each scene category.

[0082] In an optional embodiment of this disclosure, the step of constructing prompt templates for multiple scene categories based on text embedding feature vectors of multiple scene images includes: obtaining a preset prompt word length; and constructing prompt templates for multiple scene categories based on the preset prompt word length and text embedding feature vectors of multiple scene images.

[0083] The preset prompt word length can be a pre-set length value. Assuming the preset prompt word length is L and the number of scene categories is K, the prompt template P composed of K scene categories is as shown in the following formula (1):

[0084]

[0085] In formula (1), p1, p2, p3, ..., p K These are the prompt templates corresponding to K scene categories; where, when the obtained preset prompt word length is L, the prompt templates p1, p2, p3, ..., p for the K scene categories are... K The following formulas (2) can be used respectively:

[0086]

[0087] In formula (2), p K,L-1 The text representation (i.e., text feature) characterizes the Kth scene category and has a prompt word length of L-1. The meanings of other representations can be understood based on the above analogy.

[0088] Based on P in the above formula (1), the prompt templates for K scene categories are formed.

[0089] In the above approach, the prompting learning model pre-constructs prompt templates for multiple scene categories. This allows the model to better utilize contextual information through prompt information in the input parameters, thereby improving the model's performance, comprehension, and generalization ability.

[0090] For example, after constructing prompt templates for K scene categories, these can be used as input to a text encoder to obtain the text embedding feature vector of the target scene category that matches the target scene image. Assuming the number of target scene categories is C, the text embedding feature vector w of the target scene category that matches the target scene image is as shown in the following formula (3):

[0091]

[0092] In formula (3), C is the number of target scene categories that match the target scene image; w is the text embedding feature vector of the target scene category that matches the target scene image; w1, w2, w2, ..., w CThese are the text embedding feature vectors corresponding to the C target scene categories to be determined.

[0093] Among them, the formulas in the above formula (3) can be represented as shown in the following formula (4):

[0094]

[0095] In formula (2), K is the number of scene categories; w C,K The text embedding feature vector used to characterize the Cth target scene category to be determined and the prompt template of the Kth scene category.

[0096] With the help of the prompt templates for K scene categories, we can obtain the text embedding feature vectors corresponding to c target scene categories to be judged based on the prompt templates for K scene categories.

[0097] In step S303, for each target sub-vector in the text embedding feature vector, the probability distribution between the target sub-vector and the image feature vector is determined, and the scene feature vector corresponding to each target scene category is determined based on the probability distribution.

[0098] Wherein, the target sub-vector can be formula (4) w1, w2, w2, ..., w C Each subvector w in c,k Where 0≤c≤C, 0≤k≤K, that is:

[0099] w 1,1 ,w 1,2 ,w 1,3 ,...,w 1,K ,w 2,1 ,w 2,2 ,w 2,3 ,...,w 2,K ,w 3,1 ,w 3,2 ,w 3,3 ,...,w 3,K ,w c,1 ,w c,2 ,w c,3 ,...,w c,K For each of the above target subvectors w c,k Both can determine the corresponding probability distribution function for generation.

[0100] For example, the probability distribution between the target subvector and the image feature vector is determined to reflect the visual representation of the target scene image.

[0101] In an optional embodiment of this disclosure, the probability distribution described above is a Gaussian probability distribution.

[0102] For example, for each target subvector w c,k The probability distribution function p(z|w) c,k ) is defined as having a center vector μ c,k and the diagonal covariance matrix σ c,k Factorization of Gaussian probability distribution:

[0103] p(z|w c,k )~N(μ c,k ,σ c,k )Formula (5)

[0104] Among them, the center vector μ in formula (5) c,k Set as the following formula (6):

[0105] μ c,k =w c,k ,0≤c≤C,0≤k≤K Formula (6)

[0106] In formula (6), C is the total number of target scene categories that match the target scene image; K is the K prompt templates.

[0107] The diagonal covariance matrix σ in formula (5) c,k Set as the image feature vector g obtained by the image encoder in step S301 d Then, the above formula (5) models the distribution of the prompts to be classified as a Gaussian probability model p(z|w) c,k As shown in formula (7):

[0108]

[0109] In the above formula (7): z is the scene feature vector corresponding to each target scene category, as shown in the following formula (8):

[0110]

[0111] In formula (8), z1, z2, z2, ..., z C The values ​​are shown in the following formula (9):

[0112]

[0113] In formula (9), C is the total number of target scene categories that match the target scene image; K is the K prompt templates.

[0114] In an optional embodiment of this disclosure, for each target sub-vector, the probability distribution between the target sub-vector and the image feature vector is determined to obtain the initial scene feature vector corresponding to each target scene category; the product between the image feature vector and the normal distribution based on the unit vector is determined, and the parameters of the initial scene feature vector are updated based on the product and the sum of each target sub-vector in the text embedding feature vector to obtain the scene feature vector corresponding to each target scene category.

[0115] For each target subvector w c,k Based on the above formulas (5) to (9), the initial scene feature vector z corresponding to each target scene category is determined. The initial scene feature vector obtained above can also be reparameterized by the resampling unit to update the parameters of the initial scene feature vector and obtain the scene feature vector corresponding to each target scene category.

[0116] For example, the above reparameterization process can be performed based on the following formula (10):

[0117]

[0118] In formula (10), z c ′ represents the scene feature vector obtained after reparameterization. Let p(z|w) be the probability distribution function mentioned above. c,k The mean of z; σ is the probability distribution function p(z|w) mentioned above. c,k The standard deviation of ε; ε follows a normal distribution, as shown in the following formula (11):

[0119] Formula (11) for ε~N(0,I)

[0120] In formula (11), I is the identity matrix.

[0121] Through the above embodiments, the scene feature vector z or z′ corresponding to each target scene category can be determined based on the probability distribution function between the target sub-vector and the image feature vector, which can further improve the accuracy of determining the subsequent scene category.

[0122] In step S304, the similarity between the feature vectors of each scene and the feature vectors of the image is calculated.

[0123] For example, after calculating the scene feature vectors and image feature vectors based on the above steps, the similarity between the scene feature vectors and the image feature vectors can be calculated so as to determine the scene category to which the target scene image belongs based on the similarity.

[0124] Figure 4This schematically illustrates a flowchart of a method for determining the similarity between scene feature vectors and image feature vectors according to one of the exemplary embodiments of this invention. Please refer to [link to flowchart illustration]. Figure 4 In an optional embodiment of this disclosure, step S304, which determines the similarity between each scene feature vector and the image feature vector, includes the following steps S401-S403:

[0125] Step S401: For each first feature sub-vector in the image feature vector, calculate the difference with each second feature sub-vector in the scene feature vector, and perform absolute value processing on the difference to obtain multiple target values.

[0126] Step S402: Count the number of elements among multiple target values ​​that are greater than a preset value.

[0127] Step S403: Calculate the similarity between the feature vectors of each scene and the feature vectors of the image based on the number of elements and the total number of elements.

[0128] The total number of elements is the sum of the number of first feature vectors contained in the image feature vector and the number of second feature vectors contained in the scene feature vector.

[0129] For example, based on the above steps, the feature vectors of each scene are obtained (the following will use z in formula (8) as an example for explanation), and the image feature vector g output by the image encoder is calculated. d The Jaccard Distance between z and z is used to calculate the similarity in order to arrive at the final scene category result.

[0130] The specific process described above is as follows:

[0131] Image feature vector g d It can be represented as shown in the following formula (12):

[0132] g d ={g d,1 ,g d,2 ,g d,3 ,...,g d,K} Formula (12)

[0133] First, regarding the image feature vector g d The number of elements ε between the scene feature vector z and the scene feature vector z (d,h) As shown in formula (13):

[0134] ε (d,h) =count(|g d,i -z h,i |>1),1≤i≤K,1≤h≤C Formula (13)

[0135] In formula (13), g d,i -z h,i For image feature vector g d Find the difference between each first feature vector and each second feature vector of the scene feature vector z, and solve for its absolute value |g d,i -z h,i |, The preset value in step S402 is 1 in formula (13); the count function is used to count the number of elements that are greater than the preset value among multiple target values.

[0136] In determining the image feature vector g d The number of elements ε between the scene feature vector z and the scene feature vector z (d,h) Then, the similarity χ between the feature vectors of each scene and the feature vectors of the image can be calculated based on the following formula (14). (d,h) :

[0137]

[0138] In formula (14), the image feature vector g d If the total number of elements in the scene feature vector z is K, then the total number of elements is 2K.

[0139] Through the above embodiments, the image feature vector g is determined based on the number of elements and the total number of elements. d The Jaccard distance between the image and the scene feature vector z is used to accurately calculate the loss of matching image pixels with text, thereby improving the accuracy of determining the location scene category in the visual network.

[0140] In step S305, the scene feature vector with the highest similarity is determined as the target scene feature vector, and the position information of the camera corresponding to the target scene image is determined according to the scene category corresponding to the target scene feature vector.

[0141] For example, the scene feature vector with the highest similarity is determined as the target scene feature vector. That is, based on the above formula (14), the target scene feature vector can be determined based on the following formula (15):

[0142] ξ||(d,h)=max(χ (d,h) ) 1:C Formula (15)

[0143] In formula (15), ξ|(d,h) is the feature vector of the target scene, and the max function is the similarity χ between each scene feature vector and the image feature vector. (d,h) Take the maximum value from the middle.

[0144] Based on the above embodiments, the scene category corresponding to the target scene feature vector is determined as the target scene category corresponding to the target scene image, thereby determining the camera's location information corresponding to the target scene image based on the target scene category. For example, if the target scene category is determined to be a garbage room scene category based on the above embodiments, then the camera that captured the target scene image can be determined to be a camera installed at the location of the garbage room.

[0145] The following will refer to Figure 5 The entire process of determining the camera position according to the exemplary embodiments of the present disclosure will be described in detail.

[0146] In an optional embodiment, a target scene image whose location information needs to be determined is acquired, and the target scene image is input to an image encoder to determine an image feature vector g for the target scene image based on the image encoder. d .

[0147] In an optional embodiment, prompt templates for multiple scene categories trained on a large model are input into a text encoder, and a text embedding feature vector w for the target scene category to be determined is determined based on a deterministic embedding module between the category and the prompt template. Then, for each target sub-vector w in the text embedding feature vector... c,k Determine the target subvector w c,k With image feature vector g d The probability distribution between p(z|w) c,k ).

[0148] Then, through the probability distribution p(z|w) c,k ), determine the scene feature vector z corresponding to each target scene category, referring to Figure 5 It can be seen that it contains z1, z2, z2, ..., z C .

[0149] Finally, calculate the feature vectors z1, z2, z2, ..., z for each scene. C With image feature vector g d The similarity between them. For example, the Jaccard distance from the above embodiments can be used to calculate the similarity so that the scene feature vector with the highest similarity is determined as the target scene feature vector (i.e., Figure 5 The judgment result is used to determine the camera position information corresponding to the target scene image based on the scene category corresponding to the target scene feature vector.

[0150] The final judgment result can form a feedback loop, which can be used to fine-tune the model parameters of the training prompt template based on the loss function between the judgment result and the actual result, so as to improve the accuracy of camera position determination.

[0151] To implement the above-described method for determining camera position, one embodiment of this disclosure provides a device for determining camera position. Figure 6 A schematic diagram of the device for determining the camera position is shown.

[0152] The camera position determination device 600 includes an image acquisition module 601, a prompt template input module 602, a probability distribution determination module 603, a similarity calculation module 604, and a camera position determination module 605.

[0153] The image acquisition module 601 is used to acquire a target scene image whose location information needs to be determined, and to determine the image feature vector for the target scene image based on the image encoder; the prompt template input module 602 is used to input prompt templates for multiple scene categories into the text encoder to obtain the text embedding feature vector of the target scene category that matches the target scene image; the probability distribution determination module 603 is used to determine the probability distribution between the target sub-vector and the image feature vector for each target sub-vector in the text embedding feature vector, and to determine the scene feature vector corresponding to each target scene category based on the probability distribution; the similarity calculation module 604 is used to calculate the similarity between each scene feature vector and the image feature vector; the camera position determination module 605 is used to determine the scene feature vector with the highest similarity as the target scene feature vector, and to determine the position information of the camera corresponding to the target scene image according to the scene category corresponding to the target scene feature vector.

[0154] In an optional embodiment of this disclosure, the device may further include a model training module and a template construction module. The model training module is used to acquire multiple scene images of multiple scene categories as a training sample set; input the training sample set into a pre-trained general large model to obtain text embedding feature vectors of multiple scene images in each scene category; the template construction module is used to construct prompt templates for multiple scene categories based on the text embedding feature vectors of multiple scene images.

[0155] In an optional embodiment of this disclosure, the template construction module is specifically used to obtain a preset prompt word length; and to construct prompt templates for multiple scene categories based on the preset prompt word length and the text embedding feature vectors of multiple scene images.

[0156] In an optional embodiment of this disclosure, the text encoder is a Transformer model.

[0157] In an optional embodiment of this disclosure, the probability distribution determination module 603 is specifically used to determine the probability distribution between the target sub-vector and the image feature vector for each target sub-vector, thereby obtaining the initial scene feature vector corresponding to each target scene category; determine the product between the image feature vector and the normal distribution based on the unit vector, and update the parameters of the initial scene feature vector based on the product and the sum of each target sub-vector in the text embedding feature vector, thereby obtaining the scene feature vector corresponding to each target scene category.

[0158] In an optional embodiment of this disclosure, the similarity calculation module 604 is specifically used to calculate the difference between each first feature sub-vector in the image feature vector and each second feature sub-vector in the scene feature vector, and to perform absolute value processing on the difference to obtain multiple target values; count the number of elements in the multiple target values ​​that are greater than a preset value; calculate the similarity between each scene feature vector and the image feature vector based on the number of elements and the total number of elements; the total number of elements is the sum of the number of first feature sub-vectors contained in the image feature vector and the number of second feature sub-vectors contained in the scene feature vector.

[0159] In an optional embodiment of this disclosure, the probability distribution is a Gaussian probability distribution.

[0160] The camera position determination device 600 provided in this embodiment can execute the technical solution of the camera position determination method in any of the above embodiments. Its implementation principle and beneficial effects are similar to those of the camera position determination method. Please refer to the implementation principle and beneficial effects of the camera position determination method. It will not be repeated here.

[0161] In exemplary embodiments of this disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the methods described above is stored. In some possible embodiments, various aspects of the present invention may also be implemented as a program product comprising program code that, when the program product is run on a terminal device, causes the terminal device to perform the steps of the various exemplary embodiments of the present invention described in the "Exemplary Methods" section above.

[0162] According to embodiments of the present invention, a program product for implementing the above-described method may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0163] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of readable storage media (a non-exhaustive list) include: an electrical connection having one or more wires, a portable disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0164] Computer-readable signal media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable signal medium may also be any readable medium other than a readable storage medium, capable of sending, propagating, or transmitting programs for use by or in conjunction with an instruction execution system, apparatus, or device.

[0165] The program code contained on the readable medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, radio frequency (RF), or any suitable combination thereof.

[0166] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0167] In an exemplary embodiment of this disclosure, an electronic device capable of implementing the above-described method is also provided.

[0168] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: entirely hardware implementations, entirely software implementations (including firmware, microcode, etc.), or implementations combining hardware and software aspects, collectively referred to herein as “circuits,” “modules,” or “systems.”

[0169] The following reference Figure 7 To describe an electronic device 700 according to this embodiment of the present invention. Figure 7 The electronic device 700 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0170] like Figure 7 As shown, the electronic device 700 is manifested in the form of a general-purpose computing device. The components of the electronic device 700 may include, but are not limited to: at least one processing unit 710, at least one storage unit 720, a bus 730 connecting different system components (including storage unit 720 and processing unit 710), and a display unit 740.

[0171] The storage unit stores program code, which can be executed by the processing unit 710 to perform the steps described in the "Exemplary Methods" section of this specification according to various exemplary embodiments of the present invention. For example, the processing unit 710 can perform actions such as... Figure 3 Steps S301 to S305 are shown in the diagram.

[0172] Storage unit 720 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 7201 and / or cache memory 7202, and may further include a read-only memory (ROM) 7203.

[0173] The storage unit 720 may also include a program / utility 7204 having a set (at least one) program module 7205, such program module 7205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0174] Bus 730 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0175] Electronic device 700 can also communicate with one or more external devices 1000 (e.g., keyboard, pointing device, Bluetooth device, etc.), and with one or more devices that enable a user to interact with electronic device 700, and / or with any device that enables electronic device 700 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 750. Furthermore, electronic device 700 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 760. As shown, network adapter 760 communicates with other modules of electronic device 700 via bus 730. It should be understood that, although not shown in the figures, other hardware and / or software modules can be used in conjunction with electronic device 700, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, Redundant Arrays of Independent Disks (RAID) systems, tape drives, and data backup storage systems.

[0176] From the above description of the embodiments, those skilled in the art will readily understand that the exemplary embodiments described herein can be implemented by software or by combining software with necessary hardware. Therefore, the technical solutions according to the embodiments of this disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (such as a CD-ROM, USB flash drive, external hard drive, etc.) or on a network, including several instructions to cause a computing device (such as a personal computer, server, terminal device, or network device, etc.) to execute the methods according to the embodiments of this disclosure.

[0177] Furthermore, the above figures are merely illustrative of the processes included in the method according to exemplary embodiments of the present invention, and are not intended to be limiting. It is readily understood that the processes shown in the above figures do not indicate or limit the temporal order of these processes. Additionally, it is readily understood that these processes may be executed synchronously or asynchronously, for example, in multiple modules.

[0178] It should be noted that although several modules or units for the device used to perform actions have been mentioned in the detailed description above, this division is not mandatory. In fact, according to embodiments of this disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. Conversely, the features and functions of one module or unit described above can be further divided and embodied by multiple modules or units.

[0179] Other embodiments of this disclosure will readily occur to those skilled in the art upon consideration of the specification and practice of the invention disclosed herein. This application is intended to cover any variations, uses, or adaptations of this disclosure that follow the general principles of this disclosure and include common knowledge or customary techniques in the art not disclosed herein. The specification and embodiments are to be considered exemplary only, and the true scope and spirit of this disclosure are indicated by the claims.

[0180] It should be understood that this disclosure is not limited to the precise structures described above and shown in the accompanying drawings, and various modifications and changes can be made without departing from its scope. The scope of this disclosure is defined only by the appended claims.

Claims

1. A method for determining the position of a camera, characterized in that, include: Acquire a target scene image whose location information is to be determined, and determine an image feature vector for the target scene image based on an image encoder; Multiple scene category prompt templates are input into the text encoder to obtain the text embedding feature vector of the target scene category that matches the target scene image; For each target sub-vector in the text embedding feature vector, determine the probability distribution between the target sub-vector and the image feature vector, and determine the scene feature vector corresponding to each target scene category based on the probability distribution; Calculate the similarity between each scene feature vector and the image feature vector; The scene feature vector with the highest similarity is determined as the target scene feature vector, and the position information of the camera corresponding to the target scene image is determined according to the scene category corresponding to the target scene feature vector.

2. The method according to claim 1, characterized in that, The method further includes: Obtain multiple scene images from multiple scene categories as a training sample set; The training sample set is input into a pre-trained general large model to obtain text embedding feature vectors of multiple scene images in each scene category; Based on the text embedding feature vectors of the multiple scene images, prompt templates for the multiple scene categories are constructed.

3. The method according to claim 2, characterized in that, The construction of prompt templates for the multiple scene categories based on the text embedding feature vectors of the multiple scene images includes: Get the preset prompt word length; Based on the preset prompt word length and the text embedding feature vectors of the multiple scene images, prompt templates for the multiple scene categories are constructed.

4. The method according to claim 1, characterized in that, The text encoder is a Transformer model.

5. The method according to claim 1, characterized in that, For each target sub-vector in the text embedding feature vector, determining the probability distribution between the target sub-vector and the image feature vector to obtain the scene feature vector corresponding to each target scene category includes: For each target sub-vector, determine the probability distribution between the target sub-vector and the image feature vector to obtain the initial scene feature vector corresponding to each target scene category; The product between the image feature vector and the normal distribution based on unit vectors is determined, and the parameters of the initial scene feature vector are updated based on the sum of the product and each target sub-vector in the text embedding feature vector to obtain the scene feature vector corresponding to each target scene category.

6. The method according to claim 1, characterized in that, The calculation of the similarity between each scene feature vector and the image feature vector includes: For each first feature sub-vector in the image feature vector, the difference is calculated with each second feature sub-vector in the scene feature vector, and the absolute value of the difference is processed to obtain multiple target values; Count the number of elements among the multiple target values ​​that are greater than a preset value; The similarity between each scene feature vector and the image feature vector is calculated based on the number of elements and the total number of elements; the total number of elements is the sum of the number of first feature sub-vectors contained in the image feature vector and the number of second feature sub-vectors contained in the scene feature vector.

7. The method according to claim 1, characterized in that, The probability distribution is a Gaussian probability distribution.

8. A device for determining the position of a camera, characterized in that, The device includes: The image acquisition module is used to acquire a target scene image whose location information needs to be determined, and to determine the image feature vector for the target scene image based on the image encoder; The prompt template input module is used to input prompt templates of multiple scene categories into the text encoder to obtain the text embedding feature vector of the target scene category that matches the target scene image; The probability distribution determination module is used to determine the probability distribution between the target sub-vector and the image feature vector for each target sub-vector in the text embedding feature vector, and to determine the scene feature vector corresponding to each target scene category based on the probability distribution; The similarity calculation module is used to calculate the similarity between each scene feature vector and the image feature vector; The camera position determination module is used to determine the scene feature vector with the highest similarity as the target scene feature vector, and to determine the position information of the camera corresponding to the target scene image according to the scene category corresponding to the target scene feature vector.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by the processor, it implements the method for determining the camera position as described in any one of claims 1 to 7.

10. An electronic device, characterized in that, include: processor; as well as Memory for storing the executable instructions of the processor; The processor is configured to execute the camera position determination method according to any one of claims 1 to 7 by executing the executable instructions.

Images