Point location information identification method and device, equipment and medium

Abstract

The embodiment of the invention provides a point location information identification method and device, equipment and a medium, which are used for solving the problem that automatic acquisition of point location information of each camera cannot be realized in the prior art. The method comprises the following steps: processing an image of to-be-determined point location information and a preset prompt template corresponding to each piece of point location information based on a point location identification model to obtain an image feature vector corresponding to the image and a sample scene feature vector corresponding to each preset target sample scene category; and according to the similarity between the image feature vector and each sample scene feature vector, the point location information of the camera corresponding to the image is determined, so that automatic acquisition of the point location information of each camera in the articulated naturality web is realized, and the point location identification efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and in particular to a point information recognition method, device, equipment and medium. Background Art

[0002] With the rapid development of 5G, big data and artificial intelligence, a large number of video-oriented applications have emerged, such as smart and safe cities, the Internet, driverless driving, video surveillance and other image and video-based application scenarios. Therefore, image and video-related applications will become one of the main sources of incremental traffic in the 5G and post-5G eras. In current video application scenarios, especially in video surveillance scenarios, the location information of the monitoring cameras is missing. For example, there are 40 cameras in a building, but the location information of these cameras, such as stairs, corridors, meeting rooms, kitchens, etc., cannot be obtained. At present, the location information of the cameras needs to be determined manually.

[0003] Specifically, the process of obtaining the camera point information in the current video surveillance system is as follows: Figure 1 As shown, the image input module transmits the surveillance video collected by the camera in the visual network of the input system to the manual processing module. The manual processing module manually processes the image in the surveillance video by watching the personnel, understands the point information in the image based on the human eye, and manually marks the point information for the camera one by one. However, there are about 30 million cameras monitored by the front end of the operator's visual network management platform. The method of manually watching the surveillance video to determine the point information of the camera requires huge manual investment and is very time-consuming and laborious. In addition, for some of the cameras that carry historical point information in the visual network, since some of the historical point information of these cameras is manually marked in history, there are certain errors in the changes of the scene over time, which need to be corrected. Therefore, in the current video surveillance system, the point information comparison module manually compares the existing historical point information with the point information in the current manually understood image, determines whether the historical point information carried in the collected camera is correct, obtains the point information comparison result, and determines the point information of the camera according to the point information comparison result; however, the point information comparison module in the current video surveillance system needs to be manually compared, which is a huge workload.

[0004] Therefore, in the current video surveillance system, it is impossible to automatically obtain the point information of each camera when determining the point information of the camera, which requires a lot of manpower. Summary of the invention

[0005] The embodiments of the present application provide a method, device, equipment and medium for identifying point information, so as to solve the problem that the point information of each camera cannot be automatically acquired in the prior art.

[0006] In a first aspect, an embodiment of the present application provides a method for identifying point information, the method comprising:

[0007] The image of the point information to be determined and each preset prompt template are input into the point recognition model, and the image feature vector of the image is determined based on the image encoder in the point recognition model; based on the text encoder in the point recognition model and the preset multiple target sample scene categories, the text embedding feature vector corresponding to each preset target sample scene category is determined based on each prompt template, and the Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain the sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes the corresponding point information;

[0008] According to the similarity between the image feature vector and the feature vector of each sample scene, the point information of the camera corresponding to the image is determined.

[0009] In a second aspect, an embodiment of the present application further provides a point information recognition device, the device comprising:

[0010] A processing module is used to input an image of the point information to be determined and each preset prompt template into a point recognition model, and determine an image feature vector of the image based on an image encoder in the point recognition model; based on a text encoder in the point recognition model and a plurality of preset target sample scene categories, determine a text embedding feature vector corresponding to each preset target sample scene category based on each prompt template, and a Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain a sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes corresponding point information;

[0011] The determination module is used to determine the point information of the camera corresponding to the image according to the similarity between the image feature vector and the feature vector of each sample scene.

[0012] In a third aspect, an embodiment of the present application further provides an electronic device, which includes at least a processor and a memory, and the processor is used to implement the steps of the point information identification method as described in any one of the above items when executing a computer program stored in the memory.

[0013] In a fourth aspect, an embodiment of the present application further provides a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the steps of the point information identification method as described in any one of the above items.

[0014] In an embodiment of the present application, an image of point information to be determined and a preset prompt template corresponding to each point information are processed based on a point recognition model to obtain an image feature vector corresponding to the image and a sample scene feature vector corresponding to each preset target sample scene category. According to the similarity between the image feature vector and each sample scene feature vector, the point information of the camera corresponding to the image is determined, thereby realizing automatic acquisition of the point information of each camera in the visual network and improving the efficiency of point recognition. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0016] Figure 1 A schematic diagram of the process of obtaining the point information of the camera in the current video surveillance system;

[0017] Figure 2 A schematic diagram of a point information recognition process provided in an embodiment of the present application;

[0018] Figure 3 A schematic diagram of the structure of a point recognition model provided in an embodiment of the present application;

[0019] Figure 4 A schematic diagram of the structure of a point information identification device provided in an embodiment of the present application;

[0020] Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0022] The embodiments of the present application provide a method, apparatus, device and medium for identifying point information. In the method, an image of point information to be determined and each preset prompt template are input into a point recognition model, and an image feature vector of the image is determined based on an image encoder in the point recognition model; a text embedding feature vector corresponding to each preset sample scene category is determined based on a text encoder in the point recognition model and a plurality of preset sample scene categories, and a Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain a sample scene feature vector corresponding to each sample scene category; based on the similarity between the image feature vector and each sample scene feature vector, the point information of the camera corresponding to the image is determined, thereby realizing automatic acquisition of the point information of each camera in the visual network.

[0023] Embodiment 1:

[0024] Figure 2 A schematic diagram of a point information recognition process provided in an embodiment of the present application, the process includes:

[0025] S201: Input an image of the point information to be determined and each preset prompt template into the point recognition model, and determine the image feature vector of the image based on the image encoder in the point recognition model; based on the text encoder in the point recognition model and a plurality of preset target sample scene categories, determine the text embedding feature vector corresponding to each preset target sample scene category based on each prompt template, and process the text embedding feature vector by the Gaussian distribution probability prompt learning module in the point recognition model to obtain the sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes the corresponding point information.

[0026] The point information recognition method provided in the embodiment of the present application is applied to an electronic device, which may be a personal computer (PC), a server, etc.

[0027] An image captured by a camera whose point information is to be determined is obtained, and the image is used as the image whose point information is to be determined. In the embodiment of the present application, the point information of the image is identified to determine the point information of the camera that captured the image.

[0028] The point information may be scene information, specifically the installation location of the camera, such as a conference room, kitchen, staircase, playground, cinema, etc.

[0029] Obtain a preset prompt template. In one possible implementation, the preset prompt template may be input by the user according to the format requirements of the input data of the point recognition model; in another possible implementation, the preset prompt template may be pre-saved in the electronic device according to the format requirements of the input data of the point recognition model; in another possible implementation, the preset prompt template may be a prompt template that meets the format requirements of the input data of the point recognition model based on the prompt words input by the user and obtained according to the preset splicing rules. No specific restrictions are made here. In the embodiment of the present application, there are generally multiple preset prompt templates, and the multiple preset prompt templates generally include prompt templates corresponding to all possible point information that can be recognized by the point recognition model, and each preset prompt template includes the point information corresponding to the prompt template, or each prompt template contains prompt information corresponding to the point information, that is, each preset prompt template can correspond to different scene information.

[0030] After obtaining an image of the point information to be determined and a plurality of preset prompt templates, the image is input into an image encoder in the point recognition model, the image of the point information to be determined is processed by the image encoder, and the image encoder outputs an image feature vector of the image.

[0031] Each preset prompt template is input into the text encoder in the point recognition model. For example, each preset prompt template is input into the text encoder in the point recognition model in sequence, or each preset prompt template is arranged in sequence, and each arranged prompt template is input into the text encoder in the point recognition model; after each preset prompt template is input into the text encoder, the text encoder processes each preset prompt template and multiple preset target sample scene categories. Assuming that the number of preset prompt templates is k and the number of preset target sample scene categories is c, a feature vector with c rows and k columns is obtained after processing by the text encoder, wherein each row of feature vectors corresponds to a preset target sample scene category, and each row of feature vectors is a text embedding feature vector corresponding to the target sample scene category to which the row of feature vectors belongs. Among them, for each text embedding feature vector, the elements in the text embedding feature vector sequentially represent the association relationship between the target sample scene category corresponding to the text embedding feature vector and the first to kth prompt templates.

[0032] Specifically, after being processed by the text encoder, the feature vector with c rows and k columns can satisfy the following formula: in w c is the eigenvector with c rows and k columns, w 1 、w 2 ,……,w c′is the text embedding feature vector corresponding to the target sample scene category to which the feature vector of this row belongs, c is the number of preset target sample scene categories, and c′=c, k is the number of preset prompt templates, that is, k represents k types of prompt templates.

[0033] Among them, since the preset multiple prompt templates generally include prompt templates corresponding to all possible point information, and the scene categories to be judged in different real scenes may be different, for example, there are no scene categories such as playgrounds and conference rooms in the area included in location A, so when determining the point information, the scene categories within the area included in location A can be identified; for example, the electronic device stores the location information of different real scenes and the preset multiple target sample scene categories corresponding to each location information, so in the embodiment of the present application, while obtaining the image captured by the camera of the point information to be determined, the location of the camera is obtained. According to the location of the camera and the preset multiple target sample scene categories corresponding to each location information stored in the electronic device, the preset multiple target sample scene categories corresponding to the current real scene (i.e., the location of the above-mentioned camera) are obtained, and the preset multiple target sample scene categories are combined when processing the preset multiple prompt templates through the text encoder, so as to facilitate the determination of the camera point information corresponding to the image of the point information to be determined in the scene category to be determined required for the current real scene, and reduce the misjudgment of the point information of the camera as not belonging to the scene category to be determined required for the current real scene, thereby improving the accuracy of point information recognition. It can be understood that the preset multiple target sample scene categories in the embodiment of the present application are usually a subset of the preset multiple prompt templates.

[0034] In addition, the point recognition model can be a pre-trained large model, which can identify the point information of the camera corresponding to the image of the point information to be determined based on the prompt template including the corresponding point information and the image of the point information to be determined. The point recognition model includes an image encoder, a text encoder, and a Gaussian distribution probability prompt learning module.

[0035] Specifically, the image encoder of the point recognition model is used to determine the image feature vector of the image according to the image input to the image encoder, and input the obtained image feature vector into the Gaussian distribution probability prompt learning module.

[0036] The text encoder of the point recognition model is used to determine the text embedding feature vector corresponding to each preset target sample scene category according to the preset prompt template input to the text encoder and the preset multiple target sample scene categories, and input each obtained text embedding feature vector into the Gaussian distribution probability prompt learning module. In this embodiment of the present application, the text encoder is a transformer encoder.

[0037] The Gaussian distribution probability prompt learning module of the point recognition model is used to determine the sample scene feature vector corresponding to each target sample scene category based on the image feature vector input to the Gaussian distribution probability prompt learning module and the text embedding feature vector corresponding to each target sample scene category.

[0038] Among them, in the embodiment of the present application, the Gaussian distribution probability prompt learning module is a Box-Muller algorithm module, and the Box-Muller algorithm is used to process the data input into the module in the Gaussian distribution probability prompt learning module.

[0039] For example, the image feature vector and the text embedding feature vector corresponding to each target sample scene category are input into the Gaussian distribution probability prompt learning module in the point recognition model. The Gaussian distribution probability prompt learning module uses the Box-Muller algorithm to clean each text embedding feature vector and image feature vector, so that each text embedding feature vector and image feature vector obeys the factorized Gaussian probability distribution, and obtains the sample scene feature vector corresponding to each target sample scene category, thereby concentrating the range of the feature vector of each scene category near the center point of the feature vector of the scene category, and increasing the distance between the feature vectors belonging to different scene categories, so as to facilitate the distinction of the feature vectors belonging to different scene categories, thereby facilitating the subsequent matching of the image feature vector with the sample scene feature vectors corresponding to different target sample scene categories.

[0040] Among them, the sample scene feature vectors corresponding to the preset multiple target sample scene categories can satisfy the following formula: in z c is the sample scene feature vector corresponding to the preset multiple target sample scene categories, z 1 、z 2 ,……,z c′ The sample scene feature vector corresponding to each target sample scene category.

[0041] In a possible implementation, after obtaining the sample scene feature vector corresponding to each target sample scene category, the Gaussian distribution probability prompt learning module includes but is not limited to re-parameterizing each sample scene feature vector, which is not specifically limited here. The re-parameterization process is a prior art and will not be described in detail here.

[0042] S202: Determine the point information of the camera corresponding to the image according to the similarity between the image feature vector and the feature vector of each sample scene.

[0043] It is understandable that the point recognition model also includes a similarity calculation module and an output module. Specifically, the similarity calculation module of the point recognition model is used to determine the similarity between the image feature vector and each sample scene feature vector using a similarity algorithm based on the image feature vector and each sample scene feature vector; and based on the similarity between the image feature vector and each sample scene feature vector, determine the point information of the camera corresponding to the image of the point information to be determined. And input the obtained point information of the camera corresponding to the image of the point information to be determined into the output module. Among them, the similarity algorithm includes but is not limited to the existing multiple similarity algorithms.

[0044] For example, in the similarity calculation module in the point recognition model, the sample scene feature vector that meets the similarity condition is determined based on the similarity between the image feature vector and each sample scene feature vector, and the sample scene feature vector that meets the similarity condition is determined as the target sample scene feature vector; based on the target sample scene feature vector, the target sample scene category corresponding to the target sample scene feature vector is determined as the point information of the camera corresponding to the image.

[0045] In one possible implementation, determining the sample scene feature vector that meets the similarity condition may be determining the sample scene feature vector whose similarity with the image feature vector is greater than a preset similarity threshold, and determining the number of sample scene feature vectors whose similarity is greater than the preset similarity threshold, comparing the number with the set output number, and if the number is greater than the set output number, sorting the above-mentioned sample scene feature vectors whose similarity is greater than the preset similarity threshold in descending order according to the size of the similarity value, and determining the sample scene feature vectors before the set output number after sorting as the sample scene feature vectors that meet the similarity condition and output them in sequence, that is, determining the set output number of sample scene feature vectors with larger similarities as the sample scene feature vectors that meet the similarity condition and output them in sequence, so that the user can select the point information corresponding to the camera according to the sample scene feature vector that meets the similarity condition; if the number is not greater than the set output number, all the above-mentioned sample scene feature vectors whose similarity is greater than the preset similarity threshold are determined as sample scene feature vectors that meet the similarity condition.

[0046] In addition, after the position information of the camera corresponding to the image is input into the output module, the output module of the position recognition model is used to output the position information of the camera corresponding to the image whose position information is to be determined. In the embodiment of the present application, after the position information of the camera is output, the electronic device can mark the camera according to the recognized position information of the camera, thereby realizing automatic marking of the camera point information.

[0047] In an embodiment of the present application, the point information in the image is identified by a point recognition model based on the image and a preset prompt template, which allows the point recognition model to better understand and process the connection between vision and language, has high performance and generalization capabilities, and realizes the automatic acquisition of the point information of each camera in the visual network; and the camera can be labeled according to the identified point information of the camera, thereby realizing automatic labeling of the camera point information.

[0048] Embodiment 2:

[0049] On the basis of the above-mentioned embodiment, in the embodiment of the present application, according to the similarity between the image feature vector and each prompt template feature vector, the point information of the camera corresponding to the image is determined, including:

[0050] Determine the similarity between the image feature vector and each sample scene feature vector;

[0051] The sample scene feature vector corresponding to the maximum similarity value is determined as the target sample scene feature vector, and the target sample scene category corresponding to the target sample scene feature vector is determined as the point information of the camera corresponding to the image.

[0052] Based on the image feature vector and each sample scene feature vector, a similarity algorithm can be used to determine the similarity between the image feature vector and each sample scene feature vector, wherein the similarity algorithm includes but is not limited to the Euclidean distance algorithm, the cosine similarity algorithm, the Minkowski distance algorithm, and the like.

[0053] After obtaining the similarity between the image feature vector and each sample scene feature vector, the maximum similarity value among the similarities between the image feature vector and each sample scene feature vector is determined according to the similarity between the image feature vector and each sample scene feature vector, and the sample scene feature vector corresponding to the maximum similarity value is determined.

[0054] For example, the maximum similarity between the image feature vector and the sample scene feature vector satisfies the following formula:

[0055] τ(d c |(i, z))=max(ε(c, a)) 1：c

[0056] Among them, τ(dc |(i, z)) is the maximum similarity between the image feature vector and the sample scene feature vector, that is, the maximum probability that the image belongs to the target sample scene category; c is the total number of target sample scene categories; (1:c) means from the first target sample scene category to the cth target sample scene category, that is, traversing all target sample scene categories; i represents the image feature vector, and z represents the sample scene feature vector.

[0057] Since the similarity between the image feature vector and the sample scene feature vector represents the probability that the image belongs to the target sample scene category corresponding to the sample scene feature vector, and the higher the similarity, the higher the probability that the image belongs to the target sample scene category corresponding to the sample scene feature vector, and the closer the point information of the image is to the target sample scene category, the sample scene feature vector corresponding to the maximum similarity value is determined as the target sample scene feature vector, and the target sample scene category corresponding to the target sample scene feature vector is determined, and the target sample scene category corresponding to the target sample scene feature vector is determined as the point information of the camera corresponding to the image, that is, the target sample scene category corresponding to the maximum probability of the target sample scene category to which the image belongs is determined as the point information of the camera corresponding to the image.

[0058] Embodiment 3:

[0059] On the basis of the above embodiments, in the embodiment of the present application, determining the similarity between the image feature vector and each sample scene feature vector includes:

[0060] The Minkowski distance algorithm is used to determine the corresponding Minkowski distance between the image feature vector and each sample scene feature vector;

[0061] The Minkowski distance between the image feature vector and each sample scene feature vector is used as the similarity between the image feature vector and each sample scene feature vector.

[0062] It is understandable that the point recognition model also includes a Minkowski distance calculation module. Specifically, the Minkowski distance calculation module of the point recognition model is used to determine the Minkowski distance between the image feature vector and the sample scene feature vector corresponding to each target sample scene category, using the Minkowski distance algorithm according to the image feature vector input to the Minkowski distance calculation module and the sample scene feature vector.

[0063] For example, the image feature vector and each sample scene feature vector are input into the Minkowski distance calculation module in the point recognition model. In the Minkowski distance calculation module, for each sample scene feature vector, the Minkowski distance algorithm is used according to the image feature vector and the sample scene feature vector to determine the corresponding Minkowski distance between the image feature vector and the sample scene feature vector.

[0064] Among them, the Minkowski distance between the image feature vector and each sample scene feature vector satisfies the following formula:

[0065]

[0066] Among them, ε(c,a) is the image feature vector i a With each sample scene feature vector z c The corresponding Minkowski distance between is the image feature vector i a The vector representation of is the sample scene feature vector z c is a vector representation of , and q is a preset parameter.

[0067] After obtaining the Minkowski distance between the image feature vector and each sample scene feature vector, for each sample scene feature vector, the Minkowski distance between the image feature vector and the sample scene feature vector is determined as the similarity between the image feature vector and the sample scene feature vector.

[0068] On the basis of the above embodiments, the structure of the point recognition model provided in the embodiment of the present application is as follows: Figure 3 As shown in the figure, the point recognition model includes an image encoder Image Encoder, a text encoder Text Encoder, a Gaussian distribution probability prompt learning module Gaussian distribution probability prompt learning module, a Minkowski Distance Calculation module Minkowski Distance Calculation module, a Similar Calculation module and an output module Result module.

[0069] For example, in the point recognition model, the image encoder inputs the processed image feature vector into the Gaussian distribution probability prompt learning module and the Minkowski distance calculation module; the text encoder inputs each text embedding feature vector obtained by processing into the Gaussian distribution probability prompt learning module; the Gaussian distribution probability prompt learning module inputs each sample scene feature vector obtained after processing the image feature vector and each text embedding feature vector into the Minkowski distance calculation module; the Minkowski distance calculation module can determine the corresponding Minkowski distance between the image feature vector and each sample feature vector according to the input image feature vector and each sample feature vector, and input the calculated Minkowski distance into the similarity calculation module; the similarity calculation module determines the corresponding similarity between the image feature vector and each sample feature vector according to the corresponding Minkowski distance between the image feature vector and each sample feature vector, and determines the sample feature vector corresponding to the maximum similarity, determines the point information of the camera corresponding to the image according to the target sample scene category corresponding to the sample feature vector, and inputs the point information of the camera corresponding to the image into the output module; the output module is used to output the point information of the camera corresponding to the image.

[0070] In an embodiment of the present application, by introducing a method of calculating pixel-text matching loss, that is, determining the Minkowski distance based on the image feature vector and the sample scene feature vector corresponding to each target sample scene category, the adverse effects of uncertainty can be reduced, that is, the fluctuations generated by the model when recognizing different scenes can be reduced.

[0071] Embodiment 4:

[0072] In order to make the prompt template meet the format requirements of the input data of the point recognition model, based on the above embodiments, in the embodiment of the present application, before the preset prompt template is input into the point recognition model, it includes:

[0073] For each prompt template, a preset number of prompt words are input; wherein the prompt words are prompt information corresponding to the point information; and a corresponding prompt template is determined according to the preset number of prompt words and a preset basic prompt template.

[0074] For each prompt template, a preset number of prompt words input by the user are obtained, wherein each prompt word is prompt information corresponding to the point information. For example, the prompt word can be text prompt information corresponding to the point information, such as kitchen, conference room, cinema, playground, etc.

[0075] In a possible implementation, based on the preset number of prompt words obtained, the preset number of prompt words are transformed into prompt templates that meet the format requirements of the point recognition model input data according to a preset basic prompt template, and the prompt templates corresponding to the preset number of prompt words are obtained.

[0076] For example, the preset basic prompt template satisfies the following formula: where p k Represents any prompt template, and L-1 is the preset number.

[0077] According to the above preset basic prompt template, each prompt template can be obtained as follows where p 1 、p 2 ,……,p k After obtaining each prompt template, in the embodiment of the present application, each prompt template can be obtained according to The method determines the preset multiple prompt templates in the text encoder of the input point recognition model.

[0078] In the embodiment of the present application, multiple input prompt words are processed according to a preset basic prompt template to obtain a corresponding prompt template, so that the prompt template can meet the format requirements of the input data of the point recognition model.

[0079] Embodiment 5:

[0080] Based on the same technical concept, on the basis of the above embodiments, the present application provides a point information recognition device, Figure 4 A schematic diagram of the structure of a point information identification device provided in an embodiment of the present application is shown in FIG. Figure 4 As shown, the device comprises:

[0081] The processing module 401 is used to input the image of the point information to be determined and each preset prompt template into the point recognition model, and determine the image feature vector of the image based on the image encoder in the point recognition model; based on the text encoder in the point recognition model and the preset multiple target sample scene categories, determine the text embedding feature vector corresponding to each preset target sample scene category based on each prompt template, and the Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain the sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes the corresponding point information;

[0082] The determination module 402 is used to determine the point information of the camera corresponding to the image according to the similarity between the image feature vector and the feature vector of each sample scene.

[0083] In one possible implementation, determination module 402 is specifically used to determine the similarity between the image feature vector and each sample scene feature vector; the sample scene feature vector corresponding to the maximum similarity value is determined as the target sample scene feature vector, and the target sample scene category corresponding to the target sample scene feature vector is determined as the point information of the camera corresponding to the image.

[0084] In a possible implementation, the determination module 402 is specifically used to use the Minkowski distance algorithm to determine the corresponding Minkowski distance between the image feature vector and each sample scene feature vector; and use the corresponding Minkowski distance between the image feature vector and each sample scene feature vector as the similarity between the image feature vector and each sample scene feature vector.

[0085] In a possible implementation, the Gaussian distribution probability prompt learning module is a Box-Muller algorithm module.

[0086] In a possible implementation, the text encoder is a Transformer encoder.

[0087] In a possible implementation, the processing module 401 is specifically used to obtain a preset number of prompt words for each prompt template before inputting each preset prompt template into the point recognition model; wherein the prompt words are prompt information corresponding to the point information; and determine the corresponding prompt template based on the preset number of prompt words and the preset basic prompt template.

[0088] Embodiment 6:

[0089] Based on the same technical concept, the present application also provides an electronic device, Figure 5 A schematic diagram of the structure of an electronic device provided in an embodiment of the present application is shown in FIG. Figure 5 As shown, it includes: a processor 501, a communication interface 502, a memory 503 and a communication bus 504, wherein the processor 501, the communication interface 502, and the memory 503 communicate with each other through the communication bus 504;

[0090] The memory 503 stores a computer program. When the program is executed by the processor 501, the processor 501 performs the following steps:

[0091] The image of the point information to be determined and each preset prompt template are input into the point recognition model, and the image feature vector of the image is determined based on the image encoder in the point recognition model; based on the text encoder in the point recognition model and the preset multiple target sample scene categories, the text embedding feature vector corresponding to each preset target sample scene category is determined based on each prompt template, and the Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain the sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes the corresponding point information;

[0092] According to the similarity between the image feature vector and the feature vector of each sample scene, the point information of the camera corresponding to the image is determined.

[0093] In one possible implementation, the processor 501 is specifically used to determine the similarity between the image feature vector and each sample scene feature vector; determine the sample scene feature vector corresponding to the maximum similarity value as the target sample scene feature vector, and determine the target sample scene category corresponding to the target sample scene feature vector as the point information of the camera corresponding to the image.

[0094] In a possible implementation, the processor 501 is specifically configured to use a Minkowski distance algorithm to determine the Minkowski distance corresponding to the image feature vector and each sample scene feature vector; and use the Minkowski distance corresponding to the image feature vector and each sample scene feature vector as the similarity corresponding to the image feature vector and each sample scene feature vector.

[0095] In a possible implementation, the Gaussian distribution probability prompt learning module is a Box-Muller algorithm module.

[0096] In a possible implementation, the text encoder is a Transformer encoder.

[0097] In a possible implementation, the processor 501 is specifically used to obtain a preset number of prompt words input for each prompt template before inputting each preset prompt template into the point recognition model; wherein the prompt words are prompt information corresponding to the point information; and determine the corresponding prompt template based on the preset number of prompt words and the preset basic prompt template.

[0098] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0099] The communication interface 502 is used for communication between the electronic device and other devices.

[0100] The memory may include a random access memory (RAM) or a non-volatile memory (NVM), such as at least one disk memory. Optionally, the memory may also be at least one storage device located away from the aforementioned processor.

[0101] The above-mentioned processor can be a general-purpose processor, including a central processing unit, a network processor (Network Processor, NP), etc.; it can also be a digital signal processing processor (Digital Signal Processing, DSP), an application-specific integrated circuit, a field programmable gate array or other programmable logic device, a discrete gate or transistor logic device, a discrete hardware component, etc.

[0102] Embodiment 7:

[0103] Based on the same technical concept, an embodiment of the present application provides a computer-readable storage medium, which stores a computer program that can be executed by an electronic device. When the program runs on the electronic device, the electronic device implements any of the above embodiments when executing.

[0104] The above-mentioned computer-readable storage medium can be any available medium or data storage device that can be accessed by the processor in the electronic device, including but not limited to magnetic storage such as floppy disks, hard disks, magnetic tapes, magneto-optical disks (MO), etc., optical storage such as CD, DVD, BD, HVD, etc., and semiconductor storage such as ROM, EPROM, EEPROM, non-volatile memory (NANDFLASH), solid-state drive (SSD), etc.

[0105] Those skilled in the art will appreciate that the embodiments of the present application may be provided as methods, systems, or computer program products. Therefore, the present application may adopt the form of a complete hardware embodiment, a complete software embodiment, or an embodiment in combination with software and hardware. Moreover, the present application may adopt the form of a computer program product implemented in one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) that include computer-usable program code.

[0106] The present application is described with reference to the flowcharts and / or block diagrams of the methods, devices (systems), and computer program products according to the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0107] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0108] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0109] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A method for identifying point information, characterized in that: The method comprises: An image of the point information to be determined and each preset prompt template are input into the point recognition model, and an image feature vector of the image is determined based on an image encoder in the point recognition model; a text embedding feature vector corresponding to each preset target sample scene category is determined based on a text encoder in the point recognition model and a plurality of preset target sample scene categories based on each prompt template, and a Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain a sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes corresponding point information; According to the similarity between the image feature vector and each sample scene feature vector, the point position information of the camera corresponding to the image is determined.

2. The method according to claim 1, characterized in that: Determining the point information of the camera corresponding to the image according to the similarity between the image feature vector and each prompt template feature vector includes: Determining the similarity between the image feature vector and each sample scene feature vector; The sample scene feature vector corresponding to the maximum similarity value is determined as the target sample scene feature vector, and the target sample scene category corresponding to the target sample scene feature vector is determined as the point information of the camera corresponding to the image.

3. The method according to claim 2, characterized in that The determining the similarity between the image feature vector and each sample scene feature vector comprises: Using a Minkowski distance algorithm, determine the Minkowski distance between the image feature vector and each sample scene feature vector; The Minkowski distance between the image feature vector and each of the sample scene feature vectors is used as the similarity between the image feature vector and each of the sample scene feature vectors.

4. The method according to claim 1, characterized in that: The Gaussian distribution probability prompt learning module is a Box-Muller algorithm module.

5. The method according to claim 1, characterized in that The text encoder is a Transformer encoder.

6. The method according to claim 1, characterized in that Before inputting each preset prompt template into the point recognition model, including: For each prompt template, a preset number of prompt words are input; wherein the prompt words are prompt information corresponding to the point information; and a corresponding prompt template is determined according to the preset number of prompt words and a preset basic prompt template.

7. A point information recognition device, characterized in that: The device comprises: A processing module is used to input an image of point information to be determined and each preset prompt template into a point recognition model, and determine an image feature vector of the image based on an image encoder in the point recognition model; based on a text encoder in the point recognition model and a plurality of preset target sample scene categories, determine a text embedding feature vector corresponding to each preset target sample scene category based on each prompt template, and a Gaussian distribution probability prompt learning module in the point recognition model processes the text embedding feature vector to obtain a sample scene feature vector corresponding to each target sample scene category; wherein each preset prompt template includes corresponding point information; The determination module is used to determine the point information of the camera corresponding to the image according to the similarity between the image feature vector and the feature vector of each sample scene.

8. The device according to claim 7, characterized in that The determination module is specifically used to determine the similarity between the image feature vector and each sample scene feature vector; determine the sample scene feature vector corresponding to the maximum similarity value as the target sample scene feature vector, and determine the target sample scene category corresponding to the target sample scene feature vector as the point information of the camera corresponding to the image.

9. An electronic device, characterized in that: The electronic device includes at least a processor and a memory, and the processor is used to implement the steps of the point information identification method according to any one of claims 1 to 6 when executing a computer program stored in the memory.

10. A computer storage medium, characterized in that: It stores a computer program executable by an electronic device. When the program runs on the electronic device, the electronic device executes the steps of the point information identification method according to any one of claims 1 to 6.

Images