A search method, device, equipment, medium and program product of a tilt photography model

By using a text-to-image conversion neural network to convert text into images and match targets in a pre-defined point cloud model, the problem of time-consuming and laborious single-model search in oblique photogrammetry is solved, achieving fast and efficient single-model localization.

CN120508671BActive Publication Date: 2025-11-25SHENZHEN SMARTCITY TECH DEV GRP CO LTD
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
CN202511006418.8
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2025-07-22
Publication Date
2025-11-25
Estimated Expiration
2045-07-22

AI Technical Summary

Technical Problem

In existing technologies, it is time-consuming and laborious for users to find individual models in oblique photogrammetry models, and there is a lack of effective and rapid localization methods.

Method used

The search text is converted into a search image by a pre-trained image-to-text conversion neural network, and then matched in a pre-defined point cloud model to determine the target point cloud unit model. Finally, the target unit model is located in the oblique photogrammetry model.

Benefits of technology

This allows users to quickly and easily locate individual models in oblique photogrammetry models using text, improving search efficiency and reducing manpower and time costs.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120508671B_ABST
    Figure CN120508671B_ABST
Patent Text Reader

Abstract

Embodiments of the present application provide a search method and device for a tilt photography model, an equipment, a medium and a program product. The method comprises: obtaining a search text, a first tilt photography model and a first point cloud model of the first tilt photography model; the first tilt photography model and the first point cloud model have been individualized; inputting the search text into a picture-text conversion neural network for processing to obtain a search image; the picture-text conversion neural network is obtained by pre-training a preset first neural network based on a sample image; the sample image is generated by rendering at least one sample point cloud individual model from at least one perspective; matching at least one target point cloud individual model from at least one point cloud individual model in the first point cloud model, which matches the search image; and determining the individual model corresponding to each of the at least one target point cloud individual model in the first tilt photography model as a target individual model hit by the search text.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] This application relates to the field of computer graphics, and more specifically to the field of computer three-dimensional model processing, and particularly to a method, apparatus, device, medium, and program product for searching oblique photogrammetry models. Background Technology

[0002] With the development of science and technology, the concept of smart cities has been proposed. A smart city refers to a modern urban form that uses next-generation information technologies such as the Internet of Things, big data, artificial intelligence (AI), 5G communication, and blockchain to digitally empower urban governance, public services, and industrial development, thereby achieving optimized allocation of urban resources, convenient and efficient living for residents, and sustainable environmental development.

[0003] In the construction of intelligent city system platforms based on digital twin technology, oblique photogrammetry is used to acquire multi-view images of the city, and then oblique photogrammetry models are created based on these images. Oblique photogrammetry models, to a certain extent, recreate the real-world environment in the virtual world. An oblique photogrammetry model is a three-dimensional geographic information model constructed using oblique photogrammetry technology. Its core involves using platforms such as drones and airplanes equipped with multi-lens cameras to take multi-angle photographs of surface objects from five directions: vertical, front, back, left, and right. Computer vision algorithms (such as Structure from Motion and Multi-View Stereo) are then used to transform massive amounts of image data into a three-dimensional model with realistic textures and geographic coordinates. The data structure of an oblique photogrammetry model is typically a multi-level continuous triangular facet; that is, at a microscopic level, an oblique photogrammetry model is actually composed of a large number of triangular facets pieced together. Summary of the Invention

[0004] The purpose of this application is to provide a method, apparatus, device, medium, and program product for searching oblique photogrammetry models, which can, to a certain extent, enable the searching of individual models in an oblique photogrammetry model through text, such as searching for a building or road in an oblique photogrammetry model of a city through text.

[0005] A first aspect of this application provides a method for searching an oblique photogrammetry model, the method comprising:

[0006] The search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model are obtained; both the first oblique photogrammetry model and the first point cloud model have been singled out.

[0007] The search text is input into a text-to-image conversion neural network for processing to obtain a search image. The text-to-image conversion neural network is pre-trained on a preset first neural network based on sample images. The sample images are generated by rendering at least one sample point cloud unit model from at least one viewpoint.

[0008] At least one target point cloud unit model that matches the search image is matched from at least one point cloud unit model in the first point cloud model, and the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model is determined as the target unit model hit by the search text.

[0009] A second aspect of this application provides a search apparatus for an oblique photogrammetry model, the apparatus comprising:

[0010] The first acquisition module is used to acquire the search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model; both the first oblique photogrammetry model and the first point cloud model have been individualized.

[0011] The first processing module is used to input the search text into the image-to-text conversion neural network for processing to obtain the search image; the image-to-text conversion neural network is pre-trained on a preset first neural network based on sample images, and the sample images are generated by rendering at least one sample point cloud single model from at least one viewpoint.

[0012] The first sampling module is used to match at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model, and to determine the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model as the target unit model hit by the search text.

[0013] A third aspect of this application provides an electronic device including a processor and a memory, the memory storing a program or instructions executable on the processor, the program or instructions being executed by the processor to implement the steps of the oblique photogrammetry model search method as described in the first aspect.

[0014] A fourth aspect of this application provides a readable storage medium storing a program or instructions that, when executed by a processor, implement the steps of the search method for the oblique photogrammetry model as described in the first aspect.

[0015] A fifth aspect of this application provides a chip including a processor and a communication interface, the communication interface being coupled to the processor, the processor being configured to run a program or instructions to implement the steps of the oblique photogrammetry model search method as described in the first aspect.

[0016] A sixth aspect of this application provides a computer program product stored in a storage medium, which is executed by at least one processor to implement the steps of the oblique photogrammetry model search method as described in the first aspect.

[0017] In this embodiment, a pre-trained image-text neural network converts the search text into a search image. Then, based on the search image, a matching process is performed on the first point cloud model corresponding to the first individualized oblique photogrammetry model to be searched. The target point cloud individual model that matches the search image is identified from the first point cloud model. The individual model corresponding to the target point cloud individual model in the first oblique photogrammetry model is then determined as the target individual model matched by the search text, thereby enabling the search for individual models within the oblique photogrammetry model using text. In other words, this embodiment provides a novel interactive method for interacting with oblique photogrammetry models, allowing users to search for individual models within the oblique photogrammetry model by inputting text, enabling users to find the desired individual model more quickly and conveniently. Attached Figure Description

[0018] Figure 1 This is a flowchart illustrating the search method for the oblique photography model provided in the embodiments of this application;

[0019] Figure 2 This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0020] Figure 3 This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0021] Figure 4 This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0022] Figure 5 This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0023] Figure 6 This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0024] Figure 7This is a flowchart illustrating some steps in the search method for the oblique photogrammetry model provided in the embodiments of this application;

[0025] Figure 8 This is a schematic diagram illustrating the relevant steps in training the individualized neural network and the image-to-text conversion neural network in the oblique photogrammetry model search method provided in this application embodiment;

[0026] Figure 9 This is a schematic diagram of the structure of a search device for an oblique photography model provided in an embodiment of this application;

[0027] Figure 10 This is a schematic diagram of the hardware structure of an electronic device that implements an embodiment of this application;

[0028] Figure 11 This is a schematic diagram of another hardware structure of an electronic device that implements an embodiment of this application. Detailed Implementation

[0029] The technical solutions of the embodiments of this application will be clearly described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application are within the scope of protection of this application.

[0030] The terms "first," "second," etc., used in the specification and claims of this application are used to distinguish similar objects and not to describe a specific order or sequence. It should be understood that such use of data can be interchanged where appropriate so that embodiments of this application can be implemented in orders other than those illustrated or described herein, and the objects distinguished by "first," "second," etc., are generally of the same class and the number of objects is not limited; for example, a first object can be one or more. Furthermore, in the specification and claims, "and / or" indicates at least one of the connected objects, and the character " / " generally indicates that the preceding and following objects are in an "or" relationship.

[0031] In the construction of smart city platforms based on digital twins, oblique photogrammetry technology is extensively used to collect data on urban topography and environment, and oblique photogrammetry models are constructed based on the collected multi-view images. Oblique photogrammetry models are typically composed of multi-level continuous triangular faces; that is, in oblique photogrammetry models, triangular faces (triangle meshes) are the basic geometric units constituting the 3D model. They achieve a 3D geometric representation of real-world features (such as buildings, terrain, and vegetation) by connecting three vertices in space to form triangles, and then piecing together countless triangles to form a continuous mesh surface.

[0032] In current technologies, if a user wants to find a specific individual model within an oblique photogrammetry model, there is no technology to help them quickly locate the desired model. Therefore, the user must visually identify and judge whether the desired model exists within the oblique photogrammetry model displayed in the current window. If not, the user needs to move or switch the area of ​​the oblique photogrammetry model displayed in the window to include other parts of the model. The user then checks again whether the desired model exists within the current window, and if not, moves or switches the area again. This process is repeated until the desired model is found. This process is time-consuming and laborious, hindering the user's ability to quickly locate the desired model.

[0033] The method, apparatus, device, storage medium, chip, and computer program product for searching oblique photogrammetry models provided in this application can effectively solve the above-mentioned technical problems. The following, in conjunction with the accompanying drawings, provides a detailed description of the method, apparatus, device, storage medium, chip, and computer program product for searching oblique photogrammetry models through specific embodiments and application scenarios.

[0034] like Figure 1 As shown, Figure 1 This is a flowchart illustrating the search method for an oblique photogrammetry model provided in this application embodiment. The search method for the oblique photogrammetry model can be applied to electronic devices, such as computers, smartphones, tablets, wearable smart devices, etc., or servers in distributed systems, cloud servers, intelligent cloud computing servers with artificial intelligence technology, or intelligent cloud hosts. It can also be applied to... Figure 9 The search device for the oblique photography model shown Figure 10 or Figure 11 For details regarding the electronic devices shown, please refer to the relevant descriptions below. Figure 1 The search method for the oblique photogrammetry model includes the following steps S11 to S13:

[0035] S11, obtain the search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model.

[0036] The search text can be the text used to search for a target unit model in the first oblique photogrammetry model. For example, it could be the name of a street or building manually entered by the user and received or detected by the device performing the method of the oblique photogrammetry model from other electronic devices with which it is communicatively connected.

[0037] The first oblique photogrammetry model is the oblique photogrammetry model to be searched, that is, the oblique photogrammetry model from which the target single-unit model matching the search text is expected to be found. In other words, the first oblique photogrammetry model has been singled out. Optionally, the first oblique photogrammetry model can be obtained by pre-processing single-unitization based on the second oblique photogrammetry model (see details). Figure 5 , Figure 6 and Figure 7 (As described in the relevant description of the embodiment shown), the second oblique photography model can be obtained by fusing multi-view (vertical view, front view, back view, left view, right view) images of a certain city in advance. The multi-view images are obtained by oblique photography of the terrain environment of the city in advance.

[0038] The first point cloud model is the point cloud model corresponding to the first oblique photogrammetry model. That is, the first point cloud model and the first oblique photogrammetry model reflect the same terrain and environment. The difference is that the first point cloud model is presented in the form of a point cloud, while the first oblique photogrammetry model is presented as a mesh surface model in the form of triangular facets. The first point cloud model has also been individualized. Optionally, the first point cloud model can be obtained by pre-processing individualization based on the second point cloud model (see details...). Figure 5 (See the related description of the embodiment shown). Further optionally, the second point cloud model is obtained by inverse calculation of the second oblique photogrammetry model (see details). Figure 6 (Related description of the illustrated embodiments).

[0039] In some alternative implementations, the first oblique photogrammetry model and the first point cloud model are pre-stored in a preset storage medium or storage device, and the device performing the oblique photogrammetry model method can read the first oblique photogrammetry model and the first point cloud model from the preset storage medium or storage device to obtain the first oblique photogrammetry model.

[0040] S12, the search text is input into the image-to-text conversion neural network for processing to obtain the search image.

[0041] The image-to-text conversion neural network is pre-trained on a pre-defined first neural network based on sample images. This first neural network is a multi-modal neural network (a multi-modal neural network capable of simultaneously processing two or more different types of data, such as images, text, speech, video, and sensor signals, and achieving unified semantic understanding through learning cross-modal associations. Its core goal is to simulate the human brain's ability to integrate auditory, visual, and linguistic information to solve complex real-world problems that single-modal data cannot cover). For example, it could be a CLIP (Contrastive Language-Image Pre-training) model, which performs self-supervised training on massive image-text pairs through contrastive learning. This maps images and text to the same embedding space, generating similar vector representations for semantically related images and text, while dissimilar ones are represented far apart, allowing the model to understand semantics across modalities. Therefore, after training the first neural network into an image-to-text conversion neural network, the network can convert input images into corresponding text, and vice versa.

[0042] The sample images used to train the first neural network are generated by rendering at least one sample point cloud unit model under at least one viewpoint (optionally, at least one of a vertical viewpoint, a front viewpoint, a rear viewpoint, a left viewpoint, and a right viewpoint). Optionally, the sample point cloud unit model can be extracted from the unitized sample point cloud model. Further optionally, the sample point cloud model can be obtained by inverse calculation of the sample oblique photogrammetry model (see details). Figure 6 (Related description of the illustrated embodiments).

[0043] The search text is input into a pre-trained image-to-text conversion neural network for processing, and the search image is obtained.

[0044] S13, at least one target point cloud unit model that matches the search image is matched from at least one point cloud unit model in the first point cloud model, and the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model is determined as the target unit model hit by the search text.

[0045] As mentioned earlier, the first point cloud model has been individualized, meaning that the first point cloud model includes at least one individual point cloud model. After obtaining the search image, the search image is matched with the individual point cloud models in the first point cloud model to identify at least one target individual point cloud model that matches the search image.

[0046] In some optional implementations, matching at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model includes the following steps S131 to S132, such as... Figure 2 The diagram shows a flowchart of steps S131 to S132:

[0047] S131, determine the second similarity between the search image and each point cloud unit model in the at least one point cloud unit model;

[0048] The second similarity can measure, to some extent, the semantic similarity between the search image and the semantic similarity between the individual point cloud models.

[0049] The search image and the at least one point cloud unit model can be converted into spatial vectors in the same space, and the similarity between the spatial vector of the search image and the spatial vectors corresponding to each point cloud unit model can be calculated to obtain the second similarity between the search image and each point cloud unit model.

[0050] S132, the point cloud single-unit model corresponding to the second similarity that is less than the preset second threshold is determined as the target point cloud single-unit model that matches the search image.

[0051] The second threshold can be a preset hit standard. For example, the point cloud unit models corresponding to the second similarity scores that are less than the second threshold among all the obtained second similarity scores are determined as the target point cloud models that match the search image, that is, the target point cloud models that are hit by the search text.

[0052] In some optional implementations, matching at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model includes the following steps S133 to S135, such as... Figure 3 The diagram shows a flowchart of steps S133 to S135:

[0053] S133, determine the first similarity between the search image and the search text and the second similarity between the search image and each point cloud unit model in the at least one point cloud unit model.

[0054] The first similarity score can measure the semantic similarity between the search image and the search text to a certain extent.

[0055] Both the search image and the search text can be converted into spatial vectors in the same space, and the similarity between the spatial vector of the search image and the spatial vector of the search text can be calculated to obtain a second similarity between the search images.

[0056] For details on the process of determining the second similarity between the search image and the point cloud unit model, please refer to the relevant explanation of step S131 above, which will not be repeated here.

[0057] S134, determine the difference between the first similarity and each of the second similarities.

[0058] Difference can be used to measure the difference between the first similarity and the second similarity.

[0059] First, perform a subtraction operation on the first similarity and the second similarity. Then, take the absolute value of the difference obtained from the subtraction operation to obtain the difference between the first similarity and the second similarity.

[0060] S135, the point cloud unit model corresponding to the second similarity with a difference less than a preset first threshold is determined as the target point cloud unit model that matches the search image.

[0061] The second threshold can be a preset hit standard. For example, the point cloud unit model corresponding to the second similarity score that is less than the second threshold among all the obtained differences is determined as the target point cloud model that matches the search image, that is, the target point cloud model that is hit by the search text.

[0062] After determining the target point cloud model that the searched text hits from the first point cloud model, the target single-unit model corresponding to the target point cloud model can be found in the first oblique photogrammetry model, thereby determining the target single-unit model that the searched text hits.

[0063] In some optional implementations, determining the individual model corresponding to each of the at least one target point cloud individual model in the first oblique photogrammetry model as the target individual model hit by the search text includes the following steps S136 to S137, such as... Figure 4 The diagram shows a flowchart of steps S136 to S137:

[0064] S136, determine the first spatial position of each target point cloud individual model in the first point cloud model.

[0065] S137, the individual models located at each of the first spatial locations in the first oblique photography model are determined as the target individual models hit by the search text.

[0066] As mentioned above, there is a one-to-one correspondence between each point cloud unit model in the first point cloud model and each unit model in the first oblique photogrammetry model, and the spatial position of the corresponding point cloud unit model in the first point cloud model is the same as the spatial position of the corresponding unit model in the first oblique photogrammetry model. Therefore, after determining the target point cloud unit model that the searched text matches from the first point cloud model, the first spatial position of each target point cloud unit model in the first point cloud model can be determined. Then, the unit models located at each of the first spatial positions in the first oblique photogrammetry model are identified as the target unit models that the searched text matches. This achieves the goal of finding the unit model searched by the searched text in the first oblique photogrammetry model.

[0067] In this embodiment, a pre-trained image-text neural network converts the search text into a search image. Then, based on the search image, a matching process is performed on the first point cloud model corresponding to the first individualized oblique photogrammetry model to be searched. The target point cloud individual model that matches the search image is identified from the first point cloud model. The individual model corresponding to the target point cloud individual model in the first oblique photogrammetry model is then determined as the target individual model matched by the search text, thereby enabling the search for individual models within the oblique photogrammetry model using text. In other words, this embodiment provides a novel interactive method for interacting with oblique photogrammetry models, allowing users to search for individual models within the oblique photogrammetry model by inputting text, enabling users to find the desired individual model more quickly and conveniently.

[0068] In related technologies, the acquired oblique photogrammetry models are often not yet individualized and lack the corresponding original multi-view images, making individualization processing extremely challenging. Traditional methods typically involve manually determining the 3D building base surface within the oblique photogrammetry model, then constructing a local space (similar to a bounding box) enclosing the 3D building based on this base surface. The oblique photogrammetry model is then segmented based on this local space to extract the 3D building, thus achieving individualization. However, this purely manual segmentation method not only consumes significant manpower but also substantial time. Furthermore, the segmentation process requires intensive and inefficient CPU cropping, incurring significant time and hardware overhead.

[0069] This application proposes a new single-unit segmentation method for the oblique photogrammetry model that lacks original multi-view images, which will be described below.

[0070] In some optional implementations, obtaining the first point cloud model of the first oblique photogrammetry model in step S11 above includes the following steps S111 to S113, such as... Figure 5 The diagram shows a flowchart of steps S111 to S113:

[0071] S111, Obtain the second oblique photography model.

[0072] The second oblique photogrammetry model is an oblique photogrammetry model that has not yet been individualized from the first oblique photogrammetry model. That is, the second oblique photogrammetry model and the first oblique photogrammetry model present the same terrain environment, the difference being that the second oblique photogrammetry model has not yet been individualized, while the first oblique photogrammetry model has been individualized.

[0073] S112, Perform inverse calculation on the second oblique photography model to obtain the second point cloud model;

[0074] By performing inverse calculations on the second oblique photogrammetry model, which has not yet been individualized, a corresponding second point cloud model can be obtained. That is, the second point cloud model and the second oblique photogrammetry model represent the same terrain environment; the difference is that the second point cloud model is presented as a point cloud, while the second oblique photogrammetry model is presented as a mesh surface model in the form of triangular facets. The second point cloud model has also not yet been individualized.

[0075] Optionally, step S112 includes steps S1121 to S1123, such as... Figure 6 The diagram shows a flowchart of steps S1121 to S1123:

[0076] S1121, Determine the average gradient value of each facet in the second oblique photogrammetry model and the geometric complexity of each facet; wherein, the geometric complexity of any facet is the arithmetic mean of the angles between the normals of the adjacent facets and the normal of the facet itself.

[0077] After obtaining the second oblique photogrammetry model, the following steps are taken: First, the three-dimensional spatial coordinates (X, Y, Z), normal vectors, UV coordinates (UV coordinates belong to the two-dimensional texture coordinate system, whose main function is to map the three-dimensional model surface to the two-dimensional texture image. UV coordinates are unrelated to the three-dimensional spatial coordinates of the model; they are a coordinate system specifically set for texture mapping), and texture resources are extracted based on the geometric information of the second oblique photogrammetry model.

[0078] Then, gradient calculations are performed on the texture images of each (triangular) facet in the second oblique photogrammetry model to obtain the gradient distribution map corresponding to each facet in the second oblique photogrammetry model. Optionally, the OpenCV library can be used to perform gradient calculations on the texture images. Of course, other methods can also be used to perform gradient calculations on the texture images, not limited to the OpenCV library, such as the Pillow library, Imageio library, etc.

[0079] Next, based on the UV coordinates of the three vertices of each (triangular) facet, the gradient value on the gradient distribution map corresponding to each facet is obtained, and the average gradient value corresponding to each facet is calculated. .

[0080] Next, it is necessary to calculate the geometric complexity of each facet in the second oblique photogrammetry model. As mentioned earlier, the oblique photogrammetry model is a mesh surface model in the form of triangular facets, and the second oblique photogrammetry model is no exception. Therefore, each facet in the second oblique photogrammetry model has adjacent facets, and each facet has its own normal (a line perpendicular to the triangular facet). For each facet in the second oblique photogrammetry model... Determine the relationship with the surface. Adjacent patches The normal and the surface The arithmetic mean of the angles between its own normals This value That is to say, a piece of dough The corresponding geometric complexity. Value The larger the value, the higher the geometric complexity. Conversely, the smaller the value... The smaller the value, the lower the geometric complexity.

[0081] S1122, Based on the average gradient value of each patch, the geometric complexity of each patch, and the preset weight determination rules, determine the sampling weight of each patch in the second oblique photogrammetry model.

[0082] The weighting rules are determined by the following formula:

[0083]

[0084] in, This indicates a normalization operation, which uses the range of maximum and minimum values ​​for normalization. , Converted to the range of 0 to 1. , Adjustment coefficient (default) , Of course, it can also be adjusted according to actual needs. , The value can be flexibly adjusted.

[0085] The sampling weight of each patch in the second oblique photogrammetry model is determined using the above formula. .

[0086] S1123, according to the sampling weight of each patch and the preset sampling rules, each patch in the second oblique photogrammetry model is sampled, and the vertex corresponding to each patch is determined based on the sampling points obtained from each patch, so as to obtain a second point cloud model composed of sampling points and vertices.

[0087] The sampling rule is determined by the following formula:

[0088]

[0089] in, Indicates the integer part of a floating-point number. The default value is 50, of course. The specific value can also be flexibly adjusted according to actual needs.

[0090] The sampling weights corresponding to each facet determined in step S1122 are... Substituting the sampling rule formula, we obtain the number of sampling points corresponding to each facet.

[0091] Then, the blue noise sampling point method is used to sample each patch in the second oblique photogrammetry model, and the UV coordinates of each sampled point within its respective patch are determined. Finally, for each patch... Based on the surface The patch is determined by interpolation of the sampling points and centroid coordinates obtained from the internal sampling. The UV coordinates of each vertex are obtained to determine the color value of each vertex. The second point cloud model is constructed from these sampling points sampled from the patches and the vertices of each patch.

[0092] S113, The second point cloud model is input into the single-unit neural network for processing to obtain the first point cloud model.

[0093] The individualized neural network is pre-trained on a pre-defined second neural network based on sample point cloud models and their corresponding individual model labels (see later explanations for details). The second neural network can be any of the neural networks used to process 3D point cloud data, such as Mask3D, SoftGroup, or PointGroup. The second neural network is trained using a large number of sample point cloud models and supervised based on pre-labeled individual model labels within the sample point cloud models, enabling it to perform individualized segmentation of the point cloud model, thus obtaining the individualized neural network. After obtaining the individualized neural network, the second point cloud model is input into this individualized neural network model for processing, resulting in the first point cloud model.

[0094] In some alternative implementations, after obtaining the first point cloud model that has been individualized, the second oblique photogrammetry model can be individualized based on the first point cloud model.

[0095] In some alternative implementations, obtaining the first oblique photography model in step S11 includes the following steps S114 to S115, such as... Figure 7 The diagram shows a flowchart of steps S114 to S115:

[0096] S114, determine the spatial location information of each point cloud unit model in the first point cloud model.

[0097] S115, based on the spatial position information of each point cloud unit model in the first point cloud model, the unit models located at the same position in the second oblique photography model are segmented to obtain the first oblique photography model.

[0098] As mentioned earlier, the first point cloud model, the second point cloud model, and the second oblique photogrammetry model all reflect the topographical environment of the same region. That is, the same individual object (such as a building or road) is located in the same spatial position in the first point cloud model, the second point cloud model, and the second oblique photogrammetry model. Therefore, after obtaining the individualized first point cloud model, the spatial position information of each individual point cloud model (a point cloud individual model is the local point cloud model corresponding to the physical entity in the overall point cloud model) can be determined within the first point cloud model. Then, based on the spatial position information of each individual point cloud model in the first point cloud model, it is located in the second oblique photogrammetry model. Individual models located at the same position in the second oblique photogrammetry model are segmented to obtain the first oblique photogrammetry model. For example, it is possible to base the model on the individual point cloud model... Spatial location information in the first point cloud model The spatial location information in the first oblique photography model Monomer model To perform segmentation, thereby achieving the individual model. Monomerization and cutting.

[0099] In some optional implementations, the method for searching oblique photogrammetry models provided in this application embodiment further includes the following steps:

[0100] Attributes are attached to each individual model in the first oblique photogrammetry model to perform semantic injection on each individual model in the first oblique photogrammetry model.

[0101] Attribute linking refers to the process of associating attributes (data, parameters, features) with specific objects, components, or entities. Here, "attributes" can be understood as information describing the characteristics of a single model, such as name, color, size, and functional parameters. In a digital twin city platform, this can involve associating the spatial location information of geographic entities (such as roads, buildings, and rivers) with attribute data (such as name, length, material, and population data).

[0102] After segmenting the second oblique photogrammetry model into individual units, the first oblique photogrammetry model is obtained. Attributes can be attached to each individual unit in the first oblique photogrammetry model, thereby enabling semantic injection into each individual unit in the first oblique photogrammetry model.

[0103] As mentioned above, this application embodiment involves two neural networks: a single-instance neural network and a graph-to-text conversion neural network. The training process is described below:

[0104] like Figure 8 As shown, Figure 8 This is a schematic diagram illustrating the steps involved in training the individualized neural network and the image-to-text conversion neural network in the oblique photogrammetry model search method provided in this application embodiment. For example... Figure 8 As shown, the process includes the following steps S21 to S28:

[0105] S21, Obtain the sample oblique photography model.

[0106] The sample oblique photogrammetry model is a pre-prepared oblique photogrammetry model used to train the first and second neural networks. Of course, to ensure that the trained image-to-text conversion neural network and the individualized neural network meet the requirements, a sufficient number of sample oblique photogrammetry models should be selected.

[0107] The sample oblique photography model can be stored in a preset storage medium or storage device. When the device performing the oblique photography model search method provided in the embodiments of this application trains the first neural network and the second neural network, it can read the sample oblique photography model from the storage medium or storage device to obtain the sample oblique photography model.

[0108] S22, perform inverse calculation on the sample oblique photography model to obtain the sample point cloud model.

[0109] The inverse calculation process for the sample oblique photogrammetry model is similar to that for the second oblique photogrammetry model. Those skilled in the art can understand the inverse calculation of the sample oblique photogrammetry model based on the aforementioned explanation of the inverse calculation of the second oblique photogrammetry model; therefore, the inverse calculation of the sample oblique photogrammetry model will not be specifically explained here. Correspondingly, after inverse calculation of the sample oblique photogrammetry model, the sample point cloud model is obtained.

[0110] S23, obtain the individual model label corresponding to the sample point cloud model.

[0111] Individual model labels are pre-annotated individual point cloud models from the sample point cloud models. They are used as a reference for adjusting the parameters of the second neural network during subsequent supervised training (e.g., Mask3D).

[0112] S24, the second neural network is trained based on the sample point cloud model and the individual model label until iterates to a preset number of rounds or the second neural network meets the preset requirements, thus obtaining the individualized neural network.

[0113] During training, in each round, the sample point cloud model is input into the second neural network for processing. The processing result of the second neural network is compared with the corresponding individual model label, and the parameters of the second neural network are adjusted based on the comparison result. This process is repeated until the second neural network iterates to a preset number of rounds or meets a preset requirement (e.g., the difference between the processing result of the second neural network on the sample point cloud model and the corresponding individual model label is less than a preset threshold), resulting in an individualized neural network, that is, a neural network with the ability to recognize point cloud data and segment individual units.

[0114] S25, Obtain the individualized sample point cloud model.

[0115] The sample point cloud model is a point cloud model used to train the first neural network.

[0116] In some alternative implementations, after obtaining the individualized neural network, the sample point cloud model can be input into the individualized neural network for processing to obtain the individualized sample point cloud model.

[0117] S26, extract at least one sample point cloud individual model from the individualized sample point cloud model.

[0118] After obtaining the individualized sample point cloud model, the individual sample point cloud model can be extracted from it.

[0119] S27, render the at least one sample point cloud unit model from at least one viewpoint to obtain a sample image.

[0120] The extracted sample point cloud individual models are rendered from multiple perspectives (i.e., vertical perspective, front perspective, back perspective, left perspective, and right perspective) to obtain image patches from multiple perspectives, i.e., sample images.

[0121] S28, the first neural network is trained based on the sample image until a preset number of iterations or the first neural network meets the preset requirements, thus obtaining the image-to-text conversion neural network.

[0122] During training, in each round, the sample image is input into the first neural network (e.g., the CLIP model) for processing, and the parameters of the first neural network are adjusted based on the processing results. This process is repeated until the first neural network iterates to a preset number of rounds or meets preset requirements (e.g., the matching accuracy between the processing result of the first neural network and the semantics of the sample image reaches a preset threshold), thus obtaining the image-to-text conversion neural network.

[0123] The method for searching oblique photogrammetry models provided in this application can be executed by an oblique photogrammetry model search device. This application uses an oblique photogrammetry model search device as an example to illustrate the oblique photogrammetry model search device provided in this application.

[0124] like Figure 9 The diagram shows a schematic representation of the structure of a search device for an oblique photography model provided in an embodiment of this application. Please refer to... Figure 9 The search device 10 for the oblique photography model includes:

[0125] The first acquisition module 100 is used to acquire the search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model; both the first oblique photogrammetry model and the first point cloud model have been individualized.

[0126] The first processing module 101 is used to input the search text into the image-to-text conversion neural network for processing to obtain the search image; the image-to-text conversion neural network is pre-trained on a preset first neural network based on sample images, and the sample images are generated by rendering at least one sample point cloud single model from at least one viewpoint.

[0127] The first sampling module 102 is used to match at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model, and to determine the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model as the target unit model hit by the search text.

[0128] Optionally, the first sampling module 102 includes:

[0129] The first determining submodule is used to determine the first similarity between the search image and the search text and the second similarity between the search image and each point cloud unit model in the at least one point cloud unit model;

[0130] The second determining submodule is used to determine the difference between the first similarity and each of the second similarities;

[0131] The third determining submodule is used to determine the point cloud individual model corresponding to the second similarity with a difference less than a preset first threshold as the target point cloud individual model that matches the search image.

[0132] Optionally, the first acquisition module 100 includes:

[0133] The first acquisition submodule acquires the second oblique photography model; the second oblique photography model is the oblique photography model that has not yet been individualized from the first oblique photography model.

[0134] The first inverse operator module is used to perform inverse calculation on the second oblique photogrammetry model to obtain the second point cloud model.

[0135] The first processing submodule is used to input the second point cloud model into the individualized neural network for processing to obtain the first point cloud model; the individualized neural network is obtained by pre-training a preset second neural network based on the sample point cloud model and the individualized model label corresponding to the sample point cloud model.

[0136] Optionally, the first inverse operator module includes:

[0137] The first determining unit is used to determine the average gradient value of each patch in the second oblique photogrammetry model and the geometric complexity of each patch; wherein, the geometric complexity of any patch is the arithmetic mean of the angles between the normals of the adjacent patches and the normal of the patch itself.

[0138] The second determining unit is used to determine the sampling weight of each facet in the second oblique photogrammetry model based on the average gradient value of each facet, the geometric complexity corresponding to each facet, and the preset weight determination rules.

[0139] The first sampling unit is used to sample each facet in the second oblique photogrammetry model according to the sampling weight of each facet and the preset sampling rules, and to determine the vertex corresponding to each facet based on the sampling points obtained from each facet, so as to obtain a second point cloud model composed of sampling points and vertices.

[0140] Optionally, the first acquisition module 100 includes:

[0141] The first determining submodule is used to determine the spatial location information of each point cloud unit model in the first point cloud model.

[0142] The first segmentation submodule is used to segment the individual models located at the same position in the second oblique photogrammetry model according to the spatial position information of each individual point cloud model in the first point cloud model, so as to obtain the first oblique photogrammetry model.

[0143] Optionally, the image-to-text conversion neural network is trained through the following steps:

[0144] Obtain the point cloud model of the individualized sample;

[0145] Extract at least one sample point cloud unit model from the unitized sample point cloud model;

[0146] Render the at least one sample point cloud unit model at at least one viewpoint to obtain a sample image;

[0147] The first neural network is trained based on the sample images until a preset number of iterations or the first neural network meets a preset requirement, thus obtaining the image-to-text conversion neural network.

[0148] Optionally, obtaining the individualized sample point cloud model includes:

[0149] Obtain the sample oblique photography model;

[0150] The sample oblique photography model is inversely calculated to obtain the sample point cloud model;

[0151] The sample point cloud model is input into the individualized neural network for processing to obtain the individualized sample point cloud model.

[0152] Optionally, the monolithic neural network is obtained through the following steps:

[0153] Obtain the individual model labels corresponding to the sample point cloud model;

[0154] The second neural network is trained based on the sample point cloud model and the individual model label until iteration reaches a preset number of rounds or the second neural network meets the preset requirements, thus obtaining the individualized neural network.

[0155] Optionally, the device 10 further includes:

[0156] The first attachment module is used to attach attributes to each individual model in the first oblique photogrammetry model in order to perform semantic injection on each individual model in the first oblique photogrammetry model.

[0157] Optionally, the first sampling module 102 includes:

[0158] The third determining submodule is used to determine the second similarity between the search image and each point cloud individual model in the at least one point cloud individual model;

[0159] The fourth determining submodule is used to determine the point cloud single-unit model corresponding to the second similarity that is less than the preset second threshold as the target point cloud single-unit model that matches the search image.

[0160] Optionally, the first sampling module 102 includes:

[0161] The fifth determining submodule is used to determine the first spatial position of each target point cloud individual model in the first point cloud model;

[0162] The sixth determining submodule is used to determine the individual models located at each of the first spatial locations in the first oblique photogrammetry model as the target individual models hit by the search text.

[0163] The search device 10 for the oblique photography model in this embodiment can be an electronic device or a component within an electronic device, such as an integrated circuit or a chip. The electronic device can be a terminal or other devices besides a terminal. For example, the electronic device can be a mobile phone, tablet computer, laptop computer, PDA, in-vehicle electronic device, mobile internet device (MID), augmented reality (AR) / virtual reality (VR) device, robot, wearable device, ultra-mobile personal computer (UMPC), netbook, or personal digital assistant (PDA), etc. It can also be a server, network attached storage (NAS), personal computer (PC), television (TV), ATM, or self-service machine, etc. This embodiment does not specifically limit the scope of the electronic device.

[0164] The oblique photogrammetry model search device 10 in this embodiment can be a device with an operating system. This operating system can be Android, iOS, or other possible operating systems; this embodiment does not specifically limit its use.

[0165] The oblique photography model search device 10 provided in this application embodiment can achieve Figures 1 to 8 The various processes implemented in the method implementation examples will not be described again here to avoid repetition.

[0166] In some alternative implementations, such as Figure 10 As shown, this application embodiment also provides an electronic device 1300, including a processor 1301 and a memory 1302. The memory 1302 stores a program or instructions that can run on the processor 1301. When the program or instructions are executed by the processor 1301, they implement the various steps of the above-described oblique photogrammetry model search method embodiment and can achieve the same technical effect. To avoid repetition, they will not be described again here.

[0167] It should be noted that the electronic devices in the embodiments of this application include the mobile electronic devices and non-mobile electronic devices described above.

[0168] Figure 11 A schematic diagram of the hardware structure of an electronic device to implement an embodiment of this application.

[0169] The electronic device 170 includes, but is not limited to, components such as: a radio frequency unit 1701, a network module 1702, an audio output unit 1703, an input unit 1704, a sensor 1705, a display unit 1706, a user input unit 1707, an interface unit 1708, a memory 1709, and a processor 17010. Those skilled in the art will understand that the electronic device 170 may also include a power supply (such as a battery) for powering the various components. The power supply can be logically connected to the processor 17010 through a power management system, thereby enabling functions such as charging, discharging, and power consumption management through the power management system. Figure 11 The electronic device structure shown does not constitute a limitation on the electronic device. The electronic device may include more or fewer components than shown, or combine certain components, or have different component arrangements, which will not be elaborated here.

[0170] The processor 17010 is used for:

[0171] The search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model are obtained; both the first oblique photogrammetry model and the first point cloud model have been singled out.

[0172] The search text is input into a text-to-image conversion neural network for processing to obtain a search image. The text-to-image conversion neural network is pre-trained on a preset first neural network based on sample images. The sample images are generated by rendering at least one sample point cloud unit model from at least one viewpoint.

[0173] At least one target point cloud unit model that matches the search image is matched from at least one point cloud unit model in the first point cloud model, and the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model is determined as the target unit model hit by the search text.

[0174] It should be understood that, in this embodiment, the input unit 1704 may include a graphics processing unit (GPU) 17041 and a microphone 17042. The GPU 17041 processes image data of still images or videos obtained by an image capture device (such as a camera) in video capture mode or image capture mode. The display unit 1706 may include a display panel 17061, which may be configured in the form of a liquid crystal display, an organic light-emitting diode, or the like. The user input unit 1707 includes at least one of a touch panel 17071 and other input devices 17072. The touch panel 17071 is also called a touch screen. The touch panel 17071 may include a touch detection device and a touch controller. Other input devices 17072 may include, but are not limited to, physical keyboards, function keys (such as volume control buttons, power buttons, etc.), trackballs, mice, and joysticks, which will not be described in detail here.

[0175] The memory 1709 can be used to store software programs and various data. The memory 1709 may primarily include a first storage area for storing programs or instructions and a second storage area for storing data. The first storage area may store the operating system, application programs or instructions required for at least one function (such as sound playback, image playback, etc.). Furthermore, the memory 1709 may include volatile memory or non-volatile memory, or both. The non-volatile memory may be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or flash memory. Volatile memory can be random access memory (RAM), static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDRSDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct memory bus RAM (DRRAM). The memory 1709 in the embodiments of this application includes, but is not limited to, these and any other suitable types of memory.

[0176] Processor 17010 may include one or more processing units; optionally, processor 17010 integrates an application processor and a modem processor, wherein the application processor mainly handles operations involving the operating system, user interface, and applications, and the modem processor mainly handles wireless communication signals, such as a baseband processor. It is understood that the aforementioned modem processor may also not be integrated into processor 17010.

[0177] Each of the above product embodiments can implement the various processes of the above oblique photogrammetry model search method embodiment through its own processor, and can achieve the same technical effect. To avoid repetition, they will not be described in detail.

[0178] This application also provides a readable storage medium storing a program or instructions. When executed by a processor, the program or instructions implement the various processes of the above-described oblique photogrammetry model search method embodiments and achieve the same technical effects. To avoid repetition, these will not be described again here. The processor is the processor in the electronic device or electronic system described in the above embodiments. The readable storage medium includes a computer-readable storage medium, such as a computer read-only memory (ROM), random access memory (RAM), magnetic disk, or optical disk.

[0179] This application embodiment also provides a chip, which includes a processor and a communication interface. The communication interface is coupled to the processor. The processor is used to run programs or instructions to implement the various processes of the above-described oblique photogrammetry model search method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0180] It should be understood that the chip mentioned in the embodiments of this application may also be referred to as a system-on-a-chip, system chip, chip system, or system-on-a-chip, etc.

[0181] This application provides a computer program product stored in a storage medium. The program product is executed by at least one processor to implement the various processes of the search method embodiment of the oblique photogrammetry model described above, and can achieve the same technical effect. To avoid repetition, it will not be described again here.

[0182] In the embodiments provided in this application, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative. For instance, the division of modules or units is only a logical functional division, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be an indirect coupling or communication connection between apparatuses or units through some interfaces, and may be electrical, mechanical, or other forms.

[0183] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0184] Furthermore, the functional units in the various embodiments of this application can be integrated into one processing unit, or each unit can exist physically separately, or two or more units can be integrated into one unit. The integrated unit can be implemented in hardware or as a software functional unit.

[0185] If the integrated unit is implemented as a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of this application embodiment, in essence, or the part that contributes to the prior art, or all or part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) or processor to execute all or part of the steps of the methods described in the various embodiments of this application. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0186] The above descriptions are merely embodiments of this application and do not limit the patent scope of this application. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art, under the guidance of this application, may make equivalent structural or procedural transformations based on the description and drawings of the embodiments of this application, or directly or indirectly apply them to other related technical fields, without departing from the spirit and scope of protection of the claims. All such transformations are similarly included within the patent protection scope of the embodiments of this application.

Claims

1. A search method for an oblique photogrammetry model, characterized in that, The method includes: The search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model are obtained; both the first oblique photogrammetry model and the first point cloud model have been singled out. The search text is input into a text-to-image conversion neural network for processing to obtain a search image. The text-to-image conversion neural network is pre-trained on a preset first neural network based on sample images. The sample images are generated by rendering at least one sample point cloud unit model from at least one viewpoint. Match at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model, and determine the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model as the target unit model hit by the search text. Wherein, determining the individual model corresponding to each of the at least one target point cloud individual model in the first oblique photogrammetry model as the target individual model hit by the search text includes: Determine the first spatial position of each target point cloud unit model in the first point cloud model; The individual models located at each of the first spatial locations in the first oblique photogrammetry model are identified as the target individual models that are hit by the search text.

2. The method according to claim 1, characterized in that, The step of matching at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model includes: Determine a first similarity between the search image and the search text, and a second similarity between the search image and each point cloud unit model in the at least one point cloud unit model; Determine the difference between the first similarity and each of the second similarities; The point cloud unit model corresponding to the second similarity with a difference less than a preset first threshold is determined as the target point cloud unit model that matches the search image.

3. The method according to claim 1, characterized in that, The step of obtaining the first point cloud model of the first oblique photogrammetry model includes: Obtain a second oblique photography model; the second oblique photography model is an oblique photography model that has not yet been individualized from the first oblique photography model. The second oblique photography model is inversely calculated to obtain the second point cloud model; The second point cloud model is input into a single-unit neural network for processing to obtain a first point cloud model; the single-unit neural network is obtained by pre-training a preset second neural network based on the sample point cloud model and the single-unit model label corresponding to the sample point cloud model.

4. The method according to claim 3, characterized in that, The inverse calculation of the second oblique photography model to obtain the second point cloud model includes: Determine the average gradient value of each facet in the second oblique photogrammetry model and the geometric complexity of each facet; wherein, the geometric complexity of any facet is the arithmetic mean of the angles between the normals of the adjacent facets and the normal of the facet itself. Based on the average gradient value of each patch, the geometric complexity of each patch, and the preset weight determination rules, the sampling weight of each patch in the second oblique photogrammetry model is determined. Based on the sampling weight of each facet and the preset sampling rules, each facet in the second oblique photogrammetry model is sampled, and the vertices corresponding to each facet are determined based on the sampling points obtained from each facet, thus obtaining a second point cloud model composed of sampling points and vertices.

5. The method according to claim 3, characterized in that, The acquisition of the first oblique photography model includes: Determine the spatial location information of each individual point cloud model in the first point cloud model; Based on the spatial position information of each point cloud unit model in the first point cloud model, the unit models located at the same position in the second oblique photogrammetry model are segmented to obtain the first oblique photogrammetry model.

6. The method according to claim 3, characterized in that, The image-to-text conversion neural network is trained through the following steps: Obtain the point cloud model of the individualized sample; Extract at least one sample point cloud unit model from the unitized sample point cloud model; Render the at least one sample point cloud unit model at at least one viewpoint to obtain a sample image; The first neural network is trained based on the sample images until a preset number of iterations or the first neural network meets a preset requirement, thus obtaining the image-to-text conversion neural network.

7. The method according to claim 6, characterized in that, The process of obtaining the individualized sample point cloud model includes: Obtain the sample oblique photography model; The sample oblique photography model is inversely calculated to obtain the sample point cloud model; The sample point cloud model is input into the individualized neural network for processing to obtain the individualized sample point cloud model.

8. The method according to claim 7, characterized in that, The single-unit neural network is obtained through the following steps: Obtain the individual model labels corresponding to the sample point cloud model; The second neural network is trained based on the sample point cloud model and the individual model label until iteration reaches a preset number of rounds or the second neural network meets the preset requirements, thus obtaining the individualized neural network.

9. The method according to claim 3, characterized in that, The method further includes: Attributes are attached to each individual model in the first oblique photogrammetry model to perform semantic injection on each individual model in the first oblique photogrammetry model.

10. The method according to claim 1, characterized in that, The step of matching at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model includes: Determine the second similarity between the search image and each point cloud individual model in the at least one point cloud individual model; The point cloud unit model corresponding to the second similarity that is less than the preset second threshold is determined as the target point cloud unit model that matches the search image.

11. A search device for an oblique photogrammetry model, characterized in that, The device includes: The first acquisition module is used to acquire the search text, the first oblique photogrammetry model, and the first point cloud model of the first oblique photogrammetry model; both the first oblique photogrammetry model and the first point cloud model have been individualized. The first processing module is used to input the search text into the image-to-text conversion neural network for processing to obtain the search image; the image-to-text conversion neural network is pre-trained on a preset first neural network based on sample images, and the sample images are generated by rendering at least one sample point cloud single model from at least one viewpoint. The first sampling module is used to match at least one target point cloud unit model that matches the search image from at least one point cloud unit model in the first point cloud model, and to determine the unit model corresponding to each of the at least one target point cloud unit models in the first oblique photogrammetry model as the target unit model hit by the search text. The first sampling module includes: The fifth determining submodule is used to determine the first spatial position of each target point cloud individual model in the first point cloud model; The sixth determining submodule is used to determine the individual models located at each of the first spatial locations in the first oblique photogrammetry model as the target individual models hit by the search text.

12. An electronic device, characterized in that, It includes a processor and a memory, the memory storing programs or instructions that can run on the processor, the programs or instructions being executed by the processor to implement the steps of the search method for the oblique photogrammetry model as described in any one of claims 1 to 10.

13. A readable storage medium, characterized in that, The readable storage medium stores a program or instructions that, when executed by a processor, implement the steps of the search method for the oblique photogrammetry model as described in any one of claims 1 to 10.

14. A computer program product, characterized in that, The computer program includes programs or instructions that, when executed by a processor, implement the steps of the search method for the oblique photogrammetry model as described in any one of claims 1 to 10.

Citation Information

Patent Citations

  • Monomerization method of power distribution network equipment oblique photography three-dimensional model and storage medium

    CN112419504A

  • Method for building in cloud monomer oblique photography model

    CN113689567A