A digital twin space viewpoint positioning method, device, equipment and medium

Abstract

The present invention provides a digital twin space viewpoint positioning method, device, equipment and medium, and relates to the field of digital twin technology. The method comprises: obtaining a target actual image list of a target building object, using a SIFT algorithm to obtain target feature points corresponding to the target actual image and target feature vectors corresponding to the target feature points, obtaining preset feature points in a preset BIM model and preset feature vectors corresponding to the preset feature points, using a FLANN matcher to match the target feature vector and the preset feature vector to obtain a matching result, based on the matching result, using a PnP algorithm to obtain a target position vector and a target attitude rotation matrix corresponding to an initial virtual camera in a preset BIM model, obtaining a target virtual camera, and generating a target model viewing angle based on the target actual image and the target virtual camera, thereby generating a target model and reducing the cost of hardware equipment.

Description

Technical Field

[0001] The present invention relates to the field of digital twin technology, and in particular to a method, device, equipment and medium for spatial viewpoint positioning of a digital twin. Background Art

[0002] Digital Twin is an integrated multi-physical, multi-scale, and multi-probability simulation process that completes mapping in virtual space to reflect the entire life cycle of the corresponding physical equipment. The development of digital twin model technology has brought unprecedented convenience to the field of engineering construction. However, the existing BIM model space viewpoint positioning method still has certain limitations in accuracy and efficiency and cannot fully meet the actual needs of engineering.

[0003] The existing BIM model space viewpoint positioning method has high hardware equipment requirements, large computing resource requirements, and complex operations. The existing BIM model space viewpoint positioning method usually requires the use of professional image acquisition equipment (such as 360 panoramic cameras) and position acquisition equipment. These devices are expensive and require high-computing power computing hardware for comparison and analysis. This has high requirements on hardware configuration, especially on mobile devices. It is difficult to achieve efficient comparison results, and professional operators are required to perform complex image acquisition and processing. The operation steps are cumbersome and not conducive to real-time application on site. Summary of the invention

[0004] In view of the above technical problems, the technical solution adopted by the present invention is: a method for spatial viewpoint positioning of a digital twin, the method comprising the following steps:

[0005] Acquire a target actual image list of the target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object;

[0006] For a target actual image, obtaining a target model perspective corresponding to the target actual image, thereby generating a target model;

[0007] Wherein, for a target actual image, obtaining the target model perspective corresponding to the target actual image comprises the following steps:

[0008] For the actual target image, the SIFT algorithm is used to obtain the target feature points corresponding to the actual target image and the target feature vectors corresponding to the target feature points;

[0009] Obtain a preset BIM model corresponding to the target building object, and obtain preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector;

[0010] Use the FLANN matcher to match the target feature vector with the preset feature vector to obtain the matching result;

[0011] Based on the matching results, the PnP algorithm is used to obtain the target position vector and target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model;

[0012] Based on the target position vector and the target attitude rotation matrix, the position and direction of the initial virtual camera are adjusted, and the adjusted initial virtual camera is obtained and used as the intermediate virtual camera;

[0013] Obtaining the viewing angle information of the target actual image, and adjusting the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, obtaining the adjusted intermediate virtual camera and using the adjusted intermediate virtual camera as the target virtual camera;

[0014] Based on the target actual image and the target virtual camera, the target model perspective is generated.

[0015] A digital twin space viewpoint positioning device, the device comprising:

[0016] A target actual image acquisition module, used to acquire a target actual image list of a target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object;

[0017] The target model module is used to obtain a target model perspective corresponding to a target actual image, thereby generating a target model;

[0018] Wherein, the target model module includes:

[0019] The target feature point acquisition submodule is used to obtain the target feature points corresponding to the target actual image and the target feature vectors corresponding to the target feature points using the SIFT algorithm for the target actual image;

[0020] A preset feature point acquisition submodule is used to acquire a preset BIM model corresponding to a target building object, and to acquire preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector;

[0021] The matching result acquisition submodule is used to match the target feature vector and the preset feature vector using the FLANN matcher to obtain the matching result;

[0022] The target position vector acquisition submodule is used to obtain the target position vector and target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model using the PnP algorithm based on the matching results;

[0023] The intermediate virtual camera acquisition submodule is used to adjust the position and direction of the initial virtual camera based on the target position vector and the target attitude rotation matrix, obtain the adjusted initial virtual camera and use the adjusted initial virtual camera as the intermediate virtual camera;

[0024] A target virtual camera acquisition submodule is used to acquire the viewing angle information of the target actual image, and adjust the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, acquire the adjusted intermediate virtual camera and use the adjusted intermediate virtual camera as the target virtual camera;

[0025] The target model perspective submodule is used to generate the target model perspective based on the target actual image and the target virtual camera.

[0026] According to another aspect of the present invention, a non-transitory computer-readable storage medium is provided, in which at least one instruction or at least one program is stored, and the at least one instruction or the at least one program is loaded and executed by a processor to implement the aforementioned method.

[0027] According to yet another aspect of the present invention, an electronic device is provided, comprising a processor and the aforementioned non-transitory computer-readable storage medium.

[0028] The present invention has at least the following beneficial effects: In summary, a target actual image list of a target building object is obtained, and for the target actual image, a SIFT algorithm is used to obtain target feature points corresponding to the target actual image and target feature vectors corresponding to the target feature points, a preset BIM model corresponding to the target building object is obtained, and preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points are obtained, a FLANN matcher is used to match the target feature vector and the preset feature vector to obtain a matching result, and based on the matching result, a PnP algorithm is used to obtain a target position vector and a target attitude rotation matrix corresponding to an initial virtual camera in the preset BIM model, and based on the target position vector and the target attitude rotation matrix, the position and direction of the initial virtual camera are adjusted, and the adjusted initial virtual camera is obtained and the adjusted BIM model is adjusted. The adjusted initial virtual camera is used as the intermediate virtual camera to obtain the viewing angle information of the target actual image, and the viewing angle and focal length of the intermediate virtual camera are adjusted based on the viewing angle information of the target actual image, the adjusted intermediate virtual camera is obtained and used as the target virtual camera, and the target model viewing angle is generated based on the target actual image and the target virtual camera, thereby generating a target model. Compared with the prior art that requires professional image acquisition equipment and position acquisition equipment, the present invention greatly reduces the cost of hardware equipment, and through simple fuzzy positioning and feature extraction comparison, efficient comparison results can be generated in a low computing power environment, reducing the demand for high-performance computing hardware, and can be efficient and practical on mobile devices such as mobile phones, reducing the learning curve and usage threshold, and facilitating on-site operation and rapid response. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0030] Figure 1 A flowchart of a digital twin space viewpoint positioning method provided in Example 1 of the present invention;

[0031] Figure 2 This is a flow chart of S200 provided in the first embodiment of the present invention;

[0032] Figure 3 A schematic diagram of the structure of a digital twin space viewpoint positioning device provided in Embodiment 2 of the present invention;

[0033] Figure 4 A schematic diagram of the structure of the target model module provided in the second embodiment of the present invention. DETAILED DESCRIPTION

[0034] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0035] Embodiment 1,

[0036] Embodiment 1 of the present invention provides a method for spatial viewpoint positioning of a digital twin, such as Figure 1 As shown, the method comprises the following steps:

[0037] S100, obtaining a target actual image list of a target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object.

[0038] Specifically, an ordinary mobile phone camera or other ordinary image acquisition device is used to capture the entire area of ​​the target building object under several preset viewing angle information and several lighting conditions to obtain an initial actual image, and the initial actual image is manually selected to obtain a target actual image list. The preset viewing angle information includes a preset viewing angle and a preset focal length.

[0039] Specifically, a clear initial actual image is obtained by photographing the target building object at preset intervals.

[0040] S200, for a target actual image, obtaining a target model perspective corresponding to the target actual image, thereby generating a target model. Specifically, those skilled in the art know that any method of generating a target model based on a target model perspective in the prior art belongs to the protection scope of the present invention, and will not be described in detail herein; the present invention generates a target model by obtaining a target model perspective corresponding to each target actual image.

[0041] Among them, in S200, Figure 2 As shown, for a target actual image, obtaining the target model perspective corresponding to the target actual image includes the following steps:

[0042] S210, using the SIFT algorithm to obtain target feature points corresponding to the target actual image and target feature vectors corresponding to the target feature points.

[0043] Specifically, the target feature points obtained using the SIFT algorithm are significant points in the target's actual image. These points can still be accurately detected and matched under different viewing angles, scaling, rotation, and lighting conditions. They are usually points with large changes in the target's actual image, such as corner points, edge intersections, etc., and have good scale invariance, rotation invariance, and lighting invariance. Therefore, they can provide stable feature detection and matching even in complex environments.

[0044] S220, obtaining a preset BIM model corresponding to the target building object, and obtaining preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector.

[0045] Specifically, the dimension of the target feature vector is 128. The preset BIM model is a BIM model preset for the target building object, which can be understood as the preset BIM model being a BIM model of the target building object that has not been positioned by a spatial viewpoint.

[0046] Specifically, obtaining the preset feature points in the preset BIM model includes: using BIM software to load the preset BIM model corresponding to the target building object, and using a feature point tool in the BIM software to extract the preset feature points.

[0047] S230, using a FLANN matcher to match the target feature vector with the preset feature vector to obtain a matching result.

[0048] S240, based on the matching result, using the PnP algorithm to obtain the target position vector and the target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model.

[0049] Specifically, the PnP algorithm (Perspective-n-Point) is a method for estimating the position and posture of a camera. The PnP algorithm estimates the parameters of the camera through known 3D points and their projections in 2D images. That is, in the present invention, the parameters of the initial virtual camera are obtained by calculating the successfully matched target feature points and preset feature points in the matching results.

[0050] S250, adjusting the position and direction of the initial virtual camera based on the target position vector and the target posture rotation matrix, obtaining the adjusted initial virtual camera and using the adjusted initial virtual camera as the intermediate virtual camera.

[0051] S260, obtaining the viewing angle information of the target actual image, and adjusting the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, obtaining the adjusted intermediate virtual camera and using the adjusted intermediate virtual camera as the target virtual camera. Thus, the position and viewing angle of the target virtual camera can be accurately adjusted to be consistent with the actual camera corresponding to the target actual image, thereby achieving accurate fuzzy positioning and viewing angle adjustment.

[0052] S270, generating a target model perspective based on the target actual image and the target virtual camera.

[0053] In summary, a target actual image list of the target building object is obtained, and for the target actual image, the SIFT algorithm is used to obtain the target feature points corresponding to the target actual image and the target feature vectors corresponding to the target feature points, and the preset BIM model corresponding to the target building object is obtained, and the preset feature points in the preset BIM model and the preset feature vectors corresponding to the preset feature points are obtained, and the target feature vectors and the preset feature vectors are matched by the FLANN matcher to obtain the matching results, and based on the matching results, the PnP algorithm is used to obtain the target position vector and the target attitude rotation matrix corresponding to the initial virtual camera in the preset BIM model, and based on the target position vector and the target attitude rotation matrix, the position and direction of the initial virtual camera are adjusted, and the adjusted initial virtual camera is obtained and the adjusted initial virtual camera is transferred to the preset BIM model. The initial virtual camera is used as an intermediate virtual camera to obtain the viewing angle information of the target actual image, and the viewing angle and focal length of the intermediate virtual camera are adjusted based on the viewing angle information of the target actual image, the adjusted intermediate virtual camera is obtained and used as the target virtual camera, and the target model viewing angle is generated based on the target actual image and the target virtual camera. Compared with the prior art which requires professional image acquisition equipment and position acquisition equipment, the present invention greatly reduces the cost of hardware equipment, and through simple fuzzy positioning and feature extraction comparison, efficient comparison results can be generated in a low computing power environment, reducing the demand for high-performance computing hardware, and can be efficient and practical on mobile devices such as mobile phones, reducing the learning curve and usage threshold, and facilitating on-site operation and rapid response.

[0054] Specifically, in S210, using the SIFT algorithm to obtain target feature points corresponding to the target actual image and target feature vectors corresponding to the target feature points includes:

[0055] For the actual target image, a convolution operation is performed using a Gaussian kernel with several standard deviations to obtain candidate key points.

[0056] Specifically, potential feature points are detected in different scale spaces (i.e., different resolutions of the image), and Gaussian Blur and Difference of Gaussian (DoG) methods are used to find extreme points as candidate key points.

[0057] Screen the candidate key points and obtain the target feature points.

[0058] The detected candidate key points are accurately located, low-contrast points and edge response points are removed, and stable key points are retained as target feature points.

[0059] The gradient direction of the target feature point is obtained, and based on the gradient direction of the target feature point, the main direction of the target feature point is obtained.

[0060] Specifically, one or more directions are assigned to each key point according to the gradient direction of the neighborhood of the target feature point to ensure that the feature descriptor has rotation invariance. The method also includes obtaining the amplitude of the target feature point.

[0061] Based on the main direction of the target feature point, a target feature vector corresponding to the target feature point is generated.

[0062] Specifically, in the main direction of the target feature point, a gradient histogram of the area around the target feature point is calculated to generate a target feature vector corresponding to the target feature point.

[0063] Specifically, in S230, the target feature vector and the preset feature vector are matched using the FLANN matcher to obtain a matching result, including:

[0064] The target feature vector and the preset feature vector are matched using the FLANN matcher.

[0065] If the target feature vector and the preset feature vector are matched successfully, the combination of the target feature point and the preset feature point is taken as the matching result.

[0066] Furthermore, after generating the target model perspective, it also includes:

[0067] Get the number of successfully matched target feature points.

[0068] Based on the number of successfully matched target feature points and the total number of target feature points, a target matching rate is obtained. Specifically, the target matching rate is the quotient of the number of successfully matched target feature points divided by the total number.

[0069] Based on the target matching rate and the preset matching threshold, the target model perspective is verified, wherein if the target matching rate is greater than the preset matching threshold, the verification is considered successful. If the target matching rate is not greater than the preset matching threshold, the verification is considered failed.

[0070] Specifically, the preset matching threshold may be determined according to actual needs. In one embodiment of the present invention, the preset matching threshold is 0.9.

[0071] Furthermore, after generating the target model perspective, it also includes:

[0072] Get the location information of the target feature point in the matching result and the location information of the specified feature point corresponding to the target feature point in the target model perspective.

[0073] Specifically, the location information A of the target feature point in the matching result i1 =(XA i1 , YA i1 ), the position information A of the specified feature point corresponding to the target feature point in the target model perspective i2 =(XA i2 , YA i2 ), XA i1 YA is the coordinate of the target feature point on the X axis, i1 is the coordinate of the target feature point on the Y axis, XA i2 YA is the coordinate of the preset feature point on the X axis. i2 is the coordinate of the preset feature point on the Y axis, where i ranges from 1 to m, and m is the number of target feature points in the matching result.

[0074] Based on the position information of the target feature point and the position information of the specified feature point, the position change of the target feature point is obtained; specifically, the position change is [(XA i1 -XA i2 ) 2 + (YA i1 -YA i2 ) 2 ] 1 / 2 .

[0075] All position changes are averaged to obtain the average change.

[0076] The SSIM index is used to obtain the similarity between the target actual image and the target model perspective.

[0077] Specifically, the target actual image and the target model perspective are overlapped and compared, and the SSIM index is used to measure the similarity between the target actual image and the target model perspective. The similarity value is between 0 and 1, where 1 means that the target actual image and the image corresponding to the target model perspective are exactly the same.

[0078] Based on the average change and similarity, the target model perspective is verified.

[0079] Specifically, based on the average change and similarity, the target model perspective is verified including:

[0080] Get the preset feature point tolerance value. If the average change is not greater than the preset feature point tolerance value, the verification is considered successful, otherwise the verification is considered failed.

[0081] Get the preset overlapping area threshold. If the similarity is not less than the preset overlapping area threshold, the verification is considered successful, otherwise the verification is considered failed.

[0082] Furthermore, if the test fails, the position and viewing angle of the target virtual camera are adjusted according to the average change and the similarity. Specifically, the gradient descent optimization algorithm is used to iteratively adjust the parameters of the target virtual camera to minimize the average change and maximize the similarity corresponding to the SSIM.

[0083] In summary, the present invention verifies the target model perspective by matching the number, position change, and similarity of successfully matched target feature points, so as to more accurately generate the accuracy of the view, and iteratively adjusts the target virtual camera parameters to ensure that the target model perspective and the target actual image are highly consistent. It also designs fault tolerance for the engineering industry to ensure acceptability within a certain range of differences and output calculation results normally.

[0084] In addition, in order to improve the efficiency and accuracy of model verification, the present invention also provides a comparison tool based on split-screen technology. This tool can simultaneously display the generated target model perspective and the target actual image in the same interface. Users can conduct comparative analysis through an intuitive split-screen layout, easily locate differences, and optimize the accuracy of the model.

[0085] Specifically, the comparison tool based on split-screen technology includes:

[0086] The startup module is used to open the split-screen comparison tool and load the generated target model perspective and target actual image.

[0087] The split-screen display module is used to automatically divide the target model perspective and the target actual image into two parts. Users can freely adjust the view ratio on the left and right sides to facilitate comparison from different angles.

[0088] Interactive adjustment module, used to fine-tune the contrast area by dragging the slider, focusing on specific details or the overall appearance, so as to deeply analyze the subtle differences between the two.

[0089] The feature comparison module is used to compare the feature points, feature lines and overall appearance of the target model one by one to find inconsistencies. Once a mismatch is found, the user can record it as a basis for subsequent model adjustments.

[0090] The continuous optimization module is used to adjust and optimize the target model viewing angle accordingly according to the comparison results, until the target model viewing angle is completely consistent with the target actual image or reaches the expected matching degree.

[0091] In summary, the use of a comparison tool based on split-screen technology can quickly verify the consistency between the digital twin model and the physical project. Through precise comparison, the engineering team can promptly discover errors in the model, such as dimensional deviations and structural inconsistencies, thereby effectively preventing potential quality risks and improving project quality and safety. In addition, during the project presentation, reporting or quality acceptance stage, the generated visualization results must be highly consistent with the actual project. The split-screen comparison tool can help the production team efficiently verify whether the model view accurately reflects the actual situation, ensuring that the output results are both professional and credible. In addition, it can also serve as an effective tool for internal communication and collaboration within the project team, promoting the understanding and consensus of all parties on project progress and quality standards.

[0092] Embodiment 2:

[0093] Embodiment 2 of the present invention provides a digital twin space viewpoint positioning device, such as Figure 3 As shown, the device comprises:

[0094] The target actual image acquisition module 10 is used to acquire a target actual image list of a target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object.

[0095] The target model module 20 is used to obtain a target model viewing angle corresponding to a target actual image, thereby generating a target model.

[0096] Among them, Figure 4 As shown, the target model module 20 includes:

[0097] The target feature point acquisition submodule 21 is used to obtain target feature points corresponding to the target actual image and target feature vectors corresponding to the target feature points by using the SIFT algorithm for the target actual image.

[0098] The preset feature point acquisition submodule 22 is used to acquire a preset BIM model corresponding to the target building object, and to acquire preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector.

[0099] The matching result acquisition submodule 23 is used to use the FLANN matcher to match the target feature vector and the preset feature vector to obtain the matching result.

[0100] The target position vector acquisition submodule 24 is used to obtain the target position vector and target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model using the PnP algorithm based on the matching result.

[0101] The intermediate virtual camera acquisition submodule 25 is used to adjust the position and direction of the initial virtual camera based on the target position vector and the target posture rotation matrix, obtain the adjusted initial virtual camera and use the adjusted initial virtual camera as the intermediate virtual camera.

[0102] The target virtual camera acquisition submodule 26 is used to obtain the viewing angle information of the target actual image, and adjust the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, obtain the adjusted intermediate virtual camera and use the adjusted intermediate virtual camera as the target virtual camera.

[0103] The target model viewing angle submodule 27 is used to generate a target model viewing angle based on the target actual image and the target virtual camera.

[0104] The target feature point acquisition submodule 21 includes:

[0105] The candidate key point acquisition unit is used to perform a convolution operation on the target actual image using a Gaussian kernel with several standard deviations, so as to obtain candidate key points.

[0106] The target feature point acquisition unit is used to screen the candidate key points and acquire the target feature points.

[0107] The main direction acquisition unit is used to acquire the gradient direction and amplitude of the target feature point, and acquire the main direction of the target feature point based on the gradient direction of the target feature point.

[0108] The target feature vector acquisition unit is used to generate a target feature vector corresponding to the target feature point based on the main direction of the target feature point.

[0109] Specifically, the matching result acquisition submodule 23 includes:

[0110] The matching unit is used to match the target feature vector with the preset feature vector using the FLANN matcher.

[0111] The matching result obtaining unit is used to take the combination of the target feature point and the preset feature point as the matching result if the target feature vector and the preset feature vector are matched successfully.

[0112] Specifically, the target model module 20 further includes a first check submodule, and the first check submodule includes:

[0113] The target feature point quantity acquisition unit is used to acquire the number of successfully matched target feature points.

[0114] The target matching rate acquisition unit is used to acquire the target matching rate based on the number of successfully matched target feature points and the total number of target feature points.

[0115] The first verification unit is used to verify the target model perspective based on the target matching rate and the preset matching threshold, wherein if the target matching rate is greater than the preset matching threshold, it is considered that the verification is successful.

[0116] Specifically, the target model module 20 further includes a second syndrome submodule, and the second syndrome submodule includes:

[0117] A position information acquisition unit, used to acquire the position information of the target feature point in the matching result and the position information of the designated feature point corresponding to the target feature point in the target model viewing angle;

[0118] The position change amount acquisition unit is used to acquire the position change amount of the target feature point based on the position information of the target feature point and the position information of the designated feature point.

[0119] The average change acquisition unit is used to average all position changes to obtain the average change.

[0120] The similarity acquisition unit is used to obtain the similarity between the target actual image and the target model perspective using the SSIM index.

[0121] The second verification unit is used to verify the target model perspective based on the average change amount and the similarity.

[0122] Specifically, the second verification unit includes:

[0123] The preset feature point tolerance value acquisition subunit is used to obtain the preset feature point tolerance value. If the average change is not greater than the preset feature point tolerance value, the verification is considered successful, otherwise the verification is considered failed.

[0124] The preset overlapping region threshold value obtaining subunit is used to obtain the preset overlapping region threshold value. If the similarity is not less than the preset overlapping region threshold value, the verification is considered successful, otherwise the verification is considered failed.

[0125] Embodiment 3,

[0126] Embodiment 3 of the present invention provides a non-transitory computer-readable storage medium, in which at least one instruction or at least one program is stored, and the at least one instruction or at least one program is loaded and executed by a processor to implement any method content in embodiment 1.

[0127] Embodiment 4:

[0128] Embodiment 4 of the present invention further provides an electronic device, comprising a processor and the non-transitory computer-readable storage medium described in the aforementioned embodiment 3.

[0129] Although some specific embodiments of the present invention have been described in detail by way of example, it should be understood by those skilled in the art that the above examples are only for illustration, not for limiting the scope of the present invention. It should also be understood by those skilled in the art that various modifications may be made to the embodiments without departing from the scope and spirit of the present invention.

Claims

1. A digital twin space viewpoint positioning method, characterized in that: The method comprises the following steps: Acquire a target actual image list of the target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object; For a target actual image, obtaining a target model perspective corresponding to the target actual image, thereby generating a target model; Wherein, for a target actual image, obtaining the target model perspective corresponding to the target actual image comprises the following steps: For the actual target image, the SIFT algorithm is used to obtain the target feature points corresponding to the actual target image and the target feature vectors corresponding to the target feature points; Obtain a preset BIM model corresponding to the target building object, and obtain preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector; Use the FLANN matcher to match the target feature vector with the preset feature vector to obtain the matching result; Based on the matching results, the PnP algorithm is used to obtain the target position vector and target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model; Based on the target position vector and the target attitude rotation matrix, the position and direction of the initial virtual camera are adjusted, and the adjusted initial virtual camera is obtained and used as the intermediate virtual camera; Obtaining the viewing angle information of the target actual image, and adjusting the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, obtaining the adjusted intermediate virtual camera and using the adjusted intermediate virtual camera as the target virtual camera; Based on the target actual image and the target virtual camera, the target model perspective is generated.

2. The digital twin space viewpoint positioning method according to claim 1, characterized in that: For the actual target image, the target feature points corresponding to the actual target image and the target feature vectors corresponding to the target feature points are obtained using the SIFT algorithm, including: For the actual target image, a convolution operation is performed using a Gaussian kernel with several standard deviations to obtain candidate key points; Screen candidate key points and obtain target feature points; Obtaining the gradient direction of the target feature point, and obtaining the main direction of the target feature point based on the gradient direction of the target feature point; Based on the main direction of the target feature point, a target feature vector corresponding to the target feature point is generated.

3. The digital twin space viewpoint positioning method according to claim 1, characterized in that: Obtaining preset feature points in the preset BIM model includes: Use BIM software to load the preset BIM model corresponding to the target building object, and use the feature point tool in the BIM software to extract the preset feature points.

4. The digital twin space viewpoint positioning method according to claim 1, characterized in that: The dimension of the target feature vector is 128.

5. The digital twin space viewpoint positioning method according to claim 1, characterized in that: Use the FLANN matcher to match the target feature vector with the preset feature vector to obtain the matching results, including: Use FLANN matcher to match the target feature vector with the preset feature vector; If the target feature vector and the preset feature vector match successfully, the combination of the target feature point and the preset feature point is taken as the matching result.

6. The digital twin space viewpoint positioning method according to claim 5, characterized in that: After generating the target model perspective, it also includes: Get the number of successfully matched target feature points; Obtaining a target matching rate based on the number of successfully matched target feature points and the total number of target feature points; Based on the target matching rate and the preset matching threshold, the target model perspective is verified, wherein if the target matching rate is greater than the preset matching threshold, the verification is considered successful.

7. The digital twin space viewpoint positioning method according to claim 5, characterized in that: After generating the target model perspective, it also includes: Obtain the location information of the target feature point in the matching result and the location information of the specified feature point corresponding to the target feature point in the target model perspective; Based on the position information of the target feature point and the position information of the designated feature point, obtaining the position change of the target feature point; Average all position changes to obtain the average change; Use the SSIM index to obtain the similarity between the actual target image and the target model perspective; Based on the average change and similarity, the target model perspective is verified.

8. A digital twin space viewpoint positioning device, characterized in that: The device comprises: A target actual image acquisition module, used to acquire a target actual image list of a target building object, wherein the target actual image list includes target actual images of the target building object under a plurality of preset viewing angle information, and all target actual images together can cover the external area and the internal area of ​​the entire target building object; The target model module is used to obtain a target model perspective corresponding to a target actual image, thereby generating a target model; Wherein, the target model module includes: The target feature point acquisition submodule is used to obtain the target feature points corresponding to the target actual image and the target feature vectors corresponding to the target feature points using the SIFT algorithm for the target actual image; A preset feature point acquisition submodule is used to acquire a preset BIM model corresponding to a target building object, and to acquire preset feature points in the preset BIM model and preset feature vectors corresponding to the preset feature points, wherein the dimension of the target feature vector is the same as the dimension of the preset feature vector; The matching result acquisition submodule is used to match the target feature vector and the preset feature vector using the FLANN matcher to obtain the matching result; The target position vector acquisition submodule is used to obtain the target position vector and target posture rotation matrix corresponding to the initial virtual camera in the preset BIM model using the PnP algorithm based on the matching results; The intermediate virtual camera acquisition submodule is used to adjust the position and direction of the initial virtual camera based on the target position vector and the target attitude rotation matrix, obtain the adjusted initial virtual camera and use the adjusted initial virtual camera as the intermediate virtual camera; A target virtual camera acquisition submodule is used to acquire the viewing angle information of the target actual image, and adjust the viewing angle and focal length of the intermediate virtual camera based on the viewing angle information of the target actual image, acquire the adjusted intermediate virtual camera and use the adjusted intermediate virtual camera as the target virtual camera; The target model perspective submodule is used to generate the target model perspective based on the target actual image and the target virtual camera.

9. A non-transitory computer-readable storage medium, wherein at least one instruction or at least one program is stored in the storage medium, characterized in that: The at least one instruction or the at least one program is loaded and executed by the processor to implement the digital twin spatial viewpoint positioning method as described in any one of claims 1-7.

10. An electronic device, characterized in that: The invention comprises a processor and the non-transitory computer-readable storage medium as claimed in claim 9.

Images