Method for automatically identifying house type orientation based on photographing of mobile equipment

Abstract

The invention provides a method for automatically identifying house type orientation based on mobile equipment photographing, which comprises the following steps: acquiring north-pointing angle data of a first image in a photographing process through a magnetic sensor of the mobile equipment, and finally determining the specific orientation of a whole house through the relationship between the image and depth data. And automatic collection, calculation and adjustment of house orientation data are realized. According to the method, the manual process is removed, the cost is saved, the efficiency is improved, and the deviation caused by human factors is eliminated.

Description

Technical Field

[0001] The present invention relates to the technical field of three-dimensional house modeling, and particularly to a method for automatically identifying the house orientation by taking pictures with a mobile device. Background Art

[0002] In the process of three-dimensional house reconstruction, the house type orientation is one of the important data of house information. At present, the acquisition of house orientation data mainly relies on manual operation, which has the following deficiencies: relying on manual memory, the traditional method requires the user to manually adjust the compass or rotate the angle to determine the house orientation. If the user remembers the house orientation incorrectly, it will lead to incorrect final data; insufficient accuracy, when the house orientation is not due south, due north, due east or due west, it is difficult for manual operation to accurately measure and large deviations are likely to occur; cumbersome operation, the existing technology requires additional manual intervention, the operation process is complex and depends on the user's cognitive level, increasing the time and labor costs; poor consistency, the data collected by different users or at different times may be inconsistent due to operation differences, affecting the reliability of the data. In the prior art, the house orientation is usually determined in the following two ways: after collecting color and depth data on the mobile terminal, the user manually adjusts the compass orientation to determine the house orientation; uploading the three-dimensional and color data to the background and determining the house orientation by adjusting the rotation angle similar to a compass. However, both of these two methods have obvious limitations: relying on manual operation, requiring the user to be familiar with the house orientation, and the operation process is prone to inaccurate data due to human errors; limited accuracy, for houses with non-standard orientations, it is difficult for manual operation to ensure the accuracy of the data; low efficiency, the additional manual operation increases the time cost of data collection and reduces the overall efficiency. To solve the above problems, the present invention proposes a method for automatically identifying the house type orientation based on taking pictures with a mobile device. Through the magnetometer and image data of the mobile device, the house orientation data is automatically collected, calculated and aligned, eliminating the manual intervention link and improving the accuracy of the data and the process efficiency. Summary of the Invention

[0003] The present invention aims to solve the deficiencies in the acquisition of house orientation data in the prior art and provides a method for automatically identifying the house type orientation based on taking pictures with a mobile device. This method obtains the north-pointing angle data of the first image during the shooting process through the magnetosensor of the mobile device, and finally determines the specific orientation of the entire house through the relationship between the image and the depth data, realizing the automatic collection, calculation and alignment of the house orientation data.

[0004] The technical solution adopted by the present invention to solve its technical problems is: a method for automatically identifying the house type orientation based on taking pictures with a mobile device, including the following steps:

[0005] S1 Calibrate through the magnetometer of the mobile device to obtain accurate magnetometer data;

[0006] S2 designs and runs the calibration program to calibrate the deflection angle theta between the electronic compass and the central axis of the photo taken by the mobile phone;

[0007] S3 uses the camera of the mobile device to take pictures, collect the color image and depth data of the house, and simultaneously record the deflection angle alpha of the magnetometer from the true north direction at the time of shooting 1 ;

[0008] S4 builds a three-dimensional model of the house based on the collected color images and depth data;

[0009] S5 is based on the deflection angle alpha of the magnetometer from the true north direction 1 And the calibrated deflection angle theta, calculate the rotation matrix R' and R used to correct the orientation;

[0010] S6 applies the rotation matrices R' and R to correct the coordinates of each point P (x, y, z) in the three-dimensional model to obtain a corrected new coordinate P' (x, y, z);

[0011] S7 automatically adjusts the orientation of the three-dimensional model based on the corrected coordinates P'(x, y, z) so that it faces due north;

[0012] When displaying a 3D model, S8 automatically generates and displays an icon indicating the true north direction.

[0013] Preferably, the step of calibrating by means of a magnetometer of the mobile device comprises:

[0014] S11 opens the compass tool on the mobile device and uses GPS information to correct data for different regions;

[0015] Place the S12 mobile device flat in your hand with the screen facing upwards, and move it around the axis perpendicular to the screen until the device prompts that the calibration is complete.

[0016] Preferably, the calibration procedure comprises the following steps:

[0017] S21 runs the calibration program on the mobile device, and displays the color image window captured by the camera and the electronic compass interface;

[0018] The S22 user points the phone at a marker known to be oriented toward true north, so that the edge of the marker coincides with the center line of the image window;

[0019] S23 records the deflection angle theta displayed by the electronic compass at this time and stores it in the system.

[0020] Preferably, the step of calling the camera of the mobile device to take a photo includes:

[0021] S31 captures the first color image C 1 At the same time, the deflection angle alpha of the magnetometer from the true north direction is recorded 1 ;

[0022] S32 collects a number of color images and corresponding depth data, point information and image angle information.

[0023] Preferably, the step of establishing the three-dimensional model of the house includes one of the following methods:

[0024] The depth camera and color camera on the mobile device are used to collect data and build a three-dimensional model.

[0025] Process image data through artificial intelligence algorithms to predict and generate three-dimensional models;

[0026] The 3D model is generated by manually pulling the 3D frame and filling the surface.

[0027] Preferably, the calculation steps of the rotation matrices R' and R include:

[0028] The deflection angle alpha of the magnetometer from the true north direction 1 , calculate the rotation matrix R':

[0029]

[0030] According to the calibrated deflection angle theta, calculate the rotation matrix R:

[0031]

[0032] Preferably, the coordinate correction step includes:

[0033] For each point P(x,y,z) in the 3D model, multiply it by the rotation matrices R' and R in turn to get the corrected new coordinates P'(x,y,z)

[0034] P'(x,y,z)=P(x,y,z)*R'*R

[0035] Preferably, the step of automatically correcting the orientation of the three-dimensional model comprises:

[0036] S81 generates a three-dimensional model facing due north based on the corrected coordinates P'(x, y, z);

[0037] When displaying a three-dimensional model, S82 automatically generates and displays an indicator icon for the true north direction.

[0038] Positive effects of the present invention:

[0039] 1. The present invention significantly improves the accuracy and efficiency of house orientation data through an automated data collection, calculation, and model alignment process, and has the following advantages: Automatically collect orientation information using the magnetometer, camera, and sensors of a mobile device without manual intervention, reducing errors and time costs; Through the magnetometer calibration and calibration program, accurately measure the deflection angle theta between the central axis of the camera and the due north direction, and combine it with the magnetometer deflection angle alpha to correct the 3D model using the rotation matrix to ensure data accuracy; Users only need to perform simple operations (such as aligning with a landmark and taking a photo), reducing the usage threshold; The automated process eliminates the manual adjustment step, improving the modeling efficiency and is suitable for large-scale house information collection; Automatically generate a due north direction indicator icon when displaying the 3D model, enhancing the user experience; Support multiple modeling methods (depth camera, AI algorithm, or manual method), with high flexibility; Eliminate human errors and ensure data reliability; Reduce the dependence on professionals and lower costs; The data can be stored on the mobile device or in the cloud, supporting multi-user collaboration and sharing; Suitable for fields such as house modeling, interior design, urban planning, VR / AR, etc., with wide applications; Ensure data consistency and avoid differences in manual operations; Combine multi-source data to improve the robustness of the system in complex environments; Support real-time processing and feedback, allowing users to adjust the device position in a timely manner; Reduce resource consumption, in line with the concept of green environmental protection; Promote the intelligent development of the house information collection and 3D modeling fields, providing technical support for smart cities, smart homes, etc. Through technological innovation and an automated process, the present invention significantly improves data accuracy, efficiency, and user experience, while reducing costs, and has broad application prospects and technological innovation value. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 is a flowchart of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0041] The preferred embodiments of the present invention will be described in detail below with reference to the accompanying drawings.

[0042] See Figure 1 As shown, the preferred embodiment of the present invention provides a method for automatically identifying the house type orientation based on mobile device photography, including the following steps:

[0043] S1 Calibrate through the magnetometer of the mobile device to obtain accurate magnetometer data;

[0044] The user opens the compass tool of the mobile device, and the system automatically corrects the geomagnetic field data in different regions in combination with GPS information to ensure the accuracy of the magnetometer; The system prompts the user to perform device calibration to ensure the reliability of the magnetometer data.

[0045] The user places the mobile device flat in the hand, with the screen facing up, and makes an axis - around movement (i.e., an "8" - shaped movement) in a direction perpendicular to the screen; the system detects the movement trajectory of the device in real - time. When the device prompts "Calibration completed", the magnetometer data has been calibrated.

[0046] After calibration, the system records the current magnetometer data as the benchmark for subsequent calculations.

[0047] S2 designs and runs a calibration program to calibrate the deflection angle theta between the electronic compass and the central axis of the captured photo when the mobile phone takes a photo;

[0048] Running the calibration program:

[0049] The user runs the calibration program on the mobile device. The program interface is divided into a color - image window that displays the real - time image captured by the camera, and an electronic - compass interface that shows the deflection angle of the current device relative to the due - north direction.

[0050] The system prompts the user to select a landmark known to face due north (such as a building, a signboard, etc.).

[0051] Aligning with the landmark:

[0052] The user aligns the mobile phone with the landmark, ensuring that the edge of the landmark coincides with the mid - line of the image window. The system reads the deflection angle θ (i.e., the deflection angle between the central axis of the camera and the due - north direction) shown on the electronic compass in real - time.

[0053] Recording the deflection angle:

[0054] After the user confirms the alignment of the landmark, the system records the deflection angle θ at this time and stores it in the system; this step is used to eliminate the deflection error of the device itself and ensure the matching between the shooting angle and the magnetometer data.

[0055] S3 calls the camera of the mobile device to take a photo, collects the color image and depth data of the house, and simultaneously records the deflection angle alpha of the magnetometer deviating from the due - north direction at the shooting moment 1 ;

[0056] Taking the first image:

[0057] The user calls the camera of the mobile device to take the first color image c of the house 1 , and the system simultaneously records the deflection angle alpha of the magnetometer deviating from the due - north direction at the shooting moment 1 , the image C 1 and the deflection angle alpha 1 will be used as the benchmark data for subsequent calculations.

[0058] Collecting multiple images:

[0059] The user continues to take several color images, covering all angles of the house; the system synchronously collects the depth data, point position information, and image angle information of each image; all the collected data (images, depth data, point position information, angle information) will be used for subsequent 3D model building.

[0060] S4 Based on the collected color images and depth data, build a 3D model of the house;

[0061] Data collection method:

[0062] The system supports the following three methods to build a 3D model of the house:

[0063] Method 1, depth camera: Use the depth camera and color camera built into the mobile device to collect data and generate a 3D model.

[0064] Method 2, artificial intelligence algorithm: Process the image data through an artificial intelligence algorithm to predict and generate a 3D model.

[0065] Method 3, manual method: Pull the 3D framework manually and fill in the faces to generate a 3D model.

[0066] The user can select a suitable method according to the device performance and requirements.

[0067] Model generation:

[0068] The system automatically generates a 3D model of the house based on the collected color images and depth data.

[0069] The generated 3D model contains the structural information of the house (such as walls, doors, and windows) and spatial information (such as room size, layout, etc.).

[0070] S5 According to the deflection angle alpha of the magnetometer deviating from the due north direction 1 and the calibrated deflection angle theta, calculate the rotation matrices R' and R for correcting the orientation;

[0071] According to the deflection angle alpha of the magnetometer deviating from the due north direction 1 , calculate the rotation matrix r':

[0072]

[0073] According to the calibrated deflection angle theta, calculate the rotation matrix R:

[0074]

[0075] This matrix is used to eliminate the deflection error of the device itself and ensure the accuracy of the model orientation.

[0076] S6 applies the rotation matrices R' and R to correct the coordinates of each point P (x, y, z) in the three-dimensional model to obtain a corrected new coordinate P' (x, y, z);

[0077] For each point P(x,y,z) in the 3D model, multiply it by the rotation matrices R' and R in turn to get the corrected new coordinates P'(x,y,z)

[0078] P'(x,y,z)=P(x,y,z)*R'*R.

[0079] This step ensures that the 3D model is oriented to true north.

[0080] Correct model orientation:

[0081] The system automatically generates a three-dimensional model facing due north based on the corrected coordinates P'(x, y, z); the corrected model will be used for subsequent display and application.

[0082] S7 automatically adjusts the orientation of the three-dimensional model based on the corrected coordinates P'(x, y, z) so that it faces due north;

[0083] Generate a true north model:

[0084] Based on the corrected coordinates P'(x,y,z), the system automatically generates a three-dimensional model facing due north.

[0085] Users can view the adjusted 3D model on their device to ensure the accuracy of the house's orientation.

[0086] Display true north icon:

[0087] When displaying a 3D model, the system automatically generates and displays an indicator icon for the true north direction.

[0088] Users can intuitively understand the orientation of the house through icons without manual adjustment.

[0089] When displaying a 3D model, S8 automatically generates and displays an icon indicating the true north direction.

[0090] Data Storage:

[0091] The acquired image data, depth data, orientation information and 3D models can be stored on mobile devices or in the cloud.

[0092] The system supports multi-user collaboration and sharing, and users can access and edit data at any time.

[0093] Data transmission:

[0094] Users can transfer data to other devices or platforms via the network for further analysis or application.

[0095] The system supports real-time processing and feedback, allowing users to adjust the device position or re-collect data in a timely manner.

[0096] In summary, through the automated data collection, calculation, and model alignment processes, the present invention significantly improves the accuracy and efficiency of house orientation data. By leveraging the collaborative work of the magnetometer, camera, and depth sensor of a mobile device, the system can automatically collect, calculate, and align the house orientation without manual intervention, reducing human errors and time costs. Through the magnetometer calibration and calibration procedures, the deflection angle between the central axis of the camera and the true north direction is accurately measured, and the three-dimensional model is corrected by combining the magnetometer deflection angle and the rotation matrix to ensure the accuracy of the data. Users only need to perform simple operations (such as aligning with markers and taking photos), and the system automatically completes the subsequent processing, reducing the usage threshold. In addition, the system automatically generates a true north direction indicator icon when displaying the three-dimensional model, enhancing the user experience. The present invention is applicable to fields such as house modeling, interior design, urban planning, VR / AR, etc., and has broad application prospects. At the same time, the automated process eliminates errors caused by manual operations, improves the consistency and reliability of data, and promotes the intelligent development of the field of house information collection and three-dimensional modeling.

[0097] The above are only the preferred embodiments of the present invention. It should be understood that the description of the above embodiments is only used to help understand the method and its core idea of the present invention, and is not used to limit the protection scope of the present invention. Any modifications, equivalent replacements, etc. made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for automatically identifying the orientation of a house based on taking photos with a mobile device, characterized in that: The steps include: S1 is calibrated through the magnetometer of the mobile device to obtain accurate magnetometer data; S2 designs and runs the calibration program to calibrate the deflection angle theta between the electronic compass and the central axis of the photo taken by the mobile phone; S3 calls the camera of the mobile device to take pictures, collects the color image and depth data of the house, and simultaneously records the deflection angle alpha1 of the magnetometer from the true north direction at the time of shooting; S4 builds a three-dimensional model of the house based on the collected color images and depth data; S5 calculates the rotation matrices R' and R used to correct the orientation according to the deflection angle alpha1 of the magnetometer from the true north direction and the calibrated deflection angle theta; S6 applies the rotation matrices R' and R to correct the coordinates of each point P (x, y, z) in the three-dimensional model to obtain a corrected new coordinate P' (x, y, z); S7 automatically adjusts the orientation of the three-dimensional model based on the corrected coordinates P'(x, y, z) so that it faces due north; When displaying a 3D model, S8 automatically generates and displays an icon indicating the true north direction.

2. According to claim 1, a method for automatically identifying the orientation of a house based on taking photos with a mobile device is characterized in that: The step of calibrating the magnetometer of the mobile device comprises: S11 opens the compass tool on the mobile device and uses GPS information to correct data for different regions; Place the S12 mobile device flat in your hand with the screen facing upwards, and move it around the axis perpendicular to the screen until the device prompts that the calibration is complete.

3. The method of automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The calibration procedure comprises the following steps: S21 runs the calibration program on the mobile device, and displays the color image window captured by the camera and the electronic compass interface; The S22 user points the phone at a marker known to be oriented toward true north, so that the edge of the marker coincides with the center line of the image window; S23 records the deflection angle theta displayed by the electronic compass at this time and stores it in the system.

4. The method for automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The step of calling the camera of the mobile device to take a photo includes: S31 records the deflection angle alpha1 of the magnetometer from the true north direction while taking the first color image C1; S32 collects a number of color images and corresponding depth data, point information and image angle information.

5. The method for automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The step of establishing the three-dimensional model of the house comprises one of the following methods: The depth camera and color camera on the mobile device are used to collect data and build a three-dimensional model. Process image data through artificial intelligence algorithms to predict and generate three-dimensional models; The 3D model is generated by manually pulling the 3D frame and filling the surface.

6. The method for automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The calculation steps of the rotation matrices R' and R include: According to the deflection angle alpha1 of the magnetometer from the true north direction, calculate the rotation matrix R': According to the calibrated deflection angle theta, calculate the rotation matrix R:

7. The method of automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The coordinate correction step comprises: For each point P(x,y,z) in the 3D model, multiply it by the rotation matrices R' and R in turn to get the corrected new coordinates P'(x,y,z) P'(x,y,z)=P(x,y,z)*R'*R 8. The method for automatically identifying the orientation of a house based on taking photos with a mobile device according to claim 1, characterized in that: The step of automatically correcting the orientation of the three-dimensional model comprises: S81 generates a three-dimensional model facing due north based on the corrected coordinates P'(x, y, z); When displaying a three-dimensional model, S82 automatically generates and displays an indicator icon for the true north direction.

Images