AR-based real estate intelligent surveying and mapping method and device, medium and electronic equipment

Abstract

The invention relates to an AR-based real estate intelligent surveying and mapping method and device, a medium and electronic equipment, and relates to the technical field of space surveying and mapping, and the method comprises the steps: obtaining a suitable space region suitable for surveying and mapping in a target real estate at the current time; collecting target three-dimensional space data corresponding to the suitable space area through the AR equipment, and generating a target three-dimensional model corresponding to the suitable space area through a preset SLAM algorithm based on the target three-dimensional space data; the target three-dimensional model and the real physical environment in the suitable space area are displayed in an overlapped mode, and an augmented reality result is obtained; and on the basis of an augmented reality result, calibrating the target three-dimensional model and labeling indoor ground features, and repeatedly executing the step of obtaining the suitable space region suitable for surveying and mapping in the target real estate at the current time until all the space regions in the target real estate are surveyed and mapped. The method has the effect of improving the real estate surveying and mapping accuracy.

Description

Technical Field

[0001] The present application relates to the field of spatial mapping technology, and specifically to an AR-based real estate intelligent mapping method, device, medium and electronic equipment. Background Art

[0002] Augmented Reality (AR) technology integrates the computer-generated virtual environment with the real environment around the user by means of optoelectronic display technology, interactive technology, multiple sensor technologies, and computer graphics and multimedia technologies, so that the user can be convinced from the sensory effect that the virtual environment is a part of the real environment around him. Augmented reality has the new characteristics of combining virtual and real, real-time interaction, and three-dimensional registration. Real estate refers to those properties that are fixed to the land and cannot be easily moved, or once moved, their nature will change and their value will be damaged. These properties usually include land and the buildings and other fixtures on it. In addition, real estate surveying and mapping refers to the process of measuring and mapping the geographical location, shape, size and spatial relationship of the land and the buildings, structures and other fixtures on it.

[0003] At present, the usual method for real estate surveying and mapping is as follows: traditional real estate surveying and mapping work is done by professional surveying and mapping personnel going to the site to conduct measurements, and based on the measured data, the subsequent manual drawing or the use of CAD drawing tools to finally complete the surveying and mapping. In this process, on-site surveying and mapping by personnel usually uses drone aerial photography. Once the structure of the indoor space to be surveyed is more complex, the positioning accuracy is low under this method, resulting in low accuracy of real estate surveying and mapping. Summary of the invention

[0004] In order to improve the accuracy of real estate surveying and mapping, the present application provides an AR-based real estate intelligent surveying and mapping method, device, medium and electronic equipment.

[0005] In a first aspect of the present application, a real estate intelligent surveying and mapping method based on AR is provided, which specifically includes: Acquire a suitable spatial area suitable for surveying and mapping in the target real estate at the current time, wherein the target real estate contains at least one spatial area that needs to be surveyed and mapped, and the suitable area is a spatial area where the light has the least effect on the depth camera collecting three-dimensional spatial data; The target three-dimensional spatial data corresponding to the suitable spatial area is collected by the AR device, and based on the target three-dimensional spatial data, a target three-dimensional model corresponding to the suitable spatial area is generated by a preset SLAM algorithm; The target three-dimensional model is superimposed and displayed with the real physical environment in the suitable space area to obtain an augmented reality result; Based on the augmented reality result, the target three-dimensional model is calibrated and indoor objects are annotated, and the step of obtaining suitable spatial areas in the target real estate at the current time that are suitable for surveying and mapping is repeated until the surveying and mapping of all spatial areas in the target real estate is completed.

[0006] By adopting the above technical solution, the suitable spatial area with the least impact on the current acquisition of three-dimensional spatial data is determined, so that the subsequent surveying and mapping of the suitable spatial area is more accurate. Furthermore, the target three-dimensional space of this suitable spatial area is collected through the AR device, and then the target three-dimensional model corresponding to the suitable spatial area is generated. Then, by utilizing the virtual-real fusion characteristics of the AR device, the target three-dimensional model is superimposed and displayed with the real physical environment in the suitable spatial area, so that the wearer of the AR device can more intuitively observe the fit between the target three-dimensional model and the real physical environment, so as to intuitively check and mark the target three-dimensional model, so that the target three-dimensional model obtained by the current surveying and mapping has a higher accuracy. Finally, repeat the steps of obtaining the suitable spatial area suitable for surveying and mapping in the target real estate at the current time, and determine the next spatial area that is more suitable for surveying and mapping, so as to improve the accuracy of real estate surveying and mapping.

[0007] Optionally, the obtaining of a suitable spatial area in the target real estate at the current time that is suitable for surveying and mapping specifically includes: Counting the number of first occurrences of a target area in each spatial area of ​​the target real estate, the target area being a historical area where the light intensity exceeds a preset intensity threshold; Determine the historical period in which the target area appears in each of the spatial regions, count the second occurrence times of each of the historical periods, and select the first historical period from each of the historical periods in ascending order of the second occurrence times as the important illumination period of the corresponding spatial region; Calculating a first weight of each of the spatial regions and a second weight of each of the corresponding important illumination periods, wherein the first weight is a ratio of a first occurrence number of each spatial region to a sum of first occurrence numbers of all spatial regions, and the second weight is a ratio of a second occurrence number of a single important illumination period corresponding to the spatial region to a sum of second occurrence numbers of all corresponding important illumination periods; Determining a selected spatial region currently being surveyed and mapped from each of the spatial regions according to the first weight and the corresponding second weights; A suitable spatial area suitable for surveying and mapping in the target real estate at the current time is determined from each of the candidate spatial areas.

[0008] By adopting the above technical solution, the more the first occurrence times, the more likely the corresponding spatial area is to have an area with high light intensity, and the greater the second occurrence times, the more likely the single spatial area is to have the target area in the corresponding historical period, thereby determining the important light period corresponding to the spatial area. Finally, combining the first weight and the corresponding second weights, the possibility of each spatial area having the target area at present is analyzed, so as to facilitate the subsequent accurate determination of the area currently suitable for surveying and mapping, and ensure the accuracy of spatial area surveying and mapping in the target real estate.

[0009] Optionally, determining the selected spatial region for current surveying and mapping from the spatial regions according to the first weight and the corresponding second weights specifically includes: Determine the spatial region including the current time in each corresponding important illumination period as the target spatial region, determine the important illumination period including the current time as the target illumination period, and calculate the first product of the first weight of each target spatial region and the second weight of the corresponding target illumination period; Determine a correction coefficient corresponding to each of the target spatial regions, and multiply each of the first products by the corresponding correction coefficient to obtain a corresponding first product correction result; Comparing each of the first product correction results with a preset product threshold; If the first product correction result is smaller than the product threshold, the corresponding target space area is determined as the space area to be selected.

[0010] By adopting the above technical solution, the larger the first product is, the greater the possibility that the target area will appear in the target space area during the corresponding target lighting period. Then, the first product correction result is obtained by correcting it through the corresponding correction coefficient. If the first product correction result is less than the preset product threshold, it means that the possibility that the target area will appear in the corresponding target space area after the current time is small, and the impact on the depth camera's acquisition of three-dimensional spatial data is small. Then, the corresponding target space area is determined as the selected space area, so as to facilitate the subsequent screening of suitable space areas with higher accuracy in the current acquisition of three-dimensional spatial data.

[0011] Optionally, determining the correction coefficient corresponding to each target space area includes: Calculating the weight product of the first weight of each target spatial area and the second weight of each corresponding important illumination period and summing them up to obtain the sum of the weight products; According to the sum of the weight products, a correction coefficient of the corresponding target space area is determined. The larger the sum of the weight products is, the larger the corresponding correction coefficient is, and the correction coefficient is greater than 1.

[0012] By adopting the above technical solution, the larger the sum of the weighted products is, the greater the possibility that the target area will appear in the corresponding target space area as a whole, and then the correction coefficient of the first product corresponding to each target space area is determined, and the first product is corrected and optimized by the correction coefficient, so that the corrected result more objectively reflects the possibility of the target area appearing in the corresponding target lighting period.

[0013] Optionally, determining a suitable spatial area suitable for surveying and mapping in the target real estate at the current time from each of the candidate spatial areas specifically includes: Counting the third occurrence times of each target area in a single important illumination period of the selected spatial area, and selecting the second number of target areas from each target area in descending order of the third occurrence times as the easy illumination area of ​​the corresponding important illumination period; Calculate the third weight of each of the important illumination periods and the fourth weight of each of the corresponding easy illumination areas, wherein the third weight is the ratio of the second occurrence number of a single important illumination period corresponding to the selected spatial area to the sum of the second occurrence numbers of all corresponding important illumination periods, and the fourth weight is the ratio of the third occurrence number of a single easy illumination area corresponding to the important illumination period to the sum of the third occurrence numbers of all corresponding easy illumination areas; According to the third weight of each of the candidate spatial regions and the corresponding fourth weights, a suitable spatial region suitable for surveying and mapping in the target real estate at the current time is determined from each of the candidate spatial regions.

[0014] By adopting the above technical solution, the greater the number of third occurrences, the more likely it is that the corresponding target area of ​​the selected spatial area will have a strong light intensity during the important lighting period, thereby determining the easy-to-light area corresponding to the important lighting period; finally, combining the third weight of each selected spatial area and the corresponding fourth weights, analyze and determine the situation where the light intensity in each current selected spatial area is too high, and then determine the suitable spatial area that is least affected by the light, so that the collected three-dimensional spatial data is more accurate.

[0015] Optionally, determining a suitable spatial area in the target real estate at the current time that is suitable for surveying and mapping from each of the candidate spatial areas according to the third weights of each of the candidate spatial areas and the corresponding fourth weights specifically includes: For a single selected spatial area, an important lighting period in which at least one key area exists in each corresponding easy lighting area is determined as a key lighting period, and a second product of the third weight of each key lighting period and the fourth weight of each corresponding key area is calculated and summed to obtain the sum of the first products of the corresponding key lighting period; Calculate the ratio of the sum of the first products of each of the key lighting periods of the same to-be-selected spatial area to the cumulative sum of the first products of the remaining key lighting periods, and select the minimum ratio from the ratios to determine as the selection ratio of the corresponding to-be-selected spatial area; When the current time is included in the key lighting time periods corresponding to all the selection ratios, the minimum selection ratio is selected from the selection ratios, and the selected spatial area corresponding to the minimum selection ratio is determined as the suitable spatial area in the target real estate suitable for surveying and mapping at the current time.

[0016] By adopting the above technical solution, the larger the ratio, the greater the relative possibility that the light intensity in the key area is too high in a single key light period of the same candidate spatial area compared to the remaining key light periods, then the minimum ratio is selected from the ratios corresponding to the single candidate spatial area, and the possibility that the light intensity in the key area is too high in the key light period corresponding to the minimum ratio is smaller than that in the remaining key light periods; finally, the candidate spatial area corresponding to the minimum selection ratio is determined as the suitable spatial area suitable for surveying and mapping in the target real estate at the current time. Therefore, when the suitable spatial area is currently surveyed and mapped, it is less affected by the excessive light intensity in the key area, and the obtained three-dimensional model is more accurate.

[0017] Optionally, the method further includes: Calculate the third product of the third weight of the important illumination period of the suitable spatial area including the current time and the fourth weight of each corresponding easy illumination area; summing the third products to obtain a sum of the corresponding second products; Calculate a fourth product of a third weight of the important illumination period of the current time in the remaining spatial area and a fourth weight of each corresponding easy illumination area, wherein the remaining spatial area is a spatial area other than the suitable spatial area; The fourth products are summed to obtain the corresponding sum of the third products. If the sum of the second products is less than the sum of the third products, it is determined that the appropriate spatial area has been verified correctly.

[0018] By adopting the above technical solution, the larger the sum of the second products, the greater the possibility that there is currently an area with too high light intensity in the suitable space area. The larger the sum of the third products, the greater the possibility that there is currently an area with too high light intensity in the remaining space area. If the sum of the second products is less than the sum of the third products, it means that the possibility that there will be an area with too high light intensity in the suitable space area is the smallest, so the suitable space area is verified again to confirm that it is correct. This makes the three-dimensional model of the suitable space area currently being surveyed more accurate.

[0019] In a second aspect of the present application, an AR-based real estate intelligent surveying and mapping device is provided, specifically comprising: An information acquisition module, used to acquire a suitable spatial area suitable for surveying and mapping in a target real estate at a current time, wherein the target real estate contains at least one spatial area that needs to be surveyed and mapped, and the suitable area is a spatial area where the light has the least effect on the depth camera collecting three-dimensional spatial data; A model building module, used to collect target three-dimensional spatial data corresponding to the suitable spatial area through the AR device, and generate a target three-dimensional model corresponding to the suitable spatial area through a preset SLAM algorithm based on the target three-dimensional spatial data; A superimposed display module, used for superimposing and displaying the target three-dimensional model and the real physical environment in the suitable space area to obtain an augmented reality result; The spatial mapping module is used to calibrate the target three-dimensional model and annotate indoor objects based on the augmented reality result, and repeatedly execute the step of obtaining suitable spatial areas suitable for mapping in the target real estate at the current time until the mapping of all spatial areas in the target real estate is completed.

[0020] By adopting the above technical solution, the information acquisition module obtains the appropriate spatial area, the model construction module generates the corresponding target three-dimensional model based on the target three-dimensional spatial data, and then the overlay display module overlays the target three-dimensional model with the real physical environment to obtain an augmented reality result. Finally, the spatial mapping module calibrates the target three-dimensional model and marks indoor objects.

[0021] In a third aspect of the present application, a computer-readable storage medium is provided, in which a computer program is stored. When the computer program is loaded and executed by a processor, the method steps described in any one of the first aspects are performed.

[0022] In a fourth aspect of the present application, an electronic device is provided, specifically comprising: A processor, a memory, and a computer program stored in the memory and capable of running on the processor, wherein the processor is used to load and execute the computer program stored in the memory so that the electronic device performs the method as described in any one of the first aspects.

[0023] In summary, the present application includes at least one of the following beneficial technical effects: determining a suitable spatial area with the least impact on the current acquisition of three-dimensional spatial data, so that the subsequent surveying and mapping of the suitable spatial area is more accurate. Furthermore, the target three-dimensional space of this suitable spatial area is collected by an AR device, and then a target three-dimensional model corresponding to the suitable spatial area is generated. Then, by utilizing the virtual-real fusion characteristics of the AR device, the target three-dimensional model is superimposed and displayed with the real physical environment in the suitable spatial area, so that the wearer of the AR device can more intuitively observe the fit between the target three-dimensional model and the real physical environment, so as to intuitively check and mark the target three-dimensional model, so that the target three-dimensional model obtained by the current surveying and mapping has a higher accuracy. Finally, repeat the steps of obtaining a suitable spatial area suitable for surveying and mapping in the target real estate at the current time, and determine the next spatial area that is more suitable for surveying and mapping, thereby improving the accuracy of real estate surveying and mapping. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1 It is a flowchart of an AR-based real estate intelligent surveying and mapping method provided in an embodiment of the present application; Figure 2 It is a flowchart of another AR-based real estate intelligent surveying and mapping method provided in an embodiment of the present application; Figure 3 It is a structural schematic diagram of an AR-based real estate intelligent surveying and mapping device provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of another AR-based real estate intelligent surveying and mapping device provided in an embodiment of the present application.

[0025] Explanation of the accompanying drawings: 11. Information acquisition module; 12. Model building module; 13. Overlay display module; 14. Space mapping module; 15. Area verification module. DETAILED DESCRIPTION

[0026] In order to enable those skilled in the art to better understand the technical solutions in this specification, the technical solutions in the embodiments of this specification will be clearly and completely described below in conjunction with the drawings in the embodiments of this specification. Obviously, the described embodiments are only part of the embodiments of this application, not all of the embodiments.

[0027] In the description of the embodiments of the present application, words such as "illustrative", "for example" or "for example" are used to indicate examples, illustrations or descriptions. Any embodiment or design described as "illustrative", "for example" or "for example" in the embodiments of the present application should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of words such as "illustrative", "for example" or "for example" is intended to present related concepts in a concrete way.

[0028] In the description of the embodiments of the present application, the term "and / or" is only a kind of association relationship describing the associated objects, indicating that there may be three kinds of relationships, for example, A and / or B, which can represent: A exists alone, B exists alone, and A and B exist at the same time. In addition, unless otherwise specified, the meaning of the term "multiple" refers to two or more. For example, multiple systems refer to two or more systems, and multiple screen terminals refer to two or more screen terminals. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the indicated technical features. Thus, the features defined as "first" and "second" can explicitly or implicitly include one or more of the features. The terms "include", "comprise", "have" and their variations all mean "including but not limited to", unless otherwise specifically emphasized.

[0029] See also Figure 1 The present application embodiment discloses a flowchart of an AR-based real estate intelligent mapping method, which can be implemented by a computer program or run on an AR-based real estate intelligent mapping device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application, specifically including: S101: Acquire a suitable spatial area suitable for surveying and mapping in the target real estate at the current time.

[0030] Specifically, in an embodiment of the present application, the current time is the time when the target real estate needs to be surveyed and mapped. The target real estate may be a commercial complex. In other embodiments, the target real estate may also be an apartment house, etc. The target real estate contains at least one spatial area that needs to be surveyed and mapped. For example, when the target real estate is an apartment, it contains multiple independent rooms that need to be surveyed and mapped, that is, spatial areas. In addition, in an embodiment of the present application, the execution subject of an AR-based real estate intelligent surveying and mapping method is an AR device, which is equipped with a depth camera. The AR device may be AR glasses. In other embodiments, the AR device may also be an AR helmet device, etc. Among them, the depth camera is a device that can obtain three-dimensional information (three-dimensional spatial data) of an object. It measures the distance of the object by emitting and receiving infrared light, thereby generating three-dimensional point cloud data, which can be used to create an accurate three-dimensional model. In the process of obtaining the three-dimensional spatial data of the object by the depth camera, it is easily affected by light. When the light intensity on the surface of the object is high, the sensor of the depth camera may receive too strong light, resulting in overexposure of the image. In this case, the details of the image will be lost, making it difficult for the depth camera to accurately identify and calculate the distance, thereby affecting the accuracy of the three-dimensional spatial data. Furthermore, the suitable spatial region is a spatial region where the influence of light on the collection of three-dimensional spatial data by the depth camera is minimal.

[0031] A feasible way to obtain a suitable spatial area is: at the current time, the surveying and mapping personnel wear an AR device, and through the cameras in each spatial area in the target real estate, obtain the real-time monitoring video of each spatial area and display it in the AR device. Based on each real-time monitoring video, the surveying and mapping personnel determine the suitable spatial area that needs to be surveyed at present, and make a selection gesture or select a voice command. After recognition, the AR device finally determines the suitable spatial area.

[0032] S102: Collect target three-dimensional spatial data corresponding to the suitable spatial area through the AR device, and generate a target three-dimensional model corresponding to the suitable spatial area through a preset SLAM algorithm based on the target three-dimensional spatial data.

[0033] Specifically, after the suitable spatial area is determined, the surveyor goes to the suitable spatial area and issues a voice command to start surveying. The AR device recognizes the voice command through a preset voice recognition module, starts the onboard depth camera to collect three-dimensional spatial data of the suitable spatial area, and obtains the corresponding target three-dimensional spatial data. Further, through a preset Simultaneous Localization and Mapping (SLAM) algorithm, a target three-dimensional model corresponding to the suitable spatial area is generated based on the target three-dimensional spatial data, wherein the SLAM algorithm is an advanced spatial positioning technology that allows a robot or device to determine its own position and construct an environmental map in an unknown environment through the collected three-dimensional spatial data. It should be noted that the target three-dimensional spatial data can be the three-dimensional spatial data collected by the surveyor in the entire space of the suitable spatial area when the surveyor is fixed at a certain position in the suitable spatial area. In other embodiments, the target three-dimensional spatial data can also be the three-dimensional spatial data collected by the surveyor when he moves in real time in the suitable spatial area.

[0034] S103: The target three-dimensional model is superimposed and displayed with the real physical environment in the appropriate space area to obtain an augmented reality result.

[0035] Specifically, the target 3D model of the suitable spatial area is obtained, and the AR device displays the target 3D model in the AR field of view that the surveyor can observe when equipped with the AR device. The AR field of view originally includes the real physical environment of the suitable spatial area. Further, the target 3D model is superimposed on the real physical environment of the suitable spatial area, thereby realizing the fusion of virtual and real, enhancing the reality, and obtaining an augmented reality result. This is a prior art and will not be elaborated here.

[0036] S104: Based on the augmented reality result, the target three-dimensional model is calibrated and indoor objects are annotated, and the step of obtaining suitable spatial areas in the target real estate at the current time that are suitable for surveying and mapping is repeated until all spatial areas in the target real estate are surveyed and mapped.

[0037] Specifically, after the augmented reality result corresponding to the suitable spatial area is determined, the surveying and mapping personnel verify the model of each element in the target three-dimensional model by the overlap between the target three-dimensional model in the AR field of view and the real physical environment. For example, if the model of the door and window elements in the target three-dimensional model has a high degree of overlap with the door and window in the real physical environment, it means that the construction of the door and window model in the target three-dimensional model is relatively accurate; if the model of the wall A in the target three-dimensional model has a poor degree of overlap with the wall A in the real physical environment, it means that the construction accuracy of the wall model in the target three-dimensional model is poor, and it is necessary to re-collect the three-dimensional spatial data in a targeted manner and regenerate the three-dimensional model corresponding to the suitable spatial area, so as to make the surveying and mapping results of the suitable spatial area more accurate. Furthermore, the surveying and mapping personnel call the built-in annotation tool in the AR glasses through gesture control to annotate the name, size information, etc. of each indoor object in the target three-dimensional model in the AR field of view. Among them, indoor objects include but are not limited to home appliances, walls, doors and windows in the suitable spatial area. Finally, after the surveying of this suitable spatial area is completed, the steps of obtaining the suitable spatial area in the target real estate at the current time that is suitable for surveying are repeated, and other spatial areas in the target real estate are surveyed until the surveying of all spatial areas in the target real estate is completed.

[0038] See also Figure 2 , the embodiment of the present application discloses a flowchart of another method for intelligent real estate mapping based on AR, which can be implemented by a computer program and can also be run on an AR-based intelligent real estate mapping device based on the von Neumann system. The computer program can be integrated into an application or run as an independent tool application, specifically including: S201: Count the number of first occurrences of the target area in each spatial area in the target real estate.

[0039] S202: Determine the historical time periods in which the target area appears in each spatial area, count the second occurrence times of each historical time period, and select the first historical time period from each historical time period in ascending order of the second occurrence times as the important illumination time period for the corresponding spatial area.

[0040] S203: Calculate the first weight of each spatial area and the second weight of each corresponding important lighting period.

[0041] Specifically, in an embodiment of the present application, the target area refers to a historical area where the light intensity exceeds a preset intensity threshold. Based on the historical monitoring records of the light intensity in each spatial area within a preset time, the historical monitoring records include but are not limited to different areas, corresponding light intensity information and detection time, and then the first occurrence number of the target area in a single spatial area where the light intensity exceeds the intensity threshold is counted. The more the first occurrence number, the more likely it is that the corresponding spatial area will have an area with higher light intensity. Further, based on the above historical monitoring records, the historical time period in which the target area appears in a single spatial area is determined, and the second occurrence number of each historical time period is counted. The larger the second occurrence number, the more likely it is that the target area appears in the corresponding historical time period in the single spatial area. Then, in order of the second occurrence number from small to large, the historical time period with the first number is selected from each historical time period to be determined as the important light time period of the corresponding spatial area, that is, the time period in which the target area is not likely to appear.

[0042] Furthermore, the first weight of each spatial area and the second weight of each corresponding important lighting period are calculated, the first weight being the ratio of the first number of occurrences of each spatial area to the sum of the first number of occurrences of all spatial areas, and the second weight being the ratio of the second number of occurrences of a single important lighting period corresponding to the spatial area to the sum of the second number of occurrences of all corresponding important lighting periods.

[0043] S204: Determine, from among the spatial regions, a candidate spatial region for current surveying and mapping according to the first weight and the corresponding second weights.

[0044] Specifically, in the embodiment of the present application, a feasible way to determine the selected spatial region is: determine the spatial region including the current time in each corresponding important illumination period as the target spatial region, and determine the important illumination period including the current time as the target illumination period. Calculate the first product of the first weight of each target spatial region and the second weight of the corresponding target illumination period, the larger the first product, the greater the possibility that the target spatial region appears in the corresponding target illumination period. Then determine the correction coefficient corresponding to each target space area. The determination process is: calculate the weight product of the first weight of each target space area and the second weight of each corresponding important lighting period and sum them up to obtain the sum of the weight products of the corresponding target space areas. The larger the sum of the weight products, the greater the possibility that the corresponding target space area will have a target area as a whole. Finally, according to the sum of the weight products, determine the correction coefficient of the corresponding target space area from a preset coefficient matching table. The larger the sum of the weight products, the larger the correction coefficient. The correction coefficient is greater than 1. The coefficient matching table includes range intervals of different weight product sums and corresponding correction coefficients. Exemplarily, the coefficient matching table includes: range interval 0-0.1, the corresponding correction coefficient is 1.1; range interval 0.1-0.2, the corresponding correction coefficient is 1.2... and so on.

[0045] Furthermore, each first product is multiplied by the corresponding correction coefficient to obtain a corresponding first product correction result. If the first product correction result is less than the preset product threshold, it means that the corresponding target space area is less likely to appear in the target area after the current time, and the impact on the depth camera's collection of three-dimensional space data is small, then the corresponding target space area is determined as the selected space area.

[0046] S205: Determine a suitable spatial area in the target real estate at the current time that is suitable for surveying and mapping from among the candidate spatial areas.

[0047] Specifically, after the selected spatial area is determined, the suitable spatial area at the current time is finally screened out from each of the selected spatial areas. A feasible implementation method is: for a single selected spatial area, the third occurrence times of each different target area in each important lighting period are counted. The larger the third occurrence times, the more likely it is that the corresponding target area of ​​the selected spatial area will have a strong light intensity during the important lighting period. Then, in descending order of the third occurrence times, the second number of target areas are selected from each target area, and they are all determined as the easy-to-illuminate areas of the corresponding important lighting period, that is, the areas where high light intensity is likely to appear in the corresponding important lighting period.

[0048] Further, the third weight of each important illumination period and the fourth weight of each corresponding easy illumination area are calculated, the third weight is the ratio of the second occurrence number of a single important illumination period corresponding to the selected spatial area to the sum of the second occurrence number of all corresponding important illumination periods, and the fourth weight is the ratio of the third occurrence number of a single easy illumination area corresponding to the important illumination period to the sum of the third occurrence number of all corresponding easy illumination areas. Then, according to the third weight of each spatial area to be selected and the corresponding fourth weights, a suitable spatial area is determined from each spatial area to be selected. A feasible determination method is: for a single spatial area to be selected, an important illumination period in which there is at least one key area in each corresponding easy illumination area is determined as a key illumination period, wherein, in the embodiment of the present application, the key area is the area in the spatial area that has the greatest impact on the construction of the three-dimensional model. The key area is a key structural area in the spatial area. For example, the key area can be a location area that defines the basic shape and structure of the spatial area, such as a corner, door frame, window sill, etc. The three-dimensional spatial data collected in these areas will directly affect the stability and accuracy of the constructed three-dimensional model.

[0049] Calculate the second product of the third weight of each key lighting period and the fourth weight of each corresponding key area and sum them to obtain the sum of the first products of the corresponding key lighting period. The larger the sum of the first products, the greater the possibility that the light intensity in the key area is too high in the corresponding key lighting period. Further, calculate the ratio of the sum of the first products corresponding to each key lighting period in the same spatial area to be selected and the cumulative result of the sum of the first products of the remaining key lighting periods. The larger the ratio, the greater the relative possibility that the light intensity in the key area is too high in a single key lighting period in the same spatial area to be selected compared to the remaining key lighting periods. Then select the minimum ratio from the corresponding ratios of the single spatial area to be selected. The possibility that the light intensity in the key area is too high in the key lighting period corresponding to the minimum ratio is smaller than that in the remaining key lighting periods. Then determine it as the selection ratio of the corresponding spatial area to be selected.

[0050] Furthermore, when the current time is included in the key illumination period corresponding to the selection ratio of each candidate spatial area, the minimum selection ratio is selected from each selection ratio, and the candidate spatial area corresponding to the minimum selection ratio is determined as the suitable spatial area in the target real estate suitable for surveying and mapping at the current time. Thus, the three-dimensional model finally obtained by surveying and mapping this suitable spatial area is more accurate.

[0051] In other embodiments, after the suitable spatial area is determined, the third product of the third weight of the important lighting period of the current time in the suitable spatial area and the fourth weight of the corresponding easy-to-light areas is calculated, and the third products are summed to obtain the corresponding sum of the second products. The larger the sum of the second products, the greater the possibility that the suitable spatial area currently has an area with excessive light intensity. Similarly, the fourth product of the third weight of the important lighting period of the current time in the remaining spatial area of ​​the target real estate and the fourth weight of the corresponding easy-to-light areas is calculated and summed to obtain the corresponding sum of the third products. The larger the sum of the third products, the greater the possibility that the remaining spatial area currently has an area with excessive light intensity. Among them, the remaining spatial area is the spatial area other than the suitable spatial area in the target real estate. Finally, if the sum of the second products is less than the sum of the third products, it means that the possibility that there will be an area with excessive light intensity in the suitable spatial area after the current time is the smallest, and then the suitable spatial area is verified again to be correct.

[0052] S206: Collect target three-dimensional spatial data corresponding to the suitable spatial area through the AR device, and generate a target three-dimensional model corresponding to the suitable spatial area through a preset SLAM algorithm based on the target three-dimensional spatial data.

[0053] S207: The target three-dimensional model is superimposed and displayed with the real physical environment in the appropriate space area to obtain an augmented reality result.

[0054] S208: Based on the augmented reality results, the target three-dimensional model is calibrated and indoor objects are annotated, and the step of obtaining suitable spatial areas in the target real estate at the current time that are suitable for surveying and mapping is repeated until all spatial areas in the target real estate are surveyed and mapped.

[0055] For details, please refer to steps S102-S104, which will not be described in detail here.

[0056] The implementation principle of an AR-based intelligent real estate surveying and mapping method in an embodiment of the present application is: determine a suitable spatial area that has the least impact on the currently collected three-dimensional spatial data, so that the subsequent surveying and mapping of the suitable spatial area is more accurate. Furthermore, the target three-dimensional space of this suitable spatial area is collected through an AR device, and then a target three-dimensional model corresponding to the suitable spatial area is generated. Then, by utilizing the virtual-real fusion characteristics of the AR device, the target three-dimensional model is superimposed and displayed with the real physical environment in the suitable spatial area, so that the wearer of the AR device can more intuitively observe the fit between the target three-dimensional model and the real physical environment, so as to intuitively check and mark the target three-dimensional model, so that the target three-dimensional model obtained by the current surveying and mapping has a higher accuracy. Finally, repeat the steps of obtaining a suitable spatial area suitable for surveying and mapping in the target real estate at the current time, and determine the next spatial area that is more suitable for surveying and mapping, thereby improving the accuracy of real estate surveying and mapping.

[0057] The following is an embodiment of the device of the present application, which can be used to execute the embodiment of the method of the present application. For details not disclosed in the embodiment of the device of the present application, please refer to the embodiment of the method of the present application.

[0058] See also Figure 3 , which is a schematic diagram of the structure of an AR-based real estate intelligent surveying and mapping device provided in an embodiment of the present application. The AR-based real estate intelligent surveying and mapping device can be implemented as all or part of the device through software, hardware, or a combination of both. The device includes an information acquisition module 11, a model building module 12, an overlay display module 13, and a space surveying and mapping module 14.

[0059] The information acquisition module 11 is used to acquire a suitable spatial area suitable for surveying and mapping in the target real estate at the current time, wherein the target real estate contains at least one spatial area that needs to be surveyed and mapped, and the suitable area is a spatial area where the light has the least effect on the depth camera collecting three-dimensional spatial data; The model building module 12 is used to collect the target three-dimensional spatial data corresponding to the suitable spatial area through the AR device, and generate the target three-dimensional model corresponding to the suitable spatial area through a preset SLAM algorithm based on the target three-dimensional spatial data; The superimposed display module 13 is used to superimpose and display the target three-dimensional model with the real physical environment in the appropriate space area to obtain an augmented reality result; The spatial mapping module 14 is used to calibrate the target three-dimensional model and mark indoor objects based on the augmented reality results, and repeatedly execute the step of obtaining suitable spatial areas suitable for mapping in the target real estate at the current time until all spatial areas in the target real estate are mapped.

[0060] Optionally, the information acquisition module 11 is specifically used for: Counting the number of first occurrences of a target area in each spatial area of ​​the target real estate, the target area being a historical area where the light intensity exceeds a preset intensity threshold; Determine the historical period in which the target area appears in each spatial area, count the number of second appearances in each historical period, and select the first historical period from each historical period in ascending order of the number of second appearances as the important illumination period of the corresponding spatial area; Calculate the first weight of each spatial region and the second weight of each corresponding important illumination period, the first weight being the ratio of the first occurrence number of each spatial region to the sum of the first occurrence numbers of all spatial regions, and the second weight being the ratio of the second occurrence number of a single important illumination period corresponding to the spatial region to the sum of the second occurrence numbers of all corresponding important illumination periods; Determine, from each spatial region, a candidate spatial region for current surveying and mapping according to the first weight and the corresponding second weights; Determine from among the candidate spatial areas a suitable spatial area in the target real estate that is suitable for surveying and mapping at the current time.

[0061] Optionally, the information acquisition module 11 is specifically used for: Determine the spatial region including the current time in each corresponding important illumination period as the target spatial region, determine the important illumination period including the current time as the target illumination period, and calculate the first product of the first weight of each target spatial region and the second weight of the corresponding target illumination period; Determine a correction coefficient corresponding to each target spatial region, and multiply each first product by the corresponding correction coefficient to obtain a corresponding first product correction result; Comparing each first product correction result with a preset product threshold; If the first product correction result is less than the product threshold, the corresponding target space area is determined as the space area to be selected.

[0062] Optionally, the information acquisition module 11 is specifically used for: Calculate the weight product of the first weight of each target spatial area and the second weight of each corresponding important illumination period and sum them up to obtain the sum of the weight products; According to the sum of the weight products, the correction coefficient of the corresponding target space area is determined. The larger the sum of the weight products is, the larger the corresponding correction coefficient is, and the correction coefficient is greater than 1.

[0063] Optionally, the information acquisition module 11 is further configured to: Counting the third occurrence times of each target area in a single important illumination period of the selected spatial area, and selecting the second target area from each target area in descending order of the third occurrence times as the easy illumination area of ​​the corresponding important illumination period; Calculate the third weight of each important illumination period and the fourth weight of each corresponding easy illumination area, the third weight being the ratio of the second occurrence number of a single important illumination period corresponding to the selected spatial area to the sum of the second occurrence numbers of all corresponding important illumination periods, and the fourth weight being the ratio of the third occurrence number of a single easy illumination area corresponding to the important illumination period to the sum of the third occurrence numbers of all corresponding easy illumination areas; According to the third weight of each candidate spatial area and the corresponding fourth weights, a suitable spatial area suitable for surveying and mapping in the target real estate at the current time is determined from each candidate spatial area.

[0064] Optionally, the information acquisition module 11 is further configured to: For a single to-be-selected spatial area, an important lighting period in which at least one key area exists in each corresponding easy-to-light area is determined as a key lighting period, and the second product of the third weight of each key lighting period and the fourth weight of each corresponding key area is calculated and summed to obtain the sum of the first products of the corresponding key lighting period; Calculate the ratio of the sum of the first products of each key lighting period of the same space area to be selected to the cumulative sum of the first products of the remaining key lighting periods, and select the minimum ratio from each ratio to determine it as the selection ratio of the corresponding space area to be selected; When the current time is included in the key lighting time periods corresponding to all selection ratios, the minimum selection ratio is selected from each selection ratio, and the candidate spatial area corresponding to the minimum selection ratio is determined as the suitable spatial area in the target real estate suitable for surveying and mapping at the current time.

[0065] Optional, such as Figure 4 As shown, the device also includes a region verification module 15, which is specifically used for: Calculate the third product of the third weight of the important illumination period of the suitable spatial area including the current time and the fourth weight of each corresponding easy illumination area; Summing each third product to obtain the sum of the corresponding second products; Calculate the fourth product of the third weight of the important illumination period of the current time and the fourth weight of each corresponding easy illumination area in the remaining spatial area, the remaining spatial area being the spatial area other than the suitable spatial area; The fourth products are summed to obtain the corresponding sum of the third products. If the sum of the second products is less than the sum of the third products, it is determined that the appropriate spatial area verification is correct.

[0066] It should be noted that the AR-based real estate intelligent surveying and mapping device provided in the above embodiment only uses the division of the above functional modules as an example when executing the AR-based real estate intelligent surveying and mapping method. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the internal structure of the device is divided into different functional modules to complete all or part of the functions described above. In addition, the AR-based real estate intelligent surveying and mapping device and the AR-based real estate intelligent surveying and mapping method embodiment provided in the above embodiment belong to the same concept. The implementation process is detailed in the method embodiment and will not be repeated here.

[0067] An embodiment of the present application further discloses a computer-readable storage medium, and the computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, an AR-based real estate intelligent surveying and mapping method of the above embodiment is adopted.

[0068] Among them, the computer program can be stored in a computer-readable medium, the computer program includes computer program code, the computer program code can be in the form of source code, object code, executable file or certain middleware, etc. The computer-readable medium includes any entity or device that can carry the computer program code, recording medium, USB flash drive, mobile hard disk, magnetic disk, optical disk, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signal, telecommunication signal and software distribution medium, etc. It should be noted that the computer-readable medium includes but is not limited to the above-mentioned components.

[0069] Among them, through this computer-readable storage medium, an AR-based real estate intelligent surveying and mapping method of the above embodiment is stored in a computer-readable storage medium, and is loaded and executed on a processor to facilitate the storage and application of the above method.

[0070] An embodiment of the present application also discloses an electronic device, in which a computer program is stored in a computer-readable storage medium. When the computer program is loaded and executed by a processor, the above-mentioned AR-based intelligent real estate surveying and mapping method is adopted.

[0071] The electronic device may be a desktop computer, a laptop computer, a cloud server or other electronic device, and the electronic device includes but is not limited to a processor and a memory. For example, the electronic device may also include input and output devices, a network access device, and a bus.

[0072] Among them, the processor can adopt a central processing unit (CPU). Of course, according to actual usage, other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. can also be adopted. The general-purpose processor can adopt a microprocessor or any conventional processor, etc., and this application does not impose any restrictions on this.

[0073] Among them, the memory can be an internal storage unit of the electronic device, such as a hard disk or memory of the electronic device, or it can be an external storage device of the electronic device, such as a plug-in hard disk, a smart memory card (SMC), a secure digital card (SD) or a flash memory card (FC) equipped on the electronic device. Moreover, the memory can also be a combination of an internal storage unit and an external storage device of the electronic device. The memory is used to store computer programs and other programs and data required by the electronic device. The memory can also be used to temporarily store data that has been output or is to be output, and this application does not impose any restrictions on this.

[0074] Among them, through this electronic device, an AR-based real estate intelligent surveying and mapping method of the above embodiment is stored in the memory of the electronic device, and is loaded and executed on the processor of the electronic device for easy use.

[0075] The above is only an exemplary embodiment of the present disclosure and cannot be used to limit the scope of the present disclosure. That is, any equivalent changes and modifications made according to the teachings of the present disclosure are still within the scope of the present disclosure. This application is intended to cover any variation, use or adaptive change of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary technical means in the technical field that are not recorded in the present disclosure. The description and examples are only regarded as exemplary, and the scope and spirit of the present disclosure are defined by the claims.

Claims

1. A real estate intelligent surveying and mapping method based on AR, characterized in that: Applied to an AR device, the AR device is equipped with a depth camera, and the method includes: Acquire a suitable spatial area suitable for surveying and mapping in the target real estate at the current time, wherein the target real estate contains at least one spatial area that needs to be surveyed and mapped, and the suitable area is a spatial area where the light has the least effect on the depth camera collecting three-dimensional spatial data; The target three-dimensional spatial data corresponding to the suitable spatial area is collected by the AR device, and based on the target three-dimensional spatial data, a target three-dimensional model corresponding to the suitable spatial area is generated by a preset SLAM algorithm; The target three-dimensional model is superimposed and displayed with the real physical environment in the suitable space area to obtain an augmented reality result; Based on the augmented reality result, the target three-dimensional model is calibrated and indoor objects are annotated, and the step of obtaining suitable spatial areas in the target real estate at the current time that are suitable for surveying and mapping is repeated until the surveying and mapping of all spatial areas in the target real estate is completed.

2. The AR-based real estate intelligent surveying and mapping method according to claim 1, characterized in that: The obtaining of a suitable spatial area suitable for surveying and mapping in the target real estate at the current time specifically includes: Counting the number of first occurrences of a target area in each spatial area of ​​the target real estate, the target area being a historical area where the light intensity exceeds a preset intensity threshold; Determine the historical period in which the target area appears in each of the spatial regions, count the second occurrence times of each of the historical periods, and select the first historical period from each of the historical periods in ascending order of the second occurrence times as the important illumination period of the corresponding spatial region; Calculating a first weight of each of the spatial regions and a second weight of each of the corresponding important illumination periods, wherein the first weight is a ratio of a first occurrence number of each spatial region to a sum of first occurrence numbers of all spatial regions, and the second weight is a ratio of a second occurrence number of a single important illumination period corresponding to the spatial region to a sum of second occurrence numbers of all corresponding important illumination periods; Determining a selected spatial region currently being surveyed and mapped from each of the spatial regions according to the first weight and the corresponding second weights; A suitable spatial area suitable for surveying and mapping in the target real estate at the current time is determined from each of the candidate spatial areas.

3. The AR-based real estate intelligent surveying and mapping method according to claim 2, characterized in that: Determining the selected spatial region for current surveying and mapping from the spatial regions according to the first weight and the corresponding second weights specifically includes: Determine the spatial region including the current time in each corresponding important illumination period as the target spatial region, determine the important illumination period including the current time as the target illumination period, and calculate the first product of the first weight of each target spatial region and the second weight of the corresponding target illumination period; Determine a correction coefficient corresponding to each of the target spatial regions, and multiply each of the first products by the corresponding correction coefficient to obtain a corresponding first product correction result; Comparing each of the first product correction results with a preset product threshold; If the first product correction result is smaller than the product threshold, the corresponding target space area is determined as the space area to be selected.

4. The AR-based real estate intelligent surveying and mapping method according to claim 3 is characterized in that: The determining of the correction coefficient corresponding to each target space area includes: Calculating the weight product of the first weight of each target spatial area and the second weight of each corresponding important illumination period and summing them up to obtain the sum of the weight products; According to the sum of the weight products, a correction coefficient of the corresponding target space area is determined. The larger the sum of the weight products is, the larger the corresponding correction coefficient is, and the correction coefficient is greater than 1.

5. The AR-based real estate intelligent surveying and mapping method according to claim 2, characterized in that: The determining of a suitable spatial area suitable for surveying and mapping in the target real estate at the current time from each of the candidate spatial areas specifically includes: Counting the third occurrence times of each target area in a single important illumination period of the selected spatial area, and selecting the second number of target areas from each target area in descending order of the third occurrence times as the easy illumination area of ​​the corresponding important illumination period; Calculate the third weight of each of the important illumination periods and the fourth weight of each of the corresponding easy illumination areas, wherein the third weight is the ratio of the second occurrence number of a single important illumination period corresponding to the selected spatial area to the sum of the second occurrence numbers of all corresponding important illumination periods, and the fourth weight is the ratio of the third occurrence number of a single easy illumination area corresponding to the important illumination period to the sum of the third occurrence numbers of all corresponding easy illumination areas; According to the third weight of each of the candidate spatial regions and the corresponding fourth weights, a suitable spatial region suitable for surveying and mapping in the target real estate at the current time is determined from each of the candidate spatial regions.

6. The AR-based real estate intelligent surveying and mapping method according to claim 5, characterized in that: The determining, according to the third weights of the selected spatial regions and the corresponding fourth weights, of a suitable spatial region in the target real estate at the current time from the selected spatial regions specifically includes: For a single selected spatial area, an important lighting period in which at least one key area exists in each corresponding easy lighting area is determined as a key lighting period, and a second product of the third weight of each key lighting period and the fourth weight of each corresponding key area is calculated and summed to obtain the sum of the first products of the corresponding key lighting period; Calculate the ratio of the sum of the first products of each of the key lighting periods of the same to-be-selected spatial area to the cumulative sum of the first products of the remaining key lighting periods, and select the minimum ratio from the ratios to determine as the selection ratio of the corresponding to-be-selected spatial area; When the current time is included in the key lighting time periods corresponding to all the selection ratios, the minimum selection ratio is selected from the selection ratios, and the selected spatial area corresponding to the minimum selection ratio is determined as the suitable spatial area in the target real estate suitable for surveying and mapping at the current time.

7. The AR-based real estate intelligent surveying and mapping method according to claim 6, characterized in that: The method further comprises: Calculate the third product of the third weight of the important illumination period of the suitable spatial area including the current time and the fourth weight of each corresponding easy illumination area; summing the third products to obtain a sum of the corresponding second products; Calculate a fourth product of a third weight of the important illumination period of the current time in the remaining spatial area and a fourth weight of each corresponding easy illumination area, wherein the remaining spatial area is a spatial area other than the suitable spatial area; The fourth products are summed to obtain the corresponding sum of the third products. If the sum of the second products is less than the sum of the third products, it is determined that the appropriate spatial area has been verified correctly.

8. An AR-based real estate intelligent surveying and mapping device, characterized in that: include: An information acquisition module (11) is used to acquire a suitable spatial area suitable for surveying and mapping in a target real estate at a current time, wherein the target real estate contains at least one spatial area that needs to be surveyed and mapped, and the suitable area is a spatial area where the influence of light on the acquisition of three-dimensional spatial data by the depth camera is minimal; A model building module (12) is used to collect target three-dimensional spatial data corresponding to the suitable spatial area through the AR device, and generate a target three-dimensional model corresponding to the suitable spatial area based on the target three-dimensional spatial data through a preset SLAM algorithm; A superimposed display module (13) is used to superimpose and display the target three-dimensional model and the real physical environment in the suitable space area to obtain an augmented reality result; A spatial mapping module (14) is used to calibrate the target three-dimensional model and mark indoor objects based on the augmented reality result, and repeatedly execute the step of obtaining suitable spatial areas in the target real estate that are suitable for mapping at the current time until the mapping of all spatial areas in the target real estate is completed.

9. A computer-readable storage medium having a computer program stored therein, characterized in that: When the computer program is loaded and executed by a processor, the method according to any one of claims 1 to 7 is adopted.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and capable of running on the processor, characterized in that: When the processor loads and executes the computer program, the method according to any one of claims 1 to 7 is adopted.

Images