Wall material identification method and device based on laser point cloud scanning, and electronic equipment

Abstract

The embodiment of the invention discloses a wall material identification method and device based on laser point cloud scanning and electronic equipment. A specific embodiment of the method comprises the following steps: controlling a shooting device to carry out image shooting on an original house wall so as to generate an original house wall image set; performing wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set; controlling a three-dimensional laser scanning device to perform laser scanning on the house wall body after the decoration layer is removed so as to generate house wall body point cloud data; performing point cloud file analysis on the house wall point cloud data to generate a house wall point cloud data set; and according to the wall damage image feature set and the house wall point cloud data set, wall material identification is performed on each wall after the decoration layer of the original house is removed so as to generate a wall material identification information set, and the wall material identification information set is stored. According to the embodiment, occupation of computing resources can be reduced.

Description

Technical Field

[0001] The embodiments of the present disclosure relate to the technical field of wall material recognition, and specifically to a wall material recognition method, device, and electronic equipment based on laser point cloud scanning. Background Art

[0002] House wall material identification is a technology used to identify wall material and other information. The traditional workflow of interior decoration design is: the designer goes to the project to be signed to measure the house on site, briefly records the position relationship and space size information of each room, that is, the preliminary measurement. After the original decoration is dismantled to the rough state, the construction personnel will go to the site to conduct precise measurements, evaluate the current wall material and other information, and then propose the corresponding wall construction method plan.

[0003] However, in practice, it is found that when the above method is used to identify the material of a house wall, the following technical problems often occur:

[0004] Manual detection methods are greatly affected by human factors, with low accuracy and low efficiency. Although the accuracy of traditional detection instruments has been improved, they still require manual operation and it is difficult to fully and deeply obtain the three-dimensional information and internal conditions of the wall. Therefore, multiple measurements are required to ensure accuracy. As a result, more storage resources are occupied.

[0005] The above information disclosed in this Background section is only for enhancement of understanding of the background of the inventive concept and therefore it may contain information that does not form the prior art that is already known in this country to a person of ordinary skill in the art. Summary of the invention

[0006] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0007] Some embodiments of the present disclosure propose wall material recognition methods, devices, and electronic devices based on laser point cloud scanning to solve one or more of the technical problems mentioned in the above background technology section.

[0008] In a first aspect, some embodiments of the present disclosure provide a wall material recognition method based on laser point cloud scanning, the method comprising: controlling a shooting device to capture images of original house walls to generate an original house wall image set; performing wall damage recognition on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; controlling a three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate house wall point cloud data; performing wall damage recognition on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; controlling a three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate house wall point cloud data; The point cloud data is parsed into a point cloud file to generate a house wall point cloud data set, wherein each house wall point cloud data corresponds to a wall of the original house after the decoration layer is removed; according to the wall damaged image feature set and the house wall point cloud data set, wall material identification is performed on each wall of the original house after the decoration layer is removed to generate a wall material identification information set, and the wall material identification information set is stored, wherein each wall material information in the wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

[0009] In a second aspect, some embodiments of the present disclosure provide a wall material recognition device based on laser point cloud scanning, the device comprising: an image capturing unit, configured to control a capturing device to capture images of original house walls to generate an original house wall image set; a wall damage recognition unit, configured to perform wall damage recognition on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; a laser scanning unit, configured to control a three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate house wall point cloud data; a file The parsing unit is configured to perform point cloud file parsing on the above-mentioned house wall point cloud data to generate a house wall point cloud data set, wherein each house wall point cloud feature corresponds to a wall of the above-mentioned original house after the decoration layer is removed; the wall material identification unit is configured to perform wall material identification on each wall of the above-mentioned original house after the decoration layer is removed according to the above-mentioned wall damaged image feature set and the above-mentioned house wall point cloud data set to generate a wall material identification information set, and store the above-mentioned wall material identification information set, wherein each wall material information in the above-mentioned wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

[0010] In a third aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0011] In a fourth aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0012] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the wall material recognition method based on laser point cloud scanning of some embodiments of the present disclosure, the occupation of storage resources by the generated wall material recognition information can be reduced. Specifically, the reason for occupying more storage resources is that the manual detection method is greatly affected by human factors, has low accuracy and low efficiency. And although the accuracy of the traditional detection instrument method has been improved, it still requires manual operation, and it is difficult to fully and deeply obtain the three-dimensional information and internal conditions of the wall. Therefore, multiple measurements are required to ensure accuracy. Based on this, the wall material recognition method based on laser point cloud scanning of some embodiments of the present disclosure, first of all, takes into account the situation that the house wall is prone to abnormalities. Therefore, by controlling the shooting device, the original house wall is imaged to generate an original house wall image set. Thus, the wall damage can be identified from the wall appearance of the original wall without removing the decoration layer. Thus, a wall damage image feature set can be generated. Then, considering the house wall after the decoration layer is removed, even if the surface image can be captured, the internal state is difficult to detect. Therefore, by controlling the three-dimensional laser scanning equipment, the house wall after the decoration layer is removed is laser scanned to generate the house wall point cloud data. Then, the point cloud file of the house wall point cloud data can be parsed to generate the house wall point cloud data set. Thus, the damaged wall image features and the house wall point cloud data corresponding to each wall can be determined. Thus, the material information of the wall can be comprehensively judged by wall material identification to improve the accuracy of the generated material information. Finally, the above-mentioned wall material identification information set can be stored. Here, each wall material information can include wall material data, wall age data, and wall integrity data. Furthermore, not only the accuracy of the identification can be determined, but also the diversity of the wall material identification information can be improved. Thus, there is no need to measure and store the results of multiple measurements multiple times. Furthermore, the occupation of computing resources can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0014] Figure 1 is a flow chart of some embodiments of the wall material recognition method based on laser point cloud scanning according to the present disclosure;

[0015] Figure 2 is a schematic diagram of a three-dimensional structure of a house after rendering according to some embodiments of the wall material recognition method based on laser point cloud scanning disclosed in the present invention;

[0016] Figure 3 It is a schematic diagram of the structure of some embodiments of the wall material recognition device based on laser point cloud scanning according to the present disclosure;

[0017] Figure 4 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0018] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0019] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0020] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0021] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0022] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0023] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0024] Figure 1 The process 100 of some embodiments of the wall material recognition method based on laser point cloud scanning according to the present disclosure is shown. The wall material recognition method based on laser point cloud scanning includes the following steps:

[0025] Step 101: Control a shooting device to shoot images of original house walls to generate an original house wall image set.

[0026] In some embodiments, the execution subject of the wall material recognition method based on laser point cloud scanning can control the shooting device in a wired or wireless manner to capture images of the original house wall to generate a set of original house wall images. The shooting device can be manually held or fixed by a pre-laid shooting track. Secondly, after determining that the shooting device is fixed, the shooting device can be controlled to capture images. Here, all walls of each room in the original house can be photographed. For example, the floor, ceiling and walls of the house can be photographed.

[0027] It should be noted that the above-mentioned wireless connection methods may include but are not limited to 3G / 4G connection, WiFi connection, Bluetooth connection, WiMAX connection, Zigbee connection, UWB (ultra wideband) connection, and other wireless connection methods currently known or to be developed in the future.

[0028] In some optional implementations of some embodiments, the execution subject controls the shooting device to shoot images of the original house wall to generate the original house wall image set, which may include the following steps:

[0029] The first step is to control the shooting device to shoot images of the original house wall to obtain a sequence of house interior images, wherein for each wall, each of the house interior images shot corresponds to at least one front-view image.

[0030] In the second step, the wall structure feature points of each house interior image in the above house interior image sequence are extracted to generate a wall structure feature point sequence set. The wall structure feature point extraction can be performed by a preset wall structure feature point extraction algorithm to generate a wall structure feature point sequence set. Here, feature points on the wall corner line in each house interior image can be extracted. Each wall structure feature point sequence can represent a wall corner line.

[0031] As an example, the wall structure feature point extraction algorithm may include but is not limited to at least one of the following: Surf (Speeded Up Robust Features, accelerated robust features) algorithm, Harris corner point detection, FAST corner point detection, BRIEF (Binary Robust Independent Elementary Features, feature descriptor algorithm), etc.

[0032] The third step is to crop the wall image corresponding to each wall from each house interior image in the house interior image sequence as the original house wall image according to the above-mentioned wall structure feature point sequence set, and obtain the original house wall image set. Among them, first, each wall structure feature point in each wall structure feature point sequence can be fitted to obtain the wall corner line. Then, the inner angle between each wall corner line in each house interior image can be determined. If the inner angle is less than the preset angle threshold or greater than the preset angle threshold, it is characterized that the house interior image is a wall image taken from the front. Finally, the wall image taken from the front corresponding to each wall can be determined as the original house wall image, and the original house wall image set is obtained.

[0033] Step 102 , performing wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set.

[0034] In some embodiments, the execution subject may perform wall damage recognition on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature may correspond to a wall of the original house.

[0035] In some optional implementations of some embodiments, the execution subject performs wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set, which may include the following steps:

[0036] For each original house wall image in the original house wall image set, the wall damage image features are generated by the following steps:

[0037] In practice, considering that the original house wall has abnormal problems, such as waterlogging, cracking, etc. Therefore, firstly, the color feature detection of the wall can be performed to determine whether there is an abnormality. Then, the texture feature extraction is performed to determine whether there is cracking.

[0038] In the first step, color feature detection is performed on the original house wall image to generate a wall color feature map. The color feature detection can be performed on the original house wall image to generate a wall color feature map by using a color histogram algorithm.

[0039] The second step is to extract the wall texture features of the original house wall image to generate a wall texture feature map. Among them, the wall texture features can be extracted by a preset wall texture feature extraction algorithm to generate an initial wall texture feature map. Secondly, the texture features in the initial wall texture feature map can be filtered by the grayscale co-occurrence matrix method to extract the texture features corresponding to the wall cracking state and obtain the wall texture feature map. Here, the wall feature texture is extracted to extract the characteristics of whether the wall is cracked.

[0040] As an example, the above-mentioned wall texture feature algorithm may include but is not limited to at least one of the following: GLCM (Gray Level Co-occurrence Matrix) detection algorithm, Tamura texture analysis method, GWBTF (Gabor wavelet-based texture feature extraction) algorithm, etc.

[0041] The third step is to perform wall roughness anomaly detection on the original house wall image to generate a wall roughness anomaly feature map. The roughness feature can be extracted from the initial wall texture feature map of the original house wall image by Linear Discriminant Analysis (LDA) to generate a wall roughness anomaly feature map.

[0042] In the fourth step, based on the wall color feature map, the wall texture feature map and the wall roughness abnormality feature map, a wall damage image feature is generated. The wall color feature map, the wall texture feature map and the wall roughness abnormality feature map may be superimposed to obtain the wall damage image feature.

[0043] In practice, image recognition technology based on deep learning, infrared thermal imaging technology, ultrasonic detection technology, and radar scanning technology, etc., can each, to a certain extent, realize the evaluation of wall material and health. However, these alternatives have their own limitations in practical applications. For example, image recognition technology based on deep learning is greatly affected by environmental factors such as lighting and angle; infrared thermal imaging technology mainly relies on the temperature distribution on the surface of the wall, and has limitations in judging the internal structure of the wall; although ultrasonic detection technology and radar scanning technology can conduct in-depth detection inside the wall, the operation is relatively complicated and the equipment cost is high. Therefore, if the above scheme is adopted to solve the problems mentioned in the background technology, it is often accompanied by the following technical problem 2: the judgment of the wall material and the degree of damage is not accurate enough, and there is a large difference between the rendered image and the actual scene. Thus, rendering resources are wasted. In response to the above technical problem 2, the inventor decided to adopt the following solution.

[0044] Step 103, controlling the three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate point cloud data of the house wall.

[0045] In some embodiments, the execution subject may control a 3D laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate house wall point cloud data. The 3D laser scanning device may be manually operated at a fixed point (for example, scanning each wall in the middle of the house), or may be deployed on the preset shooting track to perform laser scanning in the house to obtain house wall point cloud data.

[0046] Step 104 , performing point cloud file parsing on the house wall point cloud data to generate a house wall point cloud data set.

[0047] In some embodiments, the execution subject may perform point cloud file analysis on the house wall point cloud data to generate a house wall point cloud data set. Each house wall point cloud data may correspond to a wall of the original house after the decoration layer is removed.

[0048] In some optional implementations of some embodiments, the execution subject performs point cloud file parsing on the house wall point cloud data to generate a house wall point cloud data set, which may include the following steps:

[0049] The first step is to preprocess the house wall point cloud data to generate processed wall point cloud data. The data preprocessing may be: removing the point cloud data outside the house wall to obtain processed wall point cloud data.

[0050] The second step is to filter the processed wall point cloud data to generate filtered wall point cloud data. The filtering process can be performed by non-local mean filtering to remove point cloud data that is redundant inside the house and is affected by light, movement of people, dust and other irrelevant items, and obtain filtered wall point cloud data.

[0051] The third step is to extract structured features from the filtered wall point cloud data to generate a structured point cloud coordinate sequence set. The structured feature extraction can be performed on the filtered wall point cloud data using a point cloud structuring algorithm to generate a structured point cloud coordinate sequence. Here, the structured feature extraction can be to extract the point cloud coordinates representing the edge of the wall as a structured point cloud coordinate sequence. Each wall edge can correspond to a structured point cloud coordinate sequence.

[0052] As an example, the above-mentioned point cloud structuring algorithm may include but is not limited to at least one of the following: NARF (normal aligned radial feature) algorithm, ISS (Intrinsic Shape Signatures), a key point extraction method based on the intrinsic shape features of the point cloud, SUSAN (Smallest Univalue Segment Assimilating Nucleus), a corner detection algorithm based on image processing, etc.

[0053] In the fourth step, the house wall point cloud data is grouped according to the connection lines of each structured point cloud coordinate in the structured point cloud coordinate sequence set to generate a house wall point cloud data set. The point cloud data can be divided according to the connection lines of each structured point cloud coordinate to obtain the house wall point cloud data.

[0054] Step 105, based on the wall damaged image feature set and the house wall point cloud data set, the wall material of each wall after the decoration layer of the original house is removed is identified to generate a wall material identification information set, and the wall material identification information set is stored.

[0055] In some embodiments, the execution subject may perform wall material identification on each wall after the decoration layer of the original house is removed based on the wall damage image feature set and the house wall point cloud data set to generate a wall material identification information set, and store the wall material identification information set. Each wall material information in the wall material information set may include but is not limited to at least one of the following: wall material data, wall age data, and wall integrity data. The wall integrity data may be used to characterize the integrity of each coordinate point on the wall.

[0056] In some optional implementations of some embodiments, the execution subject performs wall material recognition on each wall of the original house after the decoration layer is removed according to the wall damage image feature set and the house wall point cloud data set to generate a wall material recognition information set, which may include the following steps:

[0057] In the first step, the above-mentioned wall damage image feature set and the above-mentioned house wall point cloud feature set are matched to determine the wall damage image features and house wall point cloud data corresponding to the same wall. Among them, each wall damage image feature and each house wall point cloud feature can be set with a corresponding wall number. Therefore, matching can be performed by number to determine the wall damage image features and house wall point cloud data corresponding to the same wall.

[0058] In the second step, for the wall damage image features and house wall point cloud data corresponding to the same wall, the wall material identification information of each wall after the decoration layer of the original house is removed is generated through the following steps:

[0059] Step 1: Use the above-mentioned house wall point cloud data to perform roughness classification on the above-mentioned wall to generate a wall roughness point cloud feature map. Among them, the roughness of the wall can be distinguished by the change in the error of the house wall point cloud data, and represented by different colors. Specifically, the standard deviation of the point cloud coordinates of the house wall point cloud data corresponding to one wall can be determined as the error of the house wall point cloud data to characterize the roughness of the wall. Here, only the standard deviation of the coordinates in the normal direction (for example, the horizontal axis direction) of the wall surface can be determined.

[0060] Step 2: extract the wall color features of the house wall point cloud data to generate a wall color point cloud feature map. First, the house wall point cloud data can be read through a preset software library. Then, the color features are extracted through the color feature extraction function in the software library. Finally, the color value corresponding to each coordinate is determined to obtain a wall color point cloud feature map.

[0061] As an example, the software library may include but is not limited to at least one of the following: liblas: an open source library for processing laser radar data. The color feature extraction function in the software library may be: GetColor() function.

[0062] Step three, determine the material feature information of the wall according to the wall roughness point cloud feature map and the wall color point cloud feature map. In practice, different colors are used to distinguish different wall materials and different flatness. For example, dark red is for areas with missing corners or large depressions in ceramsite or block walls, and light gray is for flat concrete walls. Therefore, the wall roughness point cloud feature map and the wall color point cloud feature map can be input into a pre-trained decision tree model to generate wall material feature information.

[0063] Step 4: feature fusion is performed on the material feature information and the wall damage image feature to generate a wall damage feature map, and the wall damage feature map is determined as the wall material identification information. Feature fusion is feature superposition. For example, the material feature and the wall damage feature corresponding to the same coordinate are bound to the coordinate to obtain the wall damage feature map.

[0064] Optionally, the above execution entity may further perform the following steps:

[0065] The first step is to construct a three-dimensional structure diagram of the house based on the above-mentioned house wall point cloud feature set. The above-mentioned house wall point cloud feature set can be input into a preset point cloud processing software to construct a three-dimensional structure diagram of the house in a point cloud coordinate system.

[0066] As an example, the above-mentioned point cloud processing software may include but is not limited to at least one of the following: CloudCompare, MeshLab, etc.

[0067] In the second step, the material data, wall age data, and wall integrity data included in the wall material identification information set that do not meet the preset conditions are rendered into the three-dimensional structure diagram of the house to generate a rendered three-dimensional structure diagram of the house. The actual features of the wall can be rendered in the three-dimensional structure diagram of the house according to the wall damage feature diagram in the wall material identification information to obtain the rendered three-dimensional structure diagram of the house.

[0068] As an example, the rendered 3D structure of the house can be Figure 2 As shown. Here, for areas with normal wall structures, no rendering is required. For areas with slightly abnormal wall health (marked as good), such as slight wear and aging of the wall. It is marked in blue, and the feature prompt "The wall health is good and only requires local repair" is shown. Finally, for cases with abnormal wall weight (marked as poor). For example, the wall is cracked, the wall is incomplete, etc. The image area is rendered in red, and the darker the color, the greater the abnormality. It is shown that "the dark red wall is in poor health and severely damaged". At the same time, the dimensions of different parts of the wall are also marked to represent the degree of damage at different parts of the wall.

[0069] The third step is to store the rendered three-dimensional structure diagram of the house and send it to a target display terminal for display.

[0070] Optionally, the execution subject constructs a three-dimensional structure diagram of the house according to the house wall point cloud feature set, which may include the following steps:

[0071] The first step is to connect the structured point cloud coordinates in the structured point cloud coordinate sequence in the coordinate system of the three-dimensional laser scanning device to serve as a wall dividing line.

[0072] In the second step, in the above coordinate system, each wall dividing line is used as the wall structure, and the point cloud coordinates of the house wall point cloud data corresponding to each wall are filled in as the wall surface features to obtain the three-dimensional structure diagram of the house.

[0073] The above steps 103 to 105 and their related contents serve as an inventive point of an embodiment of the present disclosure, which solves the second technical problem mentioned in the background technology, "the judgment of the wall material and the degree of damage is not accurate enough, and there is a large difference between the rendered image and the actual scene. Thus, rendering resources are wasted". The factors that lead to "waste of rendering resources" are often as follows: the judgment of the wall material and the degree of damage is not accurate enough, and there is a large difference between the rendered image and the actual scene. In order to achieve this effect, first of all, the detection accuracy and efficiency are improved: compared with the traditional manual detection method and the traditional detection instrument method, the laser point cloud scanning technology can be used to quickly and accurately obtain the three-dimensional information of the wall, so as to realize the accurate judgment of the wall material and health level. This not only greatly improves the accuracy of the detection, but also significantly shortens the detection time and improves work efficiency. Secondly, comprehensiveness and depth: the technical solution can fully cover every corner of the wall and provide detailed three-dimensional point cloud data, so that the wall material judgment and health level assessment are more comprehensive and in-depth. In contrast, this solution can reveal more detailed information about the wall. Information, providing stronger support for subsequent decoration and maintenance. Then, intelligence and automation: Through the above-mentioned related processing technologies on point cloud features and image features, the intelligent and automated recognition of wall materials can be realized. This not only reduces the impact of human factors on the detection results, but also makes the detection process more convenient and efficient. It has obvious advantages in the level of intelligence. Finally, reduce costs and improve competitiveness: Since the technical solution can significantly improve the detection efficiency and accuracy, it can reduce the detection cost and enhance the competitiveness of the product. At the same time, by providing comprehensive and accurate wall information. Furthermore, it can also help customers better plan decoration and maintenance plans, and further improve customer satisfaction and loyalty. Finally, the accuracy of house material information recognition is improved. In turn, the consumption of rendering resources is reduced.

[0074] In addition, in comparison, the technical solution of "wall material information identification based on laser point cloud scanning" has become a better choice for achieving the purpose of the invention due to its advantages of high precision, high efficiency, comprehensiveness and intelligence. This technical solution can fully cover every corner of the wall and provide detailed three-dimensional point cloud data, making the wall material determination and health assessment more accurate and efficient. Therefore, our technical solution can provide users with a better and more efficient service experience and meet the diverse needs of customers.

[0075] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the wall material recognition method based on laser point cloud scanning of some embodiments of the present disclosure, the occupation of storage resources by the generated wall material recognition information can be reduced. Specifically, the reason for occupying more storage resources is that the manual detection method is greatly affected by human factors, has low accuracy and low efficiency. And although the accuracy of the traditional detection instrument method has been improved, it still requires manual operation, and it is difficult to fully and deeply obtain the three-dimensional information and internal conditions of the wall. Therefore, multiple measurements are required to ensure accuracy. Based on this, the wall material recognition method based on laser point cloud scanning of some embodiments of the present disclosure, first of all, takes into account the situation that the house wall is prone to abnormalities. Therefore, by controlling the shooting device, the original house wall is imaged to generate an original house wall image set. Thus, the wall damage can be identified from the wall appearance of the original wall without removing the decoration layer. Thus, a wall damage image feature set can be generated. Then, considering the house wall after the decoration layer is removed, even if the surface image can be captured, the internal state is difficult to detect. Therefore, by controlling the three-dimensional laser scanning equipment, the house wall after the decoration layer is removed is laser scanned to generate the house wall point cloud data. Then, the point cloud file of the house wall point cloud data can be parsed to generate the house wall point cloud data set. Thus, the damaged wall image features and the house wall point cloud data corresponding to each wall can be determined. Thus, the material information of the wall can be comprehensively judged by wall material identification to improve the accuracy of the generated material information. Finally, the above-mentioned wall material identification information set can be stored. Here, each wall material information can include wall material data, wall age data, and wall integrity data. Furthermore, not only the accuracy of the identification can be determined, but also the diversity of the wall material identification information can be improved. Thus, there is no need to measure and store the results of multiple measurements multiple times. Furthermore, the occupation of computing resources can be reduced.

[0076] Further references Figure 3 As an implementation of the methods shown in the above figures, the present disclosure provides some embodiments of a wall material recognition device based on laser point cloud scanning. These device embodiments are similar to Figure 1 Corresponding to the method embodiments shown, the device can be specifically applied to various electronic devices.

[0077] like Figure 3 As shown, in some embodiments, the wall material recognition device 300 based on laser point cloud scanning includes: an image capturing unit 301, a wall damage recognition unit 302, a laser scanning unit 303, a file parsing unit 304 and a wall material recognition unit 305. The image capturing unit 301 is configured to control the capturing device to capture images of the original house wall to generate an original house wall image set; the wall damage recognition unit 302 is configured to perform wall damage recognition on each original house wall image in the above-mentioned original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; the laser scanning unit 303 is configured to control the three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate house wall point cloud data; the file parsing unit 304 is configured to perform wall damage recognition on the above-mentioned house wall The point cloud data is parsed into a point cloud file to generate a house wall point cloud data set, wherein each house wall point cloud feature corresponds to a wall of the original house after the decoration layer is removed; the wall material identification unit 305 is configured to perform wall material identification on each wall of the original house after the decoration layer is removed according to the wall damaged image feature set and the house wall point cloud data set to generate a wall material identification information set, and store the wall material identification information set, wherein each wall material information in the wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

[0078] It is understood that the units described in the device 300 are similar to those described in the reference Figure 1 Therefore, the operations, features and beneficial effects described above for the method are also applicable to the device 300 and the units contained therein, and will not be described in detail here.

[0079] Reference below Figure 4 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 400 suitable for implementing some embodiments of the present disclosure. Figure 4 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0080] like Figure 4As shown, the electronic device 400 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 401, which can perform various appropriate actions and processes according to a program stored in a read-only memory 402 or a program loaded from a storage device 408 into a random access memory 403. Various programs and data required for the operation of the electronic device 400 are also stored in the random access memory 403. The processing device 401, the read-only memory 402, and the random access memory 403 are connected to each other via a bus 404. An input / output interface 405 is also connected to the bus 404.

[0081] Typically, the following devices may be connected to the I / O interface 405: an input device 406 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 407 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 408 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 409. The communication device 409 may allow the electronic device 400 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 4 The electronic device 400 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 4 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0082] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 409, or installed from the storage device 408, or installed from the read-only memory 402. When the computer program is executed by the processing device 401, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0083] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, in which a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0084] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0085] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist independently without being assembled into the electronic device. The above-mentioned computer-readable medium carries one or more programs. When the above-mentioned one or more programs are executed by the electronic device, the electronic device: controls the shooting device to shoot images of the original house walls to generate an original house wall image set; performs wall damage identification on each original house wall image in the above-mentioned original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; controls the three-dimensional laser scanning device to perform laser scanning on the house wall after the removal of the decoration layer to generate house wall point cloud data; and performs wall damage identification on each original house wall image in the above-mentioned original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; controls the three-dimensional laser scanning device to perform laser scanning on the house wall after the removal of the decoration layer to generate house wall point cloud data; The point cloud file of the house wall point cloud data is parsed to generate a house wall point cloud data set, wherein each house wall point cloud data corresponds to a wall of the original house after the decoration layer is removed; according to the above-mentioned wall damaged image feature set and the above-mentioned house wall point cloud data set, the wall material of each wall of the original house after the decoration layer is removed is identified to generate a wall material identification information set, and the above-mentioned wall material identification information set is stored, wherein each wall material information in the above-mentioned wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

[0086] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0087] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0088] The units described in some embodiments of the present disclosure may be implemented by software or by hardware. The described units may also be provided in a processor, for example, may be described as: a processor including an image capturing unit, a wall damage identification unit, a laser scanning unit, a file parsing unit, and a wall material identification unit. The names of these units do not, in some cases, constitute limitations on the units themselves, for example, the laser scanning unit may also be described as "a unit for controlling a three-dimensional laser scanning device to perform laser scanning on the walls of a house after the decoration layer is removed."

[0089] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), and the like.

[0090] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A wall material recognition method based on laser point cloud scanning, comprising: Controlling the shooting device to shoot images of the original house wall to generate an original house wall image set; Performing wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; Control the 3D laser scanning equipment to perform laser scanning on the house wall after the decoration layer is removed to generate point cloud data of the house wall; Performing point cloud file parsing on the house wall point cloud data to generate a house wall point cloud data set, wherein each house wall point cloud data corresponds to a wall of the original house after the decoration layer is removed; According to the wall damaged image feature set and the house wall point cloud data set, wall material identification is performed on each wall of the original house after the decoration layer is removed to generate a wall material identification information set, and the wall material identification information set is stored, wherein each wall material information in the wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

2. The method according to claim 1, wherein: The method further comprises: Constructing a three-dimensional structure diagram of the house according to the house wall point cloud feature set; Rendering the body material data, wall age data, and wall integrity data included in the wall material identification information set that does not meet the preset conditions into the three-dimensional structure diagram of the house to generate a rendered three-dimensional structure diagram of the house; The rendered three-dimensional structure diagram of the house is stored and sent to a target display terminal for display.

3. The method according to claim 1, wherein: The controlling the shooting device to shoot images of the original house wall to generate an original house wall image set includes: Controlling the shooting device to shoot images of the original house wall to obtain a sequence of images of the interior of the house; Extracting wall structure feature points from each house interior image in the house interior image sequence to generate a wall structure feature point sequence set; According to the wall structure feature point sequence set, a wall image corresponding to each wall is cropped from each house interior image in the house interior image sequence as an original house wall image, thereby obtaining an original house wall image set.

4. The method according to claim 1, wherein: The performing wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set includes: For each original house wall image in the original house wall image set, a wall damage image feature is generated by the following steps: Performing color feature detection on the original house wall image to generate a wall color feature map; Extracting wall texture features from the original house wall image to generate a wall texture feature map; Performing wall roughness anomaly detection on the original house wall image to generate a wall roughness anomaly feature map; A wall damage image feature is generated according to the wall color feature map, the wall texture feature map and the wall roughness abnormality feature map.

5. The method according to claim 2, wherein: The performing point cloud file parsing on the house wall point cloud data to generate a house wall point cloud data set includes: Performing data preprocessing on the house wall point cloud data to generate processed wall point cloud data; Performing filtering on the processed wall point cloud data to generate filtered wall point cloud data; Performing structured feature extraction on the filtered wall point cloud data to generate a structured point cloud coordinate sequence set; According to the connection lines of each structured point cloud coordinate in the structured point cloud coordinate sequence set, the house wall point cloud data is grouped and processed to generate a house wall point cloud data set.

6. The method according to claim 5, wherein: The step of constructing a three-dimensional structure diagram of a house according to the house wall point cloud feature set comprises: In the coordinate system of the three-dimensional laser scanning device, each structured point cloud coordinate in the structured point cloud coordinate sequence is connected to serve as a wall segmentation line; In the coordinate system, each wall dividing line is taken as the wall structure, and the point cloud coordinates of the house wall point cloud data corresponding to each wall are filled in as the wall surface features to obtain a three-dimensional structure diagram of the house.

7. A wall material recognition device based on laser point cloud scanning, comprising: An image capturing unit is configured to control a capturing device to capture images of the original house wall to generate an original house wall image set; a wall damage identification unit configured to perform wall damage identification on each original house wall image in the original house wall image set to generate a wall damage image feature set, wherein each wall damage image feature corresponds to a wall of the original house; The laser scanning unit is configured to control the three-dimensional laser scanning device to perform laser scanning on the house wall after the decoration layer is removed to generate point cloud data of the house wall; A file parsing unit is configured to perform point cloud file parsing on the house wall point cloud data to generate a house wall point cloud data set, wherein each house wall point cloud feature corresponds to a wall of the original house after the decoration layer is removed; The wall material identification unit is configured to perform wall material identification on each wall of the original house after the decoration layer is removed according to the wall damaged image feature set and the house wall point cloud data set to generate a wall material identification information set, and store the wall material identification information set, wherein each wall material information in the wall material information set includes at least one of the following: wall material data, wall age data, and wall integrity data.

8. An electronic device, comprising: one or more processors; a storage device having one or more programs stored thereon, When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 6.

9. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images