Screw hole detection method, device, equipment and storage medium

Abstract

The present application relates to the field of hole detection, and discloses a screw hole detection method, device, equipment and storage medium.The method specifically comprises: acquiring a wall image to be detected and a template wall image, calculating first position information of each hole on the wall image to be detected and confidence of each hole, and generating a first detection result, identifying screw holes in the template wall image, determining detection probability of each screw hole based on second position information of each screw hole on the template wall, and generating a second detection result, fusing the first detection result and the second detection result, and obtaining a final detection result of the screw hole based on the fusion result.The present application fuses the position detection processor and the screw hole template reuse detection result to obtain the final position of the screw hole, thereby improving the screw hole detection accuracy.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of hole detection, in particular to a screw hole detection method, device, equipment and storage medium. BACKGROUND

[0002] The screw holes existing in the wall surface during the construction process need to be plugged, especially when the plugging operation is completed by using automatic equipment, the screw holes need to be detected and positioned first, but the existing technology is usually a single application of a position detection module or a template reuse, which may be disturbed by the environment or imaging quality and more false positives and false negatives may occur, resulting in low screw hole recognition accuracy. SUMMARY

[0003] The main purpose of the present application is to solve the problem of low screw hole recognition accuracy of the existing screw hole detection scheme for the construction wall surface.

[0004] The first aspect of the present application provides a screw hole detection method, comprising: acquiring a to-be-detected wall surface image and a template wall surface image, wherein the template wall surface is an actual wall surface determined based on similarity with the to-be-detected wall surface in the same building; identifying holes in the to-be-detected wall surface image and first position information of each hole located in the to-be-detected wall surface, and determining the confidence of each hole according to the first position information and the corresponding hole feature, generating a first detection result based on the first position information and the confidence, wherein the confidence is the probability of determining that the hole belongs to a screw hole; identifying screw holes in the template wall surface image, and determining the detection probability of each screw hole based on the second position information of each screw hole located in the template wall surface, and generating a second detection result based on the second position information and the detection probability, wherein the detection probability is the probability for indicating that the second position information is missed, false detected or exists a screw hole; fusing the first detection result and the second detection result, and obtaining the final detection result of the screw hole based on the fusion result.

[0005] Optionally, in the first implementation manner of the first aspect of the present application, the method further comprises: identifying the holes in the wall surface image to be detected and the first position information of each hole on the wall surface to be detected, and determining the confidence of each hole according to the first position information and the corresponding hole feature; and generating the first detection result based on the first position information and the confidence.

[0006] Optionally, in the second implementation manner of the first aspect of the present application, the method further comprises: identifying the template wall surface image by the position detection processor to obtain the second position information of each screw hole; extracting the environmental information on the second position information, and calculating the detection probability of the screw hole on the second position information based on the environmental information; and generating the second detection result based on the second position information and the detection probability.

[0007] Optionally, in the third implementation manner of the first aspect of the present application, the method further comprises: extracting the screw hole representation information and the corresponding construction information in the environmental information; calculating the probability of the existence of the screw hole on the second position and the probabilities of false detection and missed detection according to the screw hole representation information and the construction information; and determining the detection probability based on the probability of the existence of the screw hole and the probabilities of false detection and missed detection.

[0008] Optionally, in a fourth implementation form of the first aspect of the present application, the step of fusing the first detection result and the second detection result, and obtaining a final detection result of screw holes based on the fused result, comprises: comparing the position of the hole in the first detection result with the position of the screw hole in the second detection result to obtain a comparison result; if the comparison result is that the position of the hole is consistent with the position of the screw hole, determining that the hole in the wall image is a screw hole; if the comparison result is that the position of the hole is inconsistent with the position of the screw hole, based on the comparison result, counting the false detection and missed detection screw holes in the first detection result relative to the second detection result; extracting the confidence and detection probability of the screw hole, and based on the confidence and the detection probability, screening the false detection and missed detection screw holes to obtain the final detection result of the screw hole.

[0009] Optionally, in a fifth implementation form of the first aspect of the present application, if the position of the hole is inconsistent with the position of the screw hole and the position of the screw hole is a false detection position, the step of extracting the confidence and detection probability of the screw hole, and based on the confidence and the detection probability, screening the false detection and missed detection screw holes to obtain the final detection result of the screw hole, comprises: extracting the confidence of the false detection position in the first detection result and the detection probability of the false detection position in the second detection result; judging whether the detection probability corresponding to each false detection position is easy to be false detected and has a large probability of not existing a screw hole; if yes, removing the false detection position corresponding to the easy-to-be-false-detected and large-probability-of-not-existing-screw-hole from the first detection result to obtain the final detection result of the screw hole; if no, judging whether the confidence corresponding to the false detection position not corresponding to the easy-to-be-false-detected and large-probability-of-not-existing-screw-hole is less than a preset confidence threshold; if yes, removing the false detection position not corresponding to the easy-to-be-false-detected and large-probability-of-not-existing-screw-hole from the first detection result to obtain the final detection result of the screw hole; if no, taking the first detection result as the final detection result of the screw hole.

[0010] Optionally, in the sixth implementation form of the first aspect of the present application, if the positions of the holes and the positions of the screw holes are inconsistent and the positions of the screw holes are missed positions, the confidence and the detection probability of the screw holes are extracted, and the false detection and the missed detection of the screw holes are screened based on the confidence and the detection probability to obtain the final detection result of the screw holes, including: extracting the detection probability of the screw hole corresponding to the missed position in the second detection result; judging whether the detection probability corresponding to each missed position is easy to miss and there is a screw hole with a large probability; if it is easy to miss and there is a screw hole with a large probability, the missed position corresponding to the easy-to-miss and large-probability screw hole is added to the first detection result to obtain the final detection result of the screw hole; if it is not easy to miss and there is a screw hole with a large probability, it is judged whether the screw hole of the missed position meets the screw hole characteristics, if it meets, the missed position is added to the first detection result to obtain the final detection result of the screw hole; if it does not meet, the first detection result is taken as the final detection result of the screw hole.

[0011] The second aspect of the present application provides a screw hole detection device, including: an acquisition module, configured to acquire a wall image to be detected and a template wall image, wherein the template wall is an actual wall in the same building determined based on similarity with the wall to be detected; a first identification module, configured to identify holes in the wall image to be detected and first position information of each hole on the wall to be detected, and determine the confidence of each hole according to the first position information and the corresponding hole characteristics, and generate a first detection result based on the first position information and the confidence, wherein the confidence is the probability of determining that the hole belongs to a screw hole; a second identification module, configured to identify screw holes in the template wall image, and determine the detection probability of each screw hole based on second position information of each screw hole on the template wall, and generate a second detection result based on the second position information and the detection probability, wherein the detection probability is a probability for indicating that the second position information is missed, false detected or there is a screw hole; a fusion module, configured to fuse the first detection result and the second detection result, and obtain a final detection result of the screw hole based on the fusion result.

[0012] Optionally, in the first implementation manner of the second aspect of the present application, the first identification module comprises: an extraction unit configured to identify holes in the wall surface image to be detected by a position detection processor, and extract holes in the identified result that meet preset hole characteristics; a first calculation unit configured to determine positions of the holes in the wall surface image, and determine first position information of the holes in the wall surface to be detected based on the positions and a preset scaling ratio between the wall surface to be detected and the wall surface image to be detected; and a first generation unit configured to calculate a confidence level of each of the holes based on the first position information of each of the holes and the corresponding hole characteristics, and generate a first detection result based on the first position information and the confidence level of the holes.

[0013] Optionally, in the second implementation manner of the second aspect of the present application, the second identification module comprises: a processing unit configured to identify the template wall surface image by a position detection processor to obtain second position information of each of the screw hole; a second calculation unit configured to extract environmental information on the second position information, and calculate a detection probability of the existence of a screw hole on the second position information based on the environmental information; and a second generation unit configured to generate a second detection result based on the second position information and the detection probability.

[0014] Optionally, in the third implementation manner of the second aspect of the present application, the second calculation unit is specifically configured to: extract screw hole representation information and corresponding construction information in the environmental information; calculate a probability of the existence of a screw hole and probabilities of false detection and missed detection on the second position according to the screw hole representation information and the construction information; and determine the detection probability based on the probability of the existence of a screw hole and the probabilities of false detection and missed detection.

[0015] Optionally, in the fourth implementation manner of the second aspect of the present application, the fusion module comprises: a comparison unit configured to compare positions of the holes in the first detection result with positions of the screw holes in the second detection result to obtain a comparison result; a judgment unit configured to determine that the holes in the wall surface image are screw holes when the comparison result is that the positions of the holes are consistent with the positions of the screw holes; and if the comparison result is that the positions of the holes are inconsistent with the positions of the screw holes, then based on the comparison result, the judgment unit is configured to count false detection and missed detection of screw holes in the first detection result relative to the second detection result; and a screening unit configured to extract confidence levels and detection probabilities of the screw holes, and screen the false detection and missed detection of the screw holes based on the confidence levels and the detection probabilities to obtain a final detection result of the screw holes.

[0016] Optionally, in a fifth implementation form of the second aspect of the present application, the screening unit is specifically configured to: extract the confidence of the screw hole corresponding to the false detection position in the first detection result and the detection probability of the screw hole corresponding to the false detection position in the second detection result; determine whether the detection probability corresponding to each false detection position is easy to be false detected and has a large probability of not existing the screw hole; if yes, the false detection position corresponding to the false detection position easy to be false detected and having a large probability of not existing the screw hole is removed from the first detection result to obtain the final detection result of the screw hole; if not, it is determined whether the confidence corresponding to the false detection position not easy to be false detected and having a large probability of not existing the screw hole is less than a preset confidence threshold; if yes, the false detection position not easy to be false detected and having a large probability of not existing the screw hole is removed from the first detection result to obtain the final detection result of the screw hole; if not, the first detection result is taken as the final detection result of the screw hole.

[0017] Optionally, in a sixth implementation form of the second aspect of the present application, the screening unit is further configured to: extract the detection probability of the screw hole corresponding to the missed detection position in the second detection result; determine whether the detection probability corresponding to each missed detection position is easy to be missed detected and has a large probability of existing the screw hole; if yes, the missed detection position corresponding to the missed detection position easy to be missed detected and having a large probability of existing the screw hole is added to the first detection result to obtain the final detection result of the screw hole; if not, it is determined whether the screw hole corresponding to the missed detection position meets the screw hole feature; if yes, the missed detection position is added to the first detection result to obtain the final detection result of the screw hole; if not, the first detection result is taken as the final detection result of the screw hole.

[0018] The third aspect of the present application provides a screw hole detection device, which comprises a memory and at least one processor, the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the screw hole detection device to perform each step of the screw hole detection method.

[0019] The fourth aspect of the present application provides a computer readable storage medium, which stores instructions, and the instructions are executed by a processor to implement each step of the screw hole detection method.

[0020] In the technical solution provided by the present application, the positions of holes and screw rod holes in a wall surface image of a wall surface to be constructed and a template wall surface are recognized, the confidence of the holes and the detection probability of the screw rod holes are calculated, a first detection result and a second detection result are generated, the first detection result and the second detection result are fused, and the detection result of the screw rod holes in the wall surface image is obtained. Compared with the prior art, the method fuses the detection result of the position of the screw rod holes and the reuse detection result of the screw rod hole template, the detection effects of the two can be complementary after fusion, and thus the detection accuracy of the screw rod holes is improved. BRIEF DESCRIPTION OF DRAWINGS

[0021] Figure 1 A first embodiment schematic diagram of a screw rod hole detection method provided by an embodiment of the present application is provided.

[0022] Figure 2 A second embodiment schematic diagram of a screw rod hole detection method provided by an embodiment of the present application is provided.

[0023] Figure 3 A third embodiment schematic diagram of a screw rod hole detection method provided by an embodiment of the present application is provided.

[0024] Figure 4 A flowchart of wall surface image recognition by a position detection processor provided by an embodiment of the present application is provided.

[0025] Figure 5 A detection result schematic diagram of a template wall surface provided by an embodiment of the present application is provided.

[0026] Figure 6 A structural schematic diagram of a screw rod hole detection device provided by an embodiment of the present application is provided.

[0027] Figure 7 Another structural schematic diagram of a screw rod hole detection device provided by an embodiment of the present application is provided.

[0028] Figure 8 A structural schematic diagram of a screw rod hole detection device provided by an embodiment of the present application is provided. DETAILED DESCRIPTION

[0029] For the existing screw rod hole detection method, the position detection processor and the reuse detection result of the screw rod hole template are fused to obtain the final position of the screw rod hole, and the detection accuracy of the screw rod hole is improved.

[0030] The terms "first", "second", "third", "fourth" and the like in the description and in the claims of the present application, and above-mentioned drawings, if any, are used to distinguish between similar objects and not necessarily for describing a particular sequential or chronological order. It is to be understood that the use of the terms so-termed "first", "second", "third", "fourth" and the like, if any, in the description and in the claims of the present application is not used to designate a certain order or chronology, but to distinguish between similar objects. It is to be understood that the data so used can be interchanged, where appropriate, so that the embodiments described herein can be carried out in an order other than the one illustrated or described herein. Furthermore, the terms "comprising" or "having" and any variations thereof, are intended to cover a non-exclusive inclusion, for example, a process, method, article, or apparatus that comprises a list of steps or units not necessarily limited to those explicitly listed, but can include other steps or units not expressly listed or inherent to such process, method, article, or apparatus.

[0031] For the sake of understanding, the specific flow of the embodiments of the present application is described below. Please refer to Figure 1 The first embodiment of the screw hole detection method provided by the embodiments of the present application is shown in the figure, and the method specifically comprises the following steps:

[0032] 101, acquire the wall surface image to be detected and the template wall surface image.

[0033] Among them, the template wall surface is an actual wall surface in the same building which is determined based on similarity with the wall surface to be constructed, in this embodiment, the position of the screw hole detected in the wall surface to be constructed is further screened according to the position and existence of the screw hole in the template wall surface, so as to obtain a detection result of the screw hole with higher accuracy. The acquisition of the wall surface image to be detected can be realized by camera acquisition of the wall surface image. Considering that the image acquired by the camera may have problems such as unobvious target features and excessive noise, laser radar can be used to acquire the wall surface image. The time or frequency of the reflected signal collected by the collecting module is compared to obtain the three-dimensional image of the wall surface to be detected. More preferably, an IR camera can be used in combination with a light source to acquire infrared information of the wall surface to be detected, or an RGB camera can be used in combination with an IR camera to acquire infrared information and color information of the wall surface to be detected. The template wall surface is an actual wall surface with similarity to the wall surface to be detected. In the construction engineering, there are wall surfaces with similar screw hole positions in the same building as the wall surface to be constructed, which are used as template wall surfaces. The screw hole positions in different floors are approximately the same, accounting for 80%-90%. If the position of one floor is reused as a template in other floors, 80%-90% of the screw holes can be directly detected. However, considering that various conditions may cause the screw hole of the actual wall surface to be detected to be inconsistent with the template wall surface during the construction process, that is, there are undetected or misdetected screw holes, therefore, the screw hole position detection result is fused with the screw hole template reuse detection result in this embodiment, so as to improve the detection accuracy of the screw hole.

[0034] 102. Identify the holes in the wall image to be detected and the first location information of each hole on the wall to be detected, and determine the confidence level of each hole based on the first location information and the corresponding hole features, and generate the first detection result based on the first location information and the confidence level.

[0035] The confidence level is the probability of determining that the hole belongs to a screw hole. Image processing is performed on the acquired image of the wall surface to obtain the hole location. This embodiment does not limit the specific form of the acquired wall image; depending on the acquisition device, the acquired image will be different, and the image processing techniques used will also be different. For example, an SEM image of the wall surface can be acquired, the SEM image can be converted into a grayscale matrix, and the grayscale matrix area can be divided into a base region and a hole region according to the grayscale values. Analyzing the hole region yields the first location information of the hole's center, i.e., the hole location.

[0036] Hole localization can be achieved using point cloud information. The process involves acquiring an environmental image from the current viewpoint, dividing the point cloud into blocks using the RANSAC algorithm, and sequentially selecting seed points from each block to obtain planar parameters. Intra-point judgment is performed based on the color and geometric information of the environmental image, and the planar parameters are re-estimated. A region growing algorithm based on RGB-D information is used to obtain the point cloud clusters for each hole from the current viewpoint, and hole localization is achieved using this point cloud information. In practical applications, due to the inconsistency in signal reflection time between the hole and the plane, an ultrasonic position acquisition device can be used to acquire the hole's position. For ease of subsequent data comparison, a coordinate system is established on the wall surface to be constructed. The position coordinates of the hole's center, with the measured wall surface as the coordinate system, can be obtained based on the relationship between the hole's position and the wall surface.

[0037] The confidence score for each hole is calculated. The confidence score can be equal to or determined as a function of the corresponding degree. If the image processing result is in the form of a result image, a confidence map can be generated by determining the confidence score for different image pixels or image regions of the result image. In this case, the confidence score can be the pixel value of the confidence map. The confidence score is related to the similarity between the corresponding pixels or image regions of different result images. The confidence score is a number close to 1, indicating the reliability of the event. In this embodiment, the value range can be set to 0 to 1. If the position detection processor cannot calculate the confidence score, it can be set to 1. When the confidence score is 1, the probability of the event occurring is 100%. In this embodiment, when the confidence score of the hole is 1, the hole is a screw hole.

[0038] 103. Identify the screw holes in the template wall image, determine the detection probability of each screw hole based on the second position information of each screw hole on the template wall, and generate a second detection result based on the second position information and the detection probability.

[0039] The detection probability is used to represent a probability that the second position information is missed detection, false detection, or exists a screw hole. The template wall surface is an actual wall surface in the same building which is determined based on similarity with the wall surface to be constructed. According to the template wall surface, positions of most screw holes in the wall surface to be constructed can be obtained. Specifically, screw holes in the template wall surface image are recognized to obtain positions of the screw holes in the template wall surface. Then, the positions are converted into second position information in a coordinate system of the wall surface to be constructed. When the second position information is used to determine the detection probability of each screw hole, environmental information around the position and a characteristic representation of the screw hole corresponding to the environmental information can be obtained. The possibility that the screw hole is detected, i.e., the detection probability, is analyzed.

[0040] 104. The first detection result and the second detection result are fused, and a final detection result of the screw hole is obtained based on a result of the fusion.

[0041] The first position information in the first detection result is added or removed from the corresponding missed detection or false detection position to obtain a final position of the screw hole. The fusion includes comparing position information which is inconsistent in the two detection results, and then selecting the inconsistent position information based on one of the detection results as a comparison reference. The detection result as the comparison reference is updated to obtain the final detection result of the screw hole. In this embodiment, the first detection result and the second detection result are compared. Missed detection and false detection of the first detection result relative to the second detection result are counted according to the position information of the screw hole. The false detection position information is the position information which appears in the first detection result but does not appear in the second detection result. The missed detection position information is the position information which appears in the second detection result but does not appear in the first detection result. In actual application, the second detection result can also be used as the comparison reference. The relevant position information in the first detection result can be added or removed from the second detection result.

[0042] The missed detection and false detection position information is determined whether it is real missed detection and real false detection. Specifically, for the missed detection position information, whether it is real missed detection is determined according to its existence in the second detection result and its environmental information in the first detection result. For the false detection position information, whether it is real false detection is determined according to its existence in the second detection result and its confidence in the first detection result. The position information which is determined as real false detection is removed from the first detection result. The second position information in the second detection result which is determined as real missed detection is added to the first position information. The first detection result is updated according to the removal and addition operations to obtain the detection result of the screw hole in the wall surface image.

[0043] More preferably, the template wall surface is updated according to the final screw hole detection result, and the accuracy of the screw hole detection is improved. The screw hole detection results of different floors can be obtained by fusing the detection result of the position detection processor and the template reuse result. On the other hand, there is a great similarity between different floors of the same building, that is, 80%-90% of the screw hole positions of different floors are approximately the same. If the position of one floor is used as a template and reused to other floors, 80%-90% of the screw holes can be directly detected. The precision can be improved by dynamically updating the adjacent floor as a template. Assuming that the first floor is selected as a template, the second floor screw hole prediction result can be obtained by the fusion method. When calculating the result of the third floor, the second floor prediction result can be used as a new template, and then the third floor screw hole prediction result can be obtained by the fusion method. In this way, the dynamic updating of the adjacent floor as a template can be used to further improve the prediction accuracy of the screw hole.

[0044] The scheme identifies the positions of the holes and screw holes in the wall surface image of the wall surface to be constructed and the template wall surface, calculates the confidence of the holes and the detection probability of the screw holes, generates the first detection result and the second detection result, and fuses the first detection result and the second detection result to obtain the detection result of the screw holes in the wall surface image, thereby improving the accuracy of the screw hole detection.

[0045] Please refer to Figure 2 The second embodiment of the screw hole detection method provided by the embodiment of the present application is shown in the figure, and the method specifically includes the following steps:

[0046] 201, acquire a wall surface image to be detected and a template wall surface image.

[0047] The structured light generator is used to send structured light to a target object, at least one image sensor is used to acquire structured light images of the structured light from different positions and at different angles, then the depth value corresponding to each pixel in the image acquired by the visible light image sensor (for example, an RGB sensor) is calculated according to the structured light images sent by the image sensor, a depth image of the measured wall surface is obtained, the red, green and blue three components can be measured and reconstructed respectively, and then the three components are fused to obtain a color image of the measured wall surface. The image features of the acquired wall surface images such as the depth image and the color image are processed, including using the structured light, the Time Of Flight (TOF) method, the Stereo Matching method and the like to obtain the position information of the screw hole center position, and a coordinate system is established to obtain the first position information of the screw hole center position.

[0048] 202, identify the holes in the wall surface image to be detected by a position detection processor, and extract the holes in the identified result that meet the preset hole characteristics.

[0049] The hole extraction method in the prior art is mainly based on point cloud data processing. Specifically, the hole extraction based on scattered point cloud is mainly to extract the boundary points of the scattered point cloud according to the maximum angle between adjacent vectors in the local reference point set, to fit a micro tangent plane according to the sampling point and its K-neighborhood points, to project it to the micro tangent plane, to construct the vector from the sampling point to its K-neighborhood, and to extract the hole feature point by the maximum angle between adjacent vectors. In the embodiment, the wall surface image obtained is subjected to hole extraction, and the hole features extracted are analyzed to determine whether they conform to the hole features. The hole features are black in the middle and bright around, and the comparison standard is determined according to the performance of the hole features in different wall surface images when compared.

[0050] 203、According to the positions of the holes in the wall surface image to be detected, first position information of the holes on the wall surface to be detected is obtained, and the confidence of each hole is calculated based on the first position information and the corresponding hole features.

[0051] The positions of the holes in the wall surface image to be detected are determined, and based on the positions and a preset scaling ratio between the wall surface to be detected and the wall surface image to be detected, first position information of the holes on the wall surface to be detected is calculated; the confidence of each hole is calculated based on the first position information of each hole and the corresponding hole features, and a first detection result is generated based on the first position information and the confidence of the hole. Based on the equal scaling up or down relationship between the wall surface image to be detected and the wall surface to be detected, that is, the scaling ratio, a coordinate system is established in the wall surface image, and the wall surface image can also be projected into the wall surface to be detected in equal scale according to the mapping relationship of the wall surface image, to obtain the coordinate representation of the image features. For the holes in the wall surface image that conform to the hole features, the center position of the hole is obtained. If the shape of the hole is circular, the center of the hole is calculated as the position information of the hole. If the hole is an irregular shape, the center position of the hole is fitted and estimated according to the plurality of boundary information collected, the hole center values fitted and estimated multiple times are averaged, and the position information of the hole, that is, the first position information, is obtained.

[0052] 204、The template wall surface image is recognized by the position detection processor, and second position information of each screw hole is obtained.

[0053] 205、The environmental information on the second position information is extracted, the detection probability of the existence of the screw hole on the second position information is calculated based on the environmental information, and a second detection result is generated based on the second position information and the detection probability.

[0054] Environmental information is extracted within a circle centered on the second location information and with a preset distance as the radius. The screw hole characterization information and corresponding construction information are extracted from this environmental information. Based on the screw hole characterization information and the construction information, the probability of a screw hole existing at the second location, as well as the probabilities of false detection and missed detection, are calculated. Based on the probability of the screw hole's existence and the probabilities of false detection and missed detection, the detection probability is determined. In practical applications, the screw hole characterization information and the construction information within the specified range, along with the screw hole's appearance and the scaffolding's erection status, can be input into a preset model to obtain the screw hole's existence probability. The screw hole's existence probability characterizes the presence of the screw hole, including situations where it is easily missed but has a high probability of existence, easily falsely detected but has a high probability of not existing, and other situations where a screw hole may exist. Based on the existence situation, the corresponding detection probability is obtained from a probability correspondence table to obtain the screw hole's detection probability.

[0055] 206. Compare the positions of the holes in the first test result with the screw holes in the second test result to obtain the comparison results.

[0056] Extract the first position information of the hole from the first detection result and the second position information of the screw hole from the second detection result, and compare the two. Specifically, the second position information can be used as the comparison benchmark. The position of each hole in the first position information is matched and compared with all the second position information to determine whether the values ​​are the same.

[0057] 207. If the comparison result shows that the position of the hole matches the position of the screw hole, then the hole in the wall image is determined to be the screw hole.

[0058] Extract the first position information of the hole and the second position information of the screw hole. The position information is a numerical coordinate in the same coordinate system. Determine whether the first position information and the second position information are in one-to-one correspondence. Considering that the wall image and the template image are scaled-down versions of the wall being measured at different ratios, and calculations were performed when determining the specific coordinate information, when the first position information and the second position information are not in one-to-one correspondence, but there is only a small error between the values, it is considered that the position information of the two are in one-to-one correspondence and there is a correlation. The hole in the wall image is determined to be a screw hole.

[0059] 208. If the comparison result shows that the position of the hole is inconsistent with the position of the screw hole, then based on the comparison result, the number of screw holes that are falsely detected and missed in the first detection result relative to the second detection result will be counted.

[0060] When the numerical values of the coordinate positions of the two are inconsistent and differ greatly, the position information that is inconsistent and exists alone in the two is extracted, and it is determined according to the original source of the coordinate position that exists alone whether the position information belongs to a false detection or a missed detection, wherein the original source is the detection result of the screw hole corresponding to the coordinate position that exists alone, for example, if the original source of the coordinate position that exists alone is the first detection result, it is a false detection, and if the original source of the coordinate position that exists alone is the second detection result, it is a missed detection.

[0061] 209、When the position of the hole is inconsistent with the position of the screw hole and the position of the screw hole is a false detection position, the confidence and detection probability of the screw hole corresponding to the false detection position are extracted, and the false detection screw hole is screened based on the confidence and detection probability to obtain the final detection result of the screw hole.

[0062] The confidence of the false detection position in the first detection result and the detection probability in the second detection result are extracted; it is judged whether the detection probability corresponding to each false detection position is easy to be false and has a large probability of not existing a screw hole; if it is easy to be false and has a large probability of not existing a screw hole, the false detection position corresponding to the false detection position is removed from the first detection result, and the final detection result of the screw hole is obtained; if it is not easy to be false and has a large probability of not existing a screw hole, it is judged whether the confidence corresponding to the false detection position corresponding to the false detection position is less than a preset confidence threshold; if it is less than, the false detection position corresponding to the false detection position is removed from the first detection result, and the final detection result of the screw hole is obtained; if it is not less than, the first detection result is taken as the final detection result of the screw hole.

[0063] The confidence threshold is a preset accuracy of the hole at a specified position being a screw hole, and the value is 0-1. In order to obtain as many screw hole positions as possible, it can be set to 0.85, that is, 85%. Specifically, it is judged whether the false detection screw hole of the position detection processor is adjacent to the screw hole marked as easy to be false and having a large probability of not existing in the template. If adjacent, it is determined that the corresponding screw hole position indeed belongs to a false detection, and the screw hole is removed from the first detection result. For the position detection processor false detection screw hole that is not removed in the above, a confidence threshold is set. If the confidence is less than the threshold, it is determined that the corresponding screw hole position indeed belongs to a false detection, and the screw hole is removed from the first detection result. For the position detection processor false detection screw hole that is not removed in the above, it is reserved in the first detection result, that is, it is determined that this part of the screw hole truly exists and does not belong to a false detection. Based on the above operation, the final detection result of the screw hole is obtained.

[0064] 210、If the position of the hole is inconsistent with the position of the screw hole and the position of the screw hole is a missed detection position, the detection probability of the screw hole is extracted, and the missed detection screw hole is screened based on the detection probability to obtain the final detection result of the screw hole.

[0065] The detection probability of the missed detection position corresponding to the screw hole in the second detection result is extracted, and it is judged whether the detection probability corresponding to each of the false detection positions is a position that is easy to miss and has a relatively large probability of existing screw hole, if so, the missed detection position corresponding to the position that is easy to miss and has a relatively large probability of existing screw hole is added to the first detection result to obtain the final detection result of the screw hole, if not, it is judged whether the screw hole of the missed detection position meets the screw hole feature, if so, the missed detection position is added to the first detection result to obtain the final detection result of the screw hole, if not, the first detection result is taken as the final detection result of the screw hole.

[0066] Specifically, if the position detection processor misses the existence of the screw hole in the template is easy to miss and has a relatively large probability of existing, the corresponding screw hole in the template is added to the first detection result, and for the position detection processor missed screw hole not added to the first detection result, it is detected whether it meets the screw hole feature, if so, the corresponding screw hole in the template is added to the first detection result, and for the position detection processor missed screw hole not added to the first detection result, it is determined that the corresponding screw hole does not belong to the missed detection, that is, the corresponding screw hole does not exist. Based on the above operation, the final detection result of the screw hole is obtained.

[0067] In the scheme, the positions of the holes and screw holes in the measured wall surface and the corresponding template wall surface are obtained by different ways, two different screw hole detection results of the same measured wall surface are obtained, the screw hole detection fusion method described in the scheme compares and screens the first detection result and the second detection result to obtain the final screw hole detection result based on the first detection result, which is beneficial to the fusion of subsequent detection results and further improves the detection accuracy.

[0068] Please refer to Figure 3 A third embodiment of the screw hole detection method provided by the embodiment of the application is shown in the figure, and the method specifically comprises the following steps:

[0069] 301、Obtain the wall surface image of the wall surface to be detected by the position detection processor to obtain the position detection result.

[0070] Please refer to Figure 4The position detection processor provided by the embodiment of the application provides a flowchart for wall surface image recognition. The position detection processor collects wall surface images through a laser radar or a camera, and then detects the positions and confidence levels of screw rod holes by using necessary traditional image processing or deep learning modules. The screw rod holes detected by the position detection processor are represented by Points_a. Points_a includes a plurality of screw rod holes. The position and confidence level of each screw rod hole are represented by Xa, Ya and C. Xa and Ya represent the center position of the screw rod hole, and C represents the confidence level, which has a value range of 0-1. If the position detection processor used cannot calculate the confidence level, C can be set to 1.

[0071] 302. Obtain a template wall surface corresponding to the measured wall surface and perform screw rod hole detection to obtain a screw rod hole template detection result.

[0072] The position detection processor is used to detect screw rod holes in the template wall surface. Positions where the position detection processor is prone to miss detection and where screw rod holes are likely to exist are marked as P. Positions where the position detection processor is prone to false detection and where screw rod holes are unlikely to exist are marked as N. Other positions where screw rod holes may exist are marked as O. The prepared screw rod hole template is represented by Points_t. Points_t includes a plurality of screw rod holes. Each screw rod hole is represented by a position and an existence condition. The center position of the screw rod hole is represented by Xt and Yt, and the existence condition is represented by one of P, N and O. Please refer to Figure 5 The detection result diagram of the template wall surface provided by the embodiment of the application represents the existence conditions of the screw rod holes marked in different styles.

[0073] 303. Fuse the position detection result and the template detection result to obtain a final screw rod hole detection result.

[0074] The position detection processor detection result Points_a is compared with the template detection result Points_t. The false detection FP and the missed detection FN of the position detection processor detection result relative to the template detection result are counted according to the position information of the screw rod hole.

[0075] For the false detection screw rod hole FP of the position detection processor, the following processing steps are used. It is determined whether the false detection screw rod hole FP of the position detection processor is adjacent to the screw rod hole marked as N in the template. If adjacent, it is determined that the corresponding screw rod hole position is indeed false detection, and the screw rod hole is removed from Points_a. For the false detection screw rod hole FP of the position detection processor that is not removed, a confidence level threshold TC is set. If the confidence level is less than TC, it is determined that the corresponding screw rod hole position is indeed false detection, and the screw rod hole is removed from Points_a. For the false detection screw rod hole FP of the position detection processor that is not removed, the screw rod hole is retained in Points_a, that is, it is determined that this part of the screw rod hole truly exists and is not false detection.

[0076] For the position detection processor missing screw hole FN, the processing steps are as follows: if the position detection processor missing screw hole FN in the template is P, the corresponding screw hole in the template is added to Points_a; for the position detection processor missing screw hole FN which is not added to Points_a, it is detected whether it meets the hole feature, such as the existence of black circular contour, if it meets the hole feature, the corresponding screw hole in the template is added to Points_a; for the position detection processor missing screw hole FN which is not added to Points_a, it is determined that the corresponding screw hole does not belong to the missing, that is, the corresponding screw hole does not exist. The updated Points_a is used as the final screw hole detection result of the scheme.

[0077] The scheme fuses the detection results of the screw holes of the measured wall surface detected by the position detection processor and the detection results of the screw holes of the template wall surface, realizes the fusion of the position detection processor and the template for detecting screw holes, obtains the final position of the screw hole based on the fusion result, and improves the detection accuracy of the screw hole.

[0078] The screw hole detection method in the embodiment of the application is described above, and the screw hole detection device in the embodiment of the application is described in detail from the perspective of modularized functional entities. Please refer to Figure 6 The screw hole detection device provided by the embodiment of the application has a structure diagram, which includes:

[0079] The acquisition module 401 is configured to acquire a wall surface image to be detected and a template wall surface image, wherein the template wall surface is an actual wall surface determined based on similarity with the wall surface to be detected in the same building.

[0080] The first identification module 402 is configured to identify holes in the wall surface image to be detected and first position information of each hole in the wall surface to be detected, and determine a confidence degree of each hole according to the first position information and a corresponding hole feature, and generate a first detection result based on the first position information and the confidence degree, wherein the confidence degree is a probability of determining that the hole belongs to a screw hole.

[0081] The second identification module 403 is configured to identify screw holes in the template wall surface image, and determine a detection probability of each screw hole based on second position information of each screw hole in the template wall surface, and generate a second detection result based on the second position information and the detection probability, wherein the detection probability is a probability for indicating that the second position information is missed, misdetected or exists a screw hole.

[0082] The fusion module 404 is configured to fuse the first detection result and the second detection result, and obtain a final detection result of the screw hole based on a result of the fusion.

[0083] By identifying the positions of the holes and the screw holes in the wall surface image of the wall surface to be constructed and the template wall surface image, calculating the confidence of the holes and the detection probability of the screw holes, generating the first detection result and the second detection result, and fusing the first detection result and the second detection result, the detection result of the screw hole in the wall surface image is obtained, and the accuracy of the detection of the screw hole is improved.

[0084] Referring to Figure 7 Another structural diagram of the screw hole detection device provided by the embodiment of the present application comprises:

[0085] The acquisition module 401 is configured to acquire a wall surface image to be detected and a template wall surface image, wherein the template wall surface is an actual wall surface determined based on similarity with the wall surface to be detected in the same building;

[0086] The first identification module 402 is configured to identify holes in the wall surface image to be detected and first position information of each hole on the wall surface to be detected, determine the confidence of each hole based on the first position information and corresponding hole features, and generate a first detection result based on the first position information and the confidence, wherein the confidence is a probability of determining that the hole is a screw hole.

[0087] The second identification module 403 is configured to identify screw holes in the template wall surface image, determine the detection probability of each screw hole based on second position information of each screw hole on the template wall surface, and generate a second detection result based on the second position information and the detection probability, wherein the detection probability is a probability of indicating that the second position information is missed, misdetected, or exists a screw hole.

[0088] The fusion module 404 is configured to fuse the first detection result and the second detection result, and obtain a final detection result of the screw hole based on a result of the fusion.

[0089] In the embodiment, the first identification module 402 comprises:

[0090] The extraction unit 4021 is configured to identify holes in the wall surface image to be detected by a position detection processor, and extract holes in the identified results that meet preset hole features;

[0091] The first computing unit 4022 is configured to determine positions of the holes in the wall surface image to be detected, and determine first position information of the holes in the wall surface to be detected based on the positions and a preset scaling ratio between the wall surface to be detected and the wall surface image to be detected.

[0092] The first generating unit 4023 is configured to calculate a confidence of each hole based on the first position information of the hole and the corresponding hole feature, and generate a first detection result based on the first position information and the confidence of the hole.

[0093] In this embodiment, the second identification module 403 includes:

[0094] The processing unit 4031 is configured to identify the template wall surface image by a position detection processor to obtain second position information of each screw hole.

[0095] The second computing unit 4032 is configured to extract environmental information on the second position information, and calculate a detection probability of the screw hole existing on the second position information based on the environmental information.

[0096] The second generating unit 4033 is configured to generate a second detection result based on the second position information and the detection probability.

[0097] In this embodiment, the second computing unit 4033 is specifically configured to extract screw hole representation information and corresponding construction information in the environmental information, calculate a probability of the screw hole existing on the second position and probabilities of false detection and missed detection according to the screw hole representation information and the construction information, and determine the detection probability based on the probability of the screw hole existing and the probabilities of false detection and missed detection.

[0098] In this embodiment, the fusion module 404 includes:

[0099] The comparison unit 4041 is configured to compare positions of the holes in the first detection result with positions of the screw holes in the second detection result to obtain a comparison result.

[0100] The judging unit 4042 is configured to determine that the holes in the wall surface image are screw holes when the comparison result is that the positions of the holes are consistent with the positions of the screw holes, and determine false detection and missed detection of the screw holes existing in the first detection result relative to the second detection result based on the comparison result when the comparison result is that the positions of the holes are inconsistent with the positions of the screw holes.

[0101] The screening unit 4043 is configured to extract the confidence and detection probability of the screw hole, and screen the false detection and missed detection screw holes based on the confidence and detection probability, to obtain a final detection result of the screw hole.

[0102] In this embodiment, the screening unit 4043 is specifically configured to extract the confidence of the false detection position in the first detection result and the detection probability of the false detection position in the second detection result; determine whether the detection probability of each false detection position is easy to be false detected and has a large probability of not existing a screw hole; if yes, the false detection position corresponding to the false detection position that is easy to be false detected and has a large probability of not existing a screw hole is removed from the first detection result, to obtain the final detection result of the screw hole; if no, it is determined whether the confidence of the false detection position that is not easy to be false detected and has a large probability of not existing a screw hole is less than a preset confidence threshold; if yes, the false detection position that is not easy to be false detected and has a large probability of not existing a screw hole is removed from the first detection result, to obtain the final detection result of the screw hole; if no, the first detection result is taken as the final detection result of the screw hole.

[0103] In this embodiment, the screening unit 4043 is further configured to extract the detection probability of the missed detection position in the second detection result; determine whether the detection probability of each missed detection position is easy to be missed detected and has a large probability of existing a screw hole; if yes, the missed detection position corresponding to the missed detection position that is easy to be missed detected and has a large probability of existing a screw hole is added to the first detection result, to obtain the final detection result of the screw hole; if no, it is determined whether the screw hole of the missed detection position meets the screw hole feature; if yes, the missed detection position is added to the first detection result, to obtain the final detection result of the screw hole; if no, the first detection result is taken as the final detection result of the screw hole.

[0104] Through the implementation of the above scheme, when the detection result of the screw hole is obtained, the position detection processor detection result and the template reuse result are fused, and the method of dynamically updating the adjacent floor as the template is further adopted, so that the accuracy of the screw hole detection is further improved.

[0105] The above Figures 6-7 The screw hole detection device in the embodiment of the application is described in detail from the perspective of a modular functional entity, and the screw hole detection device in the embodiment of the application is described in detail from the perspective of hardware processing.

[0106] Figure 8is a structural schematic diagram of a screw hole detection device provided by an embodiment of the present application. The screw hole detection device 600 can have great differences due to different configurations or performances, and can include one or more processors (central processing units, CPUs) 610 (for example, one or more processors) and a memory 620, one or more storage media 630 (for example, one or more mass storage devices) storing application programs 633 or data 632. The memory 620 and the storage media 630 can be temporary storage or persistent storage. The programs stored in the storage media 630 can include one or more modules (not shown in the figure), and each module can include a series of instruction operations in the screw hole detection device 600. Further, the processor 610 can be configured to communicate with the storage media 630 and execute the series of instruction operations in the storage media 630 on the screw hole detection device 600 to implement the method provided by the above embodiment.

[0107] The screw hole detection device 600 can further include one or more power supplies 640, one or more wired or wireless network interfaces 650, one or more input / output interfaces 660, and / or one or more operating devices 631, such as Windows Serve, Mac OS X, Unix, Linux, FreeBSD, and the like. Those skilled in the art can understand that the screw hole detection device structure shown does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements. Figure 8 The screw hole detection device structure shown does not constitute a limitation on the computer device provided by the present application, and can include more or fewer components than shown, or combine certain components, or different component arrangements.

[0108] The present application also provides a computer readable storage medium, which can be a non-volatile computer readable storage medium or a volatile computer readable storage medium. The computer readable storage medium has instructions stored therein, and when the instructions are run on a computer, the computer executes the steps of the screw hole detection method provided by the above embodiments.

[0109] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working process of the above-described device or apparatus, unit can refer to the corresponding process in the foregoing method embodiments, which will not be described here.

[0110] The integrated unit, if implemented in the form of a software function unit and sold or used as an independent product, can be stored in a computer readable storage medium. Based on such understanding, the technical solutions of the present application or the entire or part of the technical solutions that essentially contribute to the prior art can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the method described in the various embodiments of the present application. The aforementioned storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk, and various media that can store program codes.

[0111] The above-described embodiments are merely used to illustrate the technical solutions of the present application, rather than limit the same; even though the present application has been described in detail with reference to the foregoing embodiments, those ordinarily skilled in the art should understand that: they can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacements for some of the technical features; and these modifications or replacements do not cause the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method of detecting a screw hole, characterized by, The screw hole detection method comprises: obtaining a wall image to be detected and a template wall image, wherein the template wall is an actual wall determined based on similarity with the wall to be detected in the same building; identifying holes in the wall image to be detected and first position information of each hole on the wall to be detected, and determining a confidence level of each hole according to the first position information and corresponding hole features, and generating a first detection result based on the first position information and the confidence level, wherein the confidence level is a probability of determining that the hole is a screw hole; identifying screw holes in the template wall image, and determining a detection probability of each screw hole based on second position information of each screw hole on the template wall, and generating a second detection result based on the second position information and the detection probability, wherein the detection probability is a probability of indicating that the second position information is missed, misdetected or exists a screw hole; fusing the first detection result and the second detection result, and obtaining a final detection result of the screw hole based on the fusion result.

2. The screw channel detection method of claim 1, wherein, The identification of holes in the wall image to be detected and the first position information of each hole on the wall to be detected, and the determination of the confidence level of each hole according to the first position information and the corresponding hole features, and the generation of the first detection result based on the first position information and the confidence level, comprise: identifying holes in the wall image to be detected by a position detection processor, and extracting holes in the identified results that meet the preset hole features; determining the position of each hole in the wall image to be detected, and based on the position and the preset scaling ratio between the wall to be detected and the wall image to be detected, determining the first position information of the hole on the wall to be detected based on the position; calculating the confidence level of each hole based on the first position information of each hole and the corresponding hole features, and generating the first detection result based on the first position information and the confidence level of the hole.

3. The screw channel detection method of claim 1, wherein, The identification of screw holes in the template wall image, and the determination of the detection probability of each screw hole based on the second position information of each screw hole on the template wall, and the generation of the second detection result based on the second position information and the detection probability, comprise: identifying the template wall image by a position detection processor to obtain the second position information of each screw hole; extracting environmental information on the second position information, and calculating the detection probability of the screw hole on the second position information based on the environmental information; generating the second detection result based on the second position information and the detection probability.

4. The screw channel detection method of claim 3, wherein, The calculation of the detection probability of the screw hole on the second position information based on the environment, comprises: extracting screw hole representation information and corresponding construction information in the environmental information; calculating the probability of existence of the screw hole on the second position and the probabilities of misdetection and missed detection according to the screw hole representation information and the construction information; determining the detection probability based on the probability of existence of the screw hole and the probabilities of misdetection and missed detection.

5. The screw channel detection method of any one of claims 1-4, wherein, The first detection result and the second detection result are fused, and a final detection result of the screw hole is obtained based on a result of the fusion processing, including: Position comparison of the hole in the first detection result and the screw hole in the second detection result is performed to obtain a comparison result; If the comparison result is that the position of the hole is consistent with the position of the screw hole, the hole in the wall image is determined to be a screw hole; If the comparison result is that the position of the hole is inconsistent with the position of the screw hole, based on the comparison result, misjudged and missed screw holes existing in the first detection result relative to the second detection result are counted; Confidence and detection probability of the screw hole are extracted, and the misjudged and missed screw holes are screened based on the confidence and the detection probability to obtain the final detection result of the screw hole.

6. The screw channel detection method of claim 5, wherein, If the position of the hole is inconsistent with the position of the screw hole and the position of the screw hole is a misjudged position, the confidence and the detection probability of the screw hole are extracted, and the misjudged and missed screw holes are screened based on the confidence and the detection probability to obtain the final detection result of the screw hole, including: The confidence of the misjudged position in the first detection result and the detection probability in the second detection result are extracted; It is judged whether the detection probability corresponding to each misjudged position is easy to misjudge and has a large probability of non-existence of the screw hole; If it is easy to misjudge and has a large probability of non-existence of the screw hole, the misjudged position corresponding to the easy misjudgment and the large probability of non-existence of the screw hole is removed from the first detection result to obtain the final detection result of the screw hole; If it is not easy to misjudge and has a large probability of non-existence of the screw hole, it is judged whether the confidence corresponding to the misjudged position which is not easy to misjudge and has a large probability of non-existence of the screw hole is less than a preset confidence threshold; If it is less than, the misjudged position corresponding to the misjudged position which is not easy to misjudge and has a large probability of non-existence of the screw hole is removed from the first detection result to obtain the final detection result of the screw hole; If it is not less than, the first detection result is taken as the final detection result of the screw hole.

7. The screw channel detection method of claim 5, wherein, If the position of the hole is inconsistent with the position of the screw hole and the position of the screw hole is a missed position, the confidence and the detection probability of the screw hole are extracted, and the misjudged and missed screw holes are screened based on the confidence and the detection probability to obtain the final detection result of the screw hole, including: The detection probability of the missed position corresponding to the screw hole in the second detection result is extracted; It is judged whether the detection probability corresponding to each missed position is easy to miss and has a large probability of existence of the screw hole; If it is easy to miss and has a large probability of existence of the screw hole, the missed position corresponding to the easy misjudgment and the large probability of non-existence of the screw hole is added to the first detection result to obtain the final detection result of the screw hole; If it is not easy to miss and has a large probability of existence of the screw hole, it is judged whether the screw hole of the missed position meets the screw hole characteristics, if yes, the missed position is added to the first detection result to obtain the final detection result of the screw hole; If not, the first detection result is taken as the final detection result of the screw hole.

8. A screw hole detection device characterized by comprising: The screw hole detection device comprises: An acquisition module is configured to acquire a wall image to be detected and a template wall image, wherein the template wall is an actual wall determined based on similarity with the wall to be detected in the same building; A first identification module is configured to identify holes in the wall image to be detected and first position information of each hole on the wall to be detected, determine a confidence level of each hole based on the first position information and a corresponding hole feature, generate a first detection result based on the first position information and the confidence level, wherein the confidence level is a probability of determining that the hole is a screw hole; A second identification module is configured to identify screw holes in the template wall image, determine a detection probability of each screw hole based on second position information of each screw hole on the template wall, and generate a second detection result based on the second position information and the detection probability, wherein the detection probability is a probability of indicating that the second position information is missed, misdetected, or exists a screw hole; A fusion module is configured to fuse the first detection result and the second detection result, and obtain a final detection result of the screw hole based on a result of the fusion processing.

9. A screw hole detection apparatus characterized by comprising: The screw hole detection device comprises a memory and at least one processor, and the memory stores instructions; the at least one processor invokes the instructions in the memory to enable the screw hole detection device to perform each step of the screw hole detection method according to any one of claims 1-7.

10. A computer-readable storage medium having stored thereon instructions, the instructions comprising, The instructions are executed by the processor to implement each step of the screw hole detection method according to any one of claims 1-7.

Images