Intelligent detection method, device and equipment for field wiring scheme

Abstract

The application relates to the technical field of image denoising, and discloses an intelligent detection method, device and equipment for a field wiring scheme, which comprises the following steps: collecting wiring videos from different visual angles, and extracting wiring images of different visual angles frame by frame; marking target pixel points, extracting linear eigenvalues, bending eigenvalues and scattering eigenvalues of the target pixel points, calculating abnormal wiring eigenvalues of the target pixel points, and identifying significant pixel points in the wiring images; calculating the complexity of the significant pixel points and an adjustment threshold value, assigning values to the adjustment threshold value of pixel points which are not significant pixel points, denoising the wiring images according to the adjustment threshold value, and obtaining denoised wiring images of all visual angles; performing image repair, reflection suppression and motion compensation on the denoised wiring images, extracting features of the denoised wiring images, and completing detection of the field wiring scheme. The application can improve the accuracy and stability of wiring scheme detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of image denoising technology, and particularly relates to an intelligent detection method, device and equipment for a field wiring scheme. BACKGROUND

[0002] Construction wiring detection is a process of systematically checking the wiring engineering quality in buildings, factories and other places, mainly checking the physical state of wiring, such as cable direction, connection firmness, etc., to ensure that the wiring meets the design requirements, functional requirements, and safety. The images of the wiring at the construction site can be extracted using video, and then the features of the wiring in the images are extracted, and the wiring detection result is obtained according to the extracted features, replacing the traditional visual inspection and improving the detection efficiency.

[0003] The construction site is relatively chaotic, and there are many factors that interfere with image extraction, such as occlusion of the camera view area by complex scenes, high-reflectivity areas formed by reflective objects, etc. Therefore, the images of the wiring at the construction site need to be denoised and more accurate wiring feature extraction. However, the threshold value of the denoising algorithm is often a preset parameter, and an unreasonable threshold value can affect the extraction of wiring features, and thus affect the accuracy of wiring detection. SUMMARY

[0004] The present application provides an intelligent detection method, device and equipment for a field wiring scheme to solve the problem of unreasonable threshold value affecting the extraction of wiring features during the image denoising process of the wiring, resulting in insufficient accuracy of wiring detection. The technical solution adopted is as follows:

[0005] In a first aspect, an embodiment of the present application provides an intelligent detection method for a field wiring scheme, which comprises the following steps:

[0006] Collect wiring videos from different perspectives, and extract wiring images from different perspectives frame by frame;

[0007] Any one pixel point in the wiring image is recorded as a target pixel point, the linear feature value and the bending feature value of the target pixel point are extracted according to the gradient direction difference between the target pixel point and the adjacent pixel points, the scattering feature value of the target pixel point is extracted according to the gray distribution between the target pixel point and the adjacent pixel points, the abnormal wiring feature value of the target pixel point is calculated in combination with the linear feature value and the bending feature value, and all significant pixel points in the wiring image are identified according to the difference between the abnormal wiring feature values of all pixel points in the same wiring image and the spatial distance between the pixel points;

[0008] According to all significant pixel points in wiring images of all different perspectives, and linear eigenvalue, bending eigenvalue and scattering eigenvalue of pixel points with same positions as the significant pixel points, complexity of each significant pixel point of each perspective is calculated, adjustment threshold of the significant pixel points is calculated according to the complexity of the significant pixel points, and adjustment threshold of pixel points other than the significant pixel points is assigned, wiring images are denoised according to the adjustment threshold, and denoised wiring images of all perspectives are obtained.

[0009] The denoised wiring images are subjected to image repairing, reflection suppression and motion compensation, features of the denoised wiring images are extracted, and detection of the field wiring scheme is completed.

[0010] Further, the specific acquisition method of the linear eigenvalue and the bending eigenvalue of the target pixel point is as follows:

[0011] A feature window of the target pixel point is established, and neighborhood pixel points are marked according to pixel points in the feature window of the target pixel point.

[0012] The absolute value of the difference between the gradient direction of the target pixel point and the neighborhood pixel point is recorded as the first relative gradient direction of the neighborhood pixel point. The difference between the first relative gradient direction and the second relative gradient direction of the neighborhood pixel point is recorded as the second relative gradient direction of the neighborhood pixel point.

[0013] The relative gradient directions of all neighborhood pixel points of the target pixel point are weighted and summed to obtain the linear eigenvalue of the target pixel point.

[0014] The value range of the gradient direction of the pixel point is divided into four value sub-ranges, the coefficient of variation of the gradient direction of all neighborhood pixel points in the feature window of the target pixel point and in the same value sub-range is calculated, and the normalized value of the maximum value of the coefficient of variation is recorded as the bending eigenvalue of the target pixel point.

[0015] Further, the specific acquisition method of the bending eigenvalue is as follows:

[0016] The value range of the gradient direction of the pixel point is divided into four value sub-ranges, the coefficient of variation of the gradient direction of all neighborhood pixel points in the feature window of the target pixel point and in the same value sub-range is calculated, and the normalized value of the maximum value of the coefficient of variation is recorded as the bending eigenvalue of the target pixel point.

[0017] Further, the specific acquisition method of the scattering eigenvalue of the target pixel point is as follows:

[0018] Calculate the gray level co-occurrence matrix of the feature window of the target pixel point in four different directions respectively; the mean value of the inverse difference matrix of the gray level co-occurrence matrix in the four directions is recorded as the scattering feature value of the target pixel point.

[0019] Further, the abnormal wiring feature value of the target pixel point is the weighted sum of the difference between the number 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point, and the difference between the number 1 and the scattering feature value of the target pixel point.

[0020] Further, the identification method of all significant pixel points in the wiring image is:

[0021] According to the difference between the abnormal wiring feature values of all pixel points in the same wiring image and the Euclidean distance between the pixel points, the metric distance of clustering is calculated, all pixel points in the wiring image are clustered, and a cluster is obtained;

[0022] The mean value of the abnormal wiring feature values of all pixel points in the same cluster is recorded as the abnormal feature value of the same cluster, the mean value of the abnormal feature values of all clusters is recorded as the abnormal threshold value, and all pixel points in the cluster whose abnormal feature value is greater than the abnormal threshold value are recorded as significant pixel points.

[0023] Further, the specific calculation of the complexity of each significant pixel point in each view is:

[0024] The vector composed of the linear feature value, the bending feature value and the scattering feature value of the pixel point in the wiring image is recorded as the feature vector of the pixel point;

[0025] Two different wiring images are recorded as the first wiring image and the second wiring image, any significant pixel point in the first wiring image is recorded as the target significant pixel point, and the similarity between the feature vector of the pixel point with the same coordinates as the target significant pixel point in the second wiring image and the target significant pixel point is recorded as the feature similarity of the target significant pixel point determined by the first wiring image and the second wiring image; the mean value of the feature similarity of the target significant pixel point determined by the first wiring image and all wiring images in other views is recorded as the average feature similarity of the target significant pixel point in the first wiring image; the average feature similarity sequence of the first wiring image is established according to the average feature similarity of all significant pixel points in the first wiring image; the first distance between the first wiring image and the second wiring image is recorded as the DTW distance of the average feature similarity sequence of the first wiring image and the second wiring image;

[0026] The average feature similarity of the corresponding target salient pixel point in the average feature similarity sequence of the first wiring image and the second wiring image is removed, and a first removed wiring sequence and a second removed wiring sequence are obtained respectively, the DTW distance of the first removed wiring sequence and the second removed wiring sequence is recorded as a second distance of the first wiring image and the second wiring image, and the absolute value of the difference between the first distance and the second distance of the first wiring image and the second wiring image is recorded as a third distance of the first wiring image and the second wiring image.

[0027] The normalized value of the mean of the first distance and the third distance is recorded as the feature complexity of the target salient pixel point determined by the first wiring image and the second wiring image, and the mean of the feature complexity of the target salient pixel point determined by the first wiring image and all wiring images of different angles is recorded as the complexity of the target salient pixel point.

[0028] Further, the adjustment threshold of the salient pixel point is calculated according to the complexity of the salient pixel point, and the adjustment threshold of the pixel point which is not a salient pixel point is assigned, and the specific steps include:

[0029] The product of the sum of the complexity of the salient pixel point and the number 1 and the universal threshold of the wavelet threshold denoising algorithm is recorded as the adjustment threshold of the salient pixel point.

[0030] The complexity of the pixel point which is not a salient pixel point in the wiring image is assigned as the number 0.

[0031] Further, the wiring image is denoised according to the adjustment threshold, and the specific method includes:

[0032] The adjustment threshold of the pixel point is used as the value of the threshold when the pixel point is denoised, and the wavelet threshold denoising algorithm is used to denoise all wiring images of different angles.

[0033] In a second aspect, the embodiments of the present application provide an intelligent detection device for a field wiring scheme, and the intelligent detection device comprises a wiring image extraction module, a salient pixel point identification module, a denoised image acquisition module and a wiring scheme detection module.

[0034] The wiring image extraction module is used to collect wiring videos from different angles and extract wiring images of different angles frame by frame.

[0035] The significant pixel point recognition module is configured to record any one pixel point in the wiring image as a target pixel point, extract linear feature values and bending feature values of the target pixel point according to differences in gradient directions between the target pixel point and adjacent pixel points, extract scattering feature values of the target pixel point according to gray distribution between the target pixel point and the adjacent pixel points, calculate abnormal wiring feature values of the target pixel point in combination with the linear feature values and the bending feature values, and recognize all significant pixel points in the wiring image according to differences between the abnormal wiring feature values of all pixel points in the same wiring image and spatial distances between the pixel points.

[0036] The denoising image acquisition module is configured to calculate the complexity of each significant pixel point in each view according to all significant pixel points in the wiring images in all different views and the linear feature values, the bending feature values and the scattering feature values of the pixel points at the same positions as the significant pixel points, calculate the adjustment threshold of the significant pixel points according to the complexity of the significant pixel points, assign values to the adjustment threshold of the pixel points that are not significant pixel points, denoise the wiring images according to the adjustment threshold, and acquire the denoised wiring images in all views.

[0037] The wiring scheme detection module is configured to perform image repair, anti-light suppression and motion compensation on the denoised wiring images, extract features of the denoised wiring images and complete detection of the field wiring scheme.

[0038] In a third aspect, the embodiments of the present application further provide an intelligent detection device for a field wiring scheme, which comprises a memory, a processor and a computer program stored in the memory and running on the processor, and the processor implements the steps of the method according to any one of the above aspects when executing the computer program.

[0039] The present application has the following beneficial effects:

[0040] To avoid interference and occlusion issues, this application acquires wiring images from different viewpoints and extracts the linear, curvature, and scattered features of each pixel in each wiring image. It then calculates the abnormal wiring feature value for each target pixel. A larger abnormal wiring feature value indicates a higher probability that the corresponding pixel is located on abnormal wiring. Based on the differences in abnormal wiring feature values ​​among all pixels in the same wiring image and the spatial distance between pixels, all significant pixels in the wiring image are identified. These significant pixels are those that may correspond to abnormal wiring. To more accurately determine the adaptive pre-processing for denoising the wiring image... To improve denoising performance, the complexity of each significant pixel from each viewpoint is calculated, and an adjustment threshold is calculated for each pixel based on the complexity. This adjustment threshold is the most suitable threshold value for denoising the wiring image. The wiring image is then denoised based on the adjusted threshold to obtain denoised wiring images from all viewpoints. Finally, image inpainting, reflection suppression, and motion compensation are performed on the denoised wiring images. The features of the denoised wiring images are extracted, and the on-site wiring scheme detection is completed. This addresses the problem that unreasonable threshold values ​​during the wiring image denoising process affect the extraction of wiring features, leading to insufficient accuracy in wiring detection, and improves the accuracy and stability of wiring scheme detection. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is a flowchart illustrating an intelligent detection method for field cabling solutions provided in one embodiment of the present invention.

[0043] Figure 2 This is a flowchart of a linear feature value acquisition process provided in one embodiment of the present invention;

[0044] Figure 3 This is a schematic diagram of the structure of an intelligent detection device for a field wiring scheme provided in one embodiment of the present invention. Detailed Implementation

[0045] To further illustrate the technical means and effects adopted by this application in order to achieve the intended purpose of the invention, the following, in conjunction with the accompanying drawings and preferred embodiments, details the specific implementation, structure, features and effects of an intelligent detection method, apparatus and device for field wiring scheme proposed in this application.

[0046] The application provides a field wiring scheme intelligent detection method, device and equipment.

[0047] Please refer to Figure 1 which shows a step flowchart of a field wiring scheme intelligent detection method provided by an embodiment of the application, and the method comprises the following steps.

[0048] In step S001, wiring videos are collected from different perspectives, and wiring images of different perspectives are extracted frame by frame.

[0049] Different-perspective cameras are arranged at positions to be detected for wiring in a construction site, and polarizing mirrors are added to the cameras to avoid reflection interference of glass, metal and other reflective objects on the cameras, so that the monitoring ranges of the different-perspective cameras all cover the same wiring detection area, each camera is used to collect wiring videos, wiring images are extracted frame by frame from the wiring videos, and wiring images of different perspectives at each collection moment are obtained.

[0050] The different perspectives are at least three, which are a left perspective, a front perspective and a right perspective.

[0051] Thus, wiring images of different perspectives at the same collection moment are obtained.

[0052] In step S002, any pixel point in the wiring image is recorded as a target pixel point, linear feature values and bending feature values of the target pixel point are extracted according to the gradient direction difference between the target pixel point and adjacent pixel points, scattering feature values of the target pixel point are extracted according to the gray distribution between the target pixel point and adjacent pixel points, the abnormal wiring feature values of the target pixel point are calculated in combination with the linear feature values and the bending feature values, and all significant pixel points in the wiring image are identified according to the difference between the abnormal wiring feature values of all pixel points in the same wiring image and the spatial distance between the pixel points.

[0053] The wavelet threshold denoising algorithm can accurately capture local features of an image, effectively retain edge and texture details in the image, and avoid the blur problem that is prone to occur after denoising. Meanwhile, the multi-resolution characteristic of wavelet transform can flexibly process signals of different scales, and can distinguish low-frequency contours and high-frequency details, so that more fine denoising is realized. However, for different wiring head images, the value of the universal threshold of the wavelet threshold denoising algorithm may not be reasonable, and the unreasonable threshold value can easily affect the extraction of wiring features, and further affect the accuracy of wiring detection.

[0054] When the wiring in the wiring image is normal, the position distribution of the adjacent pixel points of the corresponding line has strong linear characteristics; the abnormal line often has a curved line, and the position distribution of the adjacent pixel points corresponding to the curved position has strong curved characteristics; and the crossed and tangled chaotic wiring is relatively scattered, and the position distribution of the adjacent pixel points does not have strong linear or curved characteristics. Therefore, the linear characteristics, curved characteristics and scattered characteristics of each pixel point in each wiring image are extracted respectively.

[0055] Any one pixel point in the wiring image is recorded as a target pixel point, and the gradient directions of all pixel points in the wiring image are calculated. First, according to the gradient direction difference between the target pixel point and the adjacent pixel points, the linear characteristic value and the curved characteristic value of the target pixel point are extracted respectively.

[0056] A feature window of the target pixel point is established with the first preset length as the side length, all pixel points in the feature window of the target pixel point which are not the target pixel point are recorded as neighborhood pixel points, the absolute value of the difference between the gradient directions of the target pixel point and the neighborhood pixel points is recorded as the first relative gradient direction of the neighborhood pixel points, and the difference between the first relative gradient direction and the second relative gradient direction of the neighborhood pixel points is recorded as the second relative gradient direction of the neighborhood pixel points. The minimum value of the first relative gradient direction and the second relative gradient direction of the neighborhood pixel points is recorded as the relative gradient direction of the neighborhood pixel points; the relative gradient directions of all neighborhood pixel points determined for the target pixel point are weighted and summed using a Gaussian function to obtain the linear characteristic value of the target pixel point.

[0057] In this embodiment, the gradient direction of the pixel point is obtained using the Sobel operator, and as other embodiments, the implementer can obtain the gradient direction of the pixel point using other methods such as the Prewitt operator in the prior art on the basis of achieving the purpose of obtaining the gradient direction of the pixel point, and the present application does not make special limitations; the value of the first preset length in this embodiment is 5; when the weighted sum is performed using the Gaussian function, the Gaussian kernel parameter in this embodiment is set to 15°.

[0058] The linear characteristic value acquisition flow chart is shown in Figure 2 .

[0059] The value range of the gradient direction of the pixel point is divided into four value sub-ranges, the coefficient of variation of the gradient directions of all neighborhood pixel points in the same value sub-range in the feature window of the target pixel point is calculated, and the normalized value of the maximum value of the coefficient of variation is recorded as the curved characteristic value of the target pixel point.

[0060] The calculation of the coefficient of variation is a known technique and will not be described again. It should be noted that the normalization value is calculated according to the maximum value of the coefficient of variation determined by all pixel points in the wiring image. In the actual application process, the implementer can use other methods of prior art such as the maximum-minimum normalization method, the sigmoid function, etc. to calculate the normalization value, which is not limited herein.

[0061] When the bending feature value of the target pixel point is larger, the bending feature presented in the neighborhood range of the target pixel point is more significant.

[0062] The gray level co-occurrence matrix of the feature window of the target pixel point in 0°, 45°, 90° and 135° directions is calculated respectively, and the average of the inverse difference matrix of the gray level co-occurrence matrix in the four directions is recorded as the scattering feature value of the target pixel point.

[0063] The gray level co-occurrence matrix in different directions and the calculation of the inverse difference matrix of the gray level co-occurrence matrix are known techniques and will not be described again.

[0064] When the scattering feature value of the target pixel point is smaller, the bending scattering disorder feature presented in the neighborhood range of the target pixel point is more significant.

[0065] According to the weighted sum result of the linear feature value, the bending feature value and the scattering feature value of the target pixel point, the first wiring feature of the target pixel point is calculated.

[0066] The preset first weight coefficient, second weight coefficient and third weight coefficient are respectively taken as weights of the difference between the digital 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point and the difference between the digital 1 and the scattering feature value of the target pixel point, and the difference between the digital 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point and the difference between the digital 1 and the scattering feature value of the target pixel point are weighted and summed to obtain the abnormal wiring feature value of the target pixel point. The digital 1 is Arabic numeral 1, for example, the difference between the digital 1 and the linear feature value of the target pixel point: when the difference between the linear feature value of the target pixel point is 0.2, the difference between the digital 1 and the linear feature value of the target pixel point is 1-0.2=0.8; for example, the weighted sum result of the difference between the digital 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point and the difference between the digital 1 and the scattering feature value of the target pixel point: if the difference between the linear feature value of the target pixel point is 0.2, the bending feature value of the target pixel point is 0.3, and the scattering feature value of the target pixel point is 0.4, then the difference between the digital 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point and the difference between the digital 1 and the scattering feature value of the target pixel point are 0.8, 0.3 and 0.6 respectively, and the weighted sum result is the result obtained by weighted sum calculation of 0.8, 0.3 and 0.6.

[0067] Wherein, the sum of the first weight coefficient, the second weight coefficient and the third weight coefficient is 1; the first weight coefficient, the second weight coefficient and the third weight coefficient are respectively weights of the difference between the digital 1 and the linear feature value of the target pixel point, the bending feature value of the target pixel point and the difference between the digital 1 and the scattering feature value of the target pixel point; the values of the first weight coefficient, the second weight coefficient and the third weight coefficient in this embodiment are respectively 0.3, 0.3 and 0.4.

[0068] Specifically, the calculation formula of the abnormal wiring feature value is:

[0069]

[0070] Wherein, represents the abnormal wiring feature value of the target pixel point; , and respectively represent the first weight coefficient, the second weight coefficient and the third weight coefficient; represents the linear feature value of the target pixel point; represents the bending feature value of the target pixel point; represents the difference between the scattering feature value of the target pixel point.

[0071] The more significant the bending feature presented in the neighborhood range of the pixel point on the abnormal wiring, and the less obvious the linear feature and the scattered feature, the greater the abnormal wiring feature value, and the greater the possibility that the pixel point corresponding to the abnormal wiring feature value is located on the abnormal wiring.

[0072] According to the difference between the abnormal wiring feature values between all pixel points in the same wiring image and the Euclidean distance between the pixel points, all pixel points in the wiring image are clustered to obtain a first preset number of clustering clusters.

[0073] Preferably, in an embodiment of the present application, the calculation method of the metric distance of clustering is that the square of the difference between the abnormal wiring feature values of the pixel point and the clustering center is denoted as the first distance between the pixel point and the clustering center, the square of the Euclidean distance between the pixel point and the clustering center is denoted as the second distance between the pixel point and the clustering center, and the arithmetic square root of the sum of the first distance and the second distance between the pixel point and the clustering center is taken as the metric distance between the pixel point and the clustering center.

[0074] In this embodiment, K-means algorithm is used for clustering, and the clustering using K-means algorithm is a known technology and will not be described again; the value of the first preset number in this embodiment is 10.

[0075] The mean value of the abnormal wiring feature values of all pixel points in the same clustering cluster is denoted as the abnormal feature value of the same clustering cluster, the mean value of the abnormal feature values of all clustering clusters is denoted as the abnormal threshold value, and all pixel points in the clustering cluster with an abnormal feature value greater than the abnormal threshold value are denoted as significant pixel points.

[0076] It can be understood that the significant pixel points in the wiring image are the pixel points in the wiring image that may correspond to abnormal wiring.

[0077] At this point, all significant pixel points in the wiring image are identified.

[0078] Step S003, according to all significant pixel points in all wiring images of different viewing angles, and the linear feature value, the bending feature value and the scattered feature value of the pixel point at the same position as the significant pixel point, the complexity of each significant pixel point in each viewing angle is calculated respectively, the adjustment threshold value of the significant pixel point is calculated according to the complexity of the significant pixel point, and the adjustment threshold value of the pixel point which is not the significant pixel point is assigned, the wiring image is denoised according to the adjustment threshold value, and the denoised wiring image of all viewing angles is obtained.

[0079] According to all significant pixel points in all wiring images of different viewing angles, and the linear feature value, the bending feature value and the scattered feature value of the pixel point at the same position as the significant pixel point, the complexity of each significant pixel point is calculated respectively.

[0080] The vector composed of the linear feature value, the bending feature value and the scattering feature value of the pixel point in the wiring image is recorded as the feature vector of the pixel point. The two wiring images of different perspectives are recorded as the first wiring image and the second wiring image, any one of the significant pixel points in the first wiring image is recorded as the target significant pixel point, and the similarity between the feature vector of the target significant pixel point and the pixel point with the same coordinate in the second wiring image is recorded as the feature similarity of the target significant pixel point determined by the first wiring image and the second wiring image.

[0081] The feature similarities of all the significant pixel points determined by the first wiring image and the wiring image of each perspective can be obtained in the same way, and the feature similarities of all the significant pixel points determined by the first wiring image and the wiring image of all other perspectives can be obtained in the same way. The mean value of the feature similarities of the target significant pixel point determined by the first wiring image and the wiring image of all other perspectives is recorded as the average feature similarity of the target significant pixel point in the first wiring image.

[0082] The average feature similarity of each significant pixel point in the first wiring image can be obtained in the same way. The average feature similarities of all the significant pixel points in the first wiring image are arranged in sequence to obtain the average feature similarity sequence of the first wiring image.

[0083] In this embodiment, the average feature similarities of the significant pixel points are arranged in the order of the significant pixel points from left to right and from top to bottom. The average feature similarity sequence of the wiring image of each perspective can be obtained in the same way.

[0084] The DTW distance between the average feature similarity sequences of the first wiring image and the second wiring image is recorded as the first distance between the first wiring image and the second wiring image, the average feature similarities of the target significant pixel points in the average feature similarity sequences of the first wiring image and the second wiring image are removed to obtain the first removed wiring sequence and the second removed wiring sequence respectively, the DTW distance between the first removed wiring sequence and the second removed wiring sequence is recorded as the second distance between the first wiring image and the second wiring image, the absolute value of the difference between the first distance and the second distance is recorded as the third distance between the first wiring image and the second wiring image, and the normalized value of the mean value of the first distance and the third distance is recorded as the feature complexity of the target significant pixel point determined by the first wiring image and the second wiring image.

[0085] The feature complexity of the target significant pixel point determined by the first wiring image and the wiring image of each perspective can be obtained in the same way. The mean value of the feature complexity of the target significant pixel point determined by the first wiring image and the wiring image of all different perspectives is recorded as the complexity of the target significant pixel point.

[0086] In the embodiment, the similarity of the vectors is measured by using the Pearson correlation coefficient, and the calculation of the Pearson correlation coefficient and the DTW distance is a known technique and will not be described herein. In the embodiment, the normalized value is calculated by using a sigmoid function, and the sigmoid function is a known technique and will not be described herein. As other embodiments, the implementer can use other methods in the prior art, for example, a tanh function.

[0087] The complexity of each significant pixel point in the wiring image of each view can be obtained in the same way.

[0088] The product of the sum of the complexity of the significant pixel points and the universal threshold value of the wavelet threshold denoising algorithm is denoted as the adjusted threshold value of the significant pixel points, and the complexity of the pixel points that are not significant pixel points in the wiring image is assigned a value of 0. The adjusted threshold value of the pixel points is used as the value of the threshold value when the pixel points are denoised, and the wavelet threshold denoising algorithm is used to denoise the wiring images of all views to obtain the denoised wiring images.

[0089] In the embodiment, the calculation of the universal threshold value of the wavelet threshold denoising algorithm is a known technique, and the denoising by using the wavelet threshold denoising algorithm is a known technique and will not be described herein.

[0090] Thus, the denoised wiring images of all views are obtained.

[0091] In step S004, the denoised wiring images are subjected to image inpainting, glare suppression and motion compensation, the features of the denoised wiring images are extracted, and the detection of the field wiring scheme is completed.

[0092] The complex scene may exist to shield the camera, and the wiring images of different views can avoid the interference shielding problem as much as possible, and further, the denoised wiring images are subjected to image inpainting to obtain as complete and unshielded wiring images as possible, and the problem of complex scene shielding the camera is further solved.

[0093] Preferably, as an embodiment of the present application, the denoised wiring images are subjected to image inpainting by using a GAN generative adversarial network. Specifically, the denoised wiring images are input into the generative adversarial network, a U-Net structure generator is used, an optimizer is set as Adam, the number of training iterations is set as 200, a repair region mask generation threshold value is set as 0.5, and a to-be-repaired region is generated by pixel-by-pixel comparison. In the training process of the generative adversarial network, a context-aware loss function is used, the L1 norm distance between the known region and the generated region is calculated, the perception loss weight is set as 0.01, the features are extracted by using a VGG16 network, the spatial domain of the post-processing bilateral filter parameter is set as 10, the color domain is set as 15, the edge-preserving smoothing processing is performed on the repair boundary, and finally, the generative adversarial network can output a complete and unshielded image.

[0094] In the process of training the GAN generative adversarial network, the problem of sample imbalance may occur, for example, the abnormal phenomenon of a certain type of wiring, such as bending and label ambiguity, appears less frequently in the wiring video, resulting in insufficient accuracy when using a deep learning model to extract features. A certain number of high-quality abnormal wiring samples are generated using a data augmentation strategy to train the GAN generative adversarial network and improve the robustness of the model. Preferably, as an embodiment of the present application, the GAN generative adversarial network is used for abnormal sample enhancement, the value of the generator input noise dimension is set to 100, the number of training iterations is set to 500, the abnormal condition label dimension is set to 10, and the post-processing parameters are set. Specifically, the standard deviation of the Gaussian kernel of the Gaussian filter is set to 0.5, the generated samples are smoothed using the Gaussian filter, and high-quality abnormal wiring training samples are output; using the GAN generative adversarial network for image inpainting is a known technology and will not be described again.

[0095] There may be glass, metal and other reflective objects on the construction site, which will cause reflective interference to the camera when collecting wiring videos. Therefore, a polarizer is added to the camera, and a white balance optimization algorithm is used to suppress reflection and avoid reflective interference caused by glass, metal and other reflective objects on the construction site.

[0096] Preferably, as an embodiment of the present application, the white balance optimization algorithm is used to suppress reflection of the denoised wiring image, specifically, the gray world algorithm is used to initialize the white balance parameters, the RGB channel mean value is calculated, and the channel mean value deviation threshold of the gray world algorithm is set to 5%; the color temperature estimation model is used to optimize the color temperature in the denoised wiring image, and the color temperature estimation range is set to 4000K to 7000K; based on the 3x3 transformation matrix calibrated by the standard color card, the denoised wiring image is subjected to color space linear transformation; and the denoised wiring image with normal color temperature and effectively suppressed reflection is output.

[0097] Wherein, the white balance optimization algorithm, the gray world algorithm, the color temperature estimation model and the color space linear transformation are all known technologies and will not be described again.

[0098] There may be dynamic interference during construction, which can be solved by using the difference method and motion compensation to realize motion compensation and ensure that the image features of the wiring are extracted as accurately as possible from the wiring video.

[0099] Preferably, as an embodiment of the present application, the Lucas-Kanade optical flow method is used to calculate the global motion vector between adjacent frames of the wiring video, wherein the motion estimation block size of the Lucas-Kanade optical flow method is set to 16x16, the RANSAC algorithm is used to fit all global motion vectors, the maximum number of iterations of RANSAC is set to 100, the inlier distance threshold is set to 1.5, and 3 sets of point pairs are set as the minimum sample set. The affine transformation matrix is calculated from the minimum sample set, and the number of inliers in the model is calculated for iterative optimization. In the process of iterative optimization, the number of vectors with projection error less than 1.5 pixels under the assumption should be met. Finally, the transformation matrix with the most inliers is selected as the global motion estimation. The current frame is subjected to bilinear interpolation coordinate transformation through the affine transformation matrix to realize motion compensation.

[0100] Among them, the Lucas-Kanade optical flow method, the RANSAC algorithm, the affine transformation matrix and the bilinear interpolation coordinate transformation are all known technologies and will not be repeated here.

[0101] It can be understood that image restoration, reflection suppression and motion compensation are performed on the denoised wiring image to avoid the influence of various factors in the construction site on the wiring detection of the wiring video.

[0102] The cable and key points in the denoised wiring image are identified using a target detection model, and the features of the cable are extracted.

[0103] Preferably, as an embodiment of the present application, a labeling tool is used to label the normal wiring image, the abnormal wiring image and the denoised wiring image. The labeling tool selected in this embodiment is lable me. The normal wiring image and the abnormal wiring image are images prepared in advance by those skilled in the art. The shape is set to polygon, the optimizer is set to Adam or SGD, and the loss function is set to CIOU or DIOU. A target detection model is trained and a segmentation detection model is obtained using a target detection and segmentation algorithm. In this embodiment, the YOLOv8 model is used as the target detection model. The denoised wiring image labeled with the label is input into the segmentation detection model to extract the wiring in the denoised wiring image. The canny edge detection algorithm is used to extract the edge pixel points of the wiring in the denoised wiring image. The K-curvature estimation algorithm is used to calculate the curvature value of each edge pixel point. The non-maximum suppression algorithm is used to delete the adjacent key points. The RANSAC algorithm is used to fit each key point to obtain the curvature radius of the bending position of the extracted wiring. The piecewise linear fitting algorithm is used to linearly fit the key points. The line segment merging algorithm is used to merge the line segments that meet the conditions. The ratio of the total length of the straight line segment to the total length of the contour is calculated to obtain the length ratio of the straight cable to the overall cable.

[0104] The neighborhood search range of the K-curvature estimation algorithm is set to 5; the minimum key point distance of the non-maximum suppression algorithm is set to 10; the circular fitting radius range of the RANSAC algorithm is set to 5, the maximum iteration number is set to 100, and the distance threshold is set to 2; the straight line determination threshold of the piecewise linear fitting algorithm is set to 3, and the minimum line segment length is set to 30; the line segment merging distance of the line segment merging algorithm is set to 10; wherein the label marking, the target detection model, the canny edge detection algorithm, the K-curvature estimation algorithm, the RANSAC algorithm, the piecewise linear fitting algorithm and the line segment merging algorithm are all known technologies, and will not be described in detail.

[0105] The skeletonization algorithm is used on the denoised wiring image to obtain the skeleton of the single-pixel width wiring, and the node degree analysis algorithm is used to identify the intersection points in the single-pixel width skeleton.

[0106] The Zhang-Suen algorithm in the skeletonization algorithm is used to obtain the single-pixel width skeleton of the wiring, and the iteration round number is set to 100; the neighborhood connectivity determination rule of the node degree analysis algorithm is set to 8; wherein the skeletonization algorithm and the node degree analysis algorithm are both known technologies, and will not be described in detail.

[0107] The least square method (the direction fitting window size is set to 15) is used to calculate the direction vector of each branch of the intersection point, the vector angle formula (the angle calculation tolerance threshold is set to ± 5°) is used to calculate the adjacent branch angle of the intersection point, and the mean, standard deviation and extreme value of all adjacent branch angles of the intersection point are all used as the features of the denoised wiring image.

[0108] The side length of the direction fitting window size of the least square method is set to 15; when the vector angle formula is used, the angle calculation tolerance threshold is set to ± 5°.

[0109] Further, according to the denoised wiring image, the strong interference sources such as frequency converters and high-power power supplies and the extracted wiring, the nearest interference source distance of the wiring is calculated.

[0110] Preferably, as an embodiment of the present application, a target detection model is used to identify strong interference sources in the denoised wiring image, the minimum bounding rectangle of the strong interference sources is recorded as a bounding box, a geometric coordinate transformation algorithm is used to expand the bounding box, a Zhang-Suen skeletonization algorithm is used to extract a set of pixel points S in the skeleton of the wiring, and the Euclidean distance between each pixel point in the set of pixel points S and each pixel point of the expanded bounding box is calculated; a minimum distance threshold is set to 50 pixel points in length, the wiring corresponding to the pixel point set with an Euclidean distance less than the minimum distance threshold is recorded as a high-risk wiring disturbed, and the minimum value of the Euclidean distance corresponding to the high-risk wiring disturbed is recorded as the nearest interference source distance of the wiring. The nearest interference source distance of all wirings determined in the same wiring image is used as a feature of the denoised wiring image.

[0111] wherein the boundary box expansion pixel of the geometric coordinate transformation algorithm is set to 10 pixels, and the iteration round number of the Zhang-Suen skeletonization algorithm is set to 100; the target detection model, the geometric coordinate transformation algorithm and the Zhang-Suen skeletonization algorithm are all known technologies and will not be described in detail.

[0112] The intelligent diagnosis of the wiring is combined with the engineering specification standard to determine the conditions of the wiring identified by the denoised wiring image in terms of cable bending, wiring path, cross winding and electromagnetic safety, and to obtain a warning result.

[0113] Specifically, the cable bending of the wiring is determined by the curvature radius of the bending position of the wiring, when there is a curvature radius less than or equal to 6 times the wire diameter in the curvature radius of the bending position of all wirings identified by the denoised wiring image, it is determined that the cable bending detection is unqualified, and the corresponding diagnostic result and fault code are generated; the wiring path of the wiring is determined by the length ratio of the straight cable to the overall cable, when the length ratio of the straight cable to the overall cable is less than or equal to 80% of the engineering specification standard, it is determined that the wiring path detection is unqualified, and the corresponding diagnostic result and fault code are generated; the cross winding of the wiring is determined by the number of cross points, when there are more than 3 cross points in one meter of wiring, it is determined that the cross winding detection is unqualified, and the corresponding diagnostic result and fault code are generated; the electromagnetic safety of the wiring is determined by the nearest interference source distance, when the nearest interference source distance is less than 30 centimeters, it is determined that the electromagnetic safety detection is unqualified, and the corresponding diagnostic result and fault code are generated.

[0114] When the cable bending and the wiring path detection are detected to be unqualified, a first-level warning of slight defects is issued; when the cross winding detection is detected to be unqualified, a second-level warning of process defects is issued; and when the electromagnetic safety detection is detected to be unqualified, a third-level warning of safety risks is issued.

[0115] The AR marking system and the BIM technology are used for processing the features and early warning results of the denoised wiring image, and a diagnosis heat map is acquired.

[0116] The AR marking system and the BIM technology are all known technologies, and will not be described in detail.

[0117] Thus, the detection of the field wiring scheme is realized.

[0118] Please refer to Figure 3 , Figure 3 is a structural schematic diagram of an intelligent detection device for a field wiring scheme provided by an embodiment of the present application. In the embodiment, each unit of the device is used for executing each step in the corresponding embodiment of the intelligent detection method for a field wiring scheme. Please refer to Figure 3 , the intelligent detection device comprises a wiring image extraction module, a salient pixel point identification module, a denoised image acquisition module and a wiring scheme detection module.

[0119] The wiring image extraction module is used for collecting wiring videos from different perspectives and extracting wiring images of different perspectives frame by frame.

[0120] The salient pixel point identification module is used for recording any one pixel point in the wiring image as a target pixel point, extracting linear feature values and bending feature values of the target pixel point according to the gradient direction difference between the target pixel point and adjacent pixel points, extracting scattering feature values of the target pixel point according to the gray distribution between the target pixel point and adjacent pixel points, combining the linear feature values and the bending feature values to calculate abnormal wiring feature values of the target pixel point, and identifying all salient pixel points in the wiring image according to the difference between the abnormal wiring feature values of all pixel points in the same wiring image and the spatial distance between the pixel points.

[0121] The denoised image acquisition module is used for calculating the complexity of each salient pixel point of each perspective according to all salient pixel points in all wiring images of different perspectives and the linear feature values, bending feature values and scattering feature values of the pixel points with the same position as the salient pixel points, calculating the adjustment threshold of the salient pixel points according to the complexity of the salient pixel points, assigning values to the adjustment thresholds of pixel points that are not salient pixel points, denoising the wiring image according to the adjustment threshold, and acquiring denoised wiring images of all perspectives.

[0122] The wiring scheme detection module is used for image repair, light suppression and motion compensation on the denoised wiring image, extracting features of the denoised wiring image and completing the detection of the field wiring scheme.

[0123] Based on the same inventive concept as the above method, the embodiments of the present application also provide an intelligent detection device for a field wiring scheme, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of any one of the above intelligent detection methods for a field wiring scheme.

[0124] Based on the same inventive concept as the above method, the embodiments of the present application also provide an intelligent detection device for a field wiring scheme, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, and the processor implements the steps of any one of the above intelligent detection methods for a field wiring scheme.

[0125] The above description is merely preferred embodiments of the present application, and is not intended to limit the present application. Any modification, equivalent replacement, improvement, etc. made within the principles of the present application shall be included in the protection scope of the present application.

Claims

1. An intelligent detection method for field cabling schemes, characterized in that, The method includes the following steps: Wiring videos were captured from different perspectives, and wiring images from different perspectives were extracted frame by frame; Any pixel in the wiring image is designated as the target pixel. Based on the difference in gradient direction between the target pixel and its neighboring pixels, linear and bending feature values ​​are extracted from the target pixel. Based on the grayscale distribution between the target pixel and its neighboring pixels, scattered feature values ​​are extracted from the target pixel. Combining the linear and bending feature values, abnormal wiring feature values ​​of the target pixel are calculated. Based on the differences in abnormal wiring feature values ​​among all pixels in the same wiring image and the spatial distance between pixels, all significant pixels in the wiring image are identified. Based on all salient pixels in the wiring images from all different viewpoints, as well as the linear, curvature, and scattered feature values ​​of pixels at the same position as the salient pixels, calculate the complexity of each salient pixel in each viewpoint. Calculate the adjustment threshold of the salient pixels based on the complexity of the salient pixels, and assign adjustment thresholds to other non-salient pixels. Denoise the wiring images based on the adjustment thresholds to obtain denoised wiring images from all viewpoints. The denoised wiring images are inpainted, reflective, and motion compensated. Features of the denoised wiring images are extracted and the on-site wiring scheme is detected. The specific calculation of the complexity of each significant pixel in each viewpoint is as follows: The vector formed by arranging the linear, curved, and scattered feature values ​​of pixels in the wiring image in sequence is denoted as the feature vector of the pixel. Two wiring images from different perspectives are denoted as the first wiring image and the second wiring image, respectively. Any salient pixel in the first wiring image is denoted as the target salient pixel. The similarity between the feature vectors of the pixels in the second wiring image with the same coordinates as the target salient pixel and the target salient pixel is denoted as the feature similarity of the target salient pixel determined by the first wiring image and the second wiring image. The average feature similarity of the target salient pixels determined by the first wiring image and wiring images from all other perspectives is denoted as the average feature similarity of the target salient pixels in the first wiring image. Based on the average feature similarity of all salient pixels in the first wiring image, an average feature similarity sequence of the first wiring image is established. The DTW distance between the average feature similarity sequences of the first wiring image and the second wiring image is denoted as the first distance between the first wiring image and the second wiring image. The average feature similarity of the corresponding target significant pixels in the average feature similarity sequence of the first wiring image and the second wiring image is removed to obtain the first removed wiring sequence and the second removed wiring sequence respectively. The DTW distance between the first removed wiring sequence and the second removed wiring sequence is recorded as the second distance between the first wiring image and the second wiring image. The absolute value of the difference between the first distance and the second distance between the first wiring image and the second wiring image is recorded as the third distance between the first wiring image and the second wiring image. The normalized value of the mean of the first distance and the third distance is denoted as the feature complexity of the target salient pixel determined by the first wiring image and the second wiring image; the mean of the feature complexity of the target salient pixel determined by the first wiring image and wiring images from all different viewpoints is denoted as the complexity of the target salient pixel.

2. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The specific method for obtaining the linear and curved feature values ​​of the target pixel is as follows: Establish a feature window for the target pixel, and mark neighboring pixels based on the pixels within the feature window of the target pixel; The absolute value of the difference between the gradient directions of the target pixel and its neighboring pixels is denoted as the first relative gradient direction of the neighboring pixels. The difference between the first relative gradient direction of the neighboring pixel and the second relative gradient direction of the neighboring pixel is denoted as the second relative gradient direction of the neighboring pixel. The minimum value between the first relative gradient direction and the second relative gradient direction of the neighboring pixel is denoted as the relative gradient direction of the neighboring pixel. The linear feature value of the target pixel is obtained by weighted summing of the relative gradient directions of all neighboring pixels determined by the target pixel. Divide the gradient direction range of a pixel into four sub-ranges. Calculate the coefficient of variation of the gradient direction of all neighboring pixels within the same sub-range in the feature window of the target pixel. The normalized value of the maximum value of the coefficient of variation is recorded as the bending feature value of the target pixel.

3. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The specific method for obtaining the scattered feature values ​​of the target pixel is as follows: Calculate the gray-level co-occurrence matrix of the feature window of the target pixel in four different directions; denote the mean of the inverse difference moments of the gray-level co-occurrence matrices in the four directions as the scattered feature value of the target pixel.

4. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The abnormal wiring characteristic value of the target pixel is: the weighted sum of the difference between the linear characteristic value of the digit 1 and the target pixel, the bending characteristic value of the target pixel, and the difference between the random characteristic value of the digit 1 and the target pixel.

5. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The method for identifying all significant pixels in the wiring image is as follows: Based on the differences between abnormal wiring feature values ​​and the Euclidean distance between pixels in the same wiring image, the clustering metric distance is calculated, and all pixels in the wiring image are clustered to obtain clusters. The mean of the abnormal wiring feature values ​​of all pixels in the same cluster is recorded as the abnormal feature value of the same cluster. The mean of the abnormal feature values ​​of all clusters is recorded as the abnormal threshold. All pixels in the clusters whose abnormal feature values ​​are greater than the abnormal threshold are recorded as significant pixels.

6. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The specific steps involved in calculating the adjustment threshold for significant pixels based on their complexity and assigning adjustment thresholds to other non-significant pixels are as follows: The product of the complexity of the significant pixel and the sum of the digits 1 and the general threshold of the wavelet thresholding algorithm is denoted as the adjustment threshold of the significant pixel. Assign a complexity value of 0 to pixels that are not significant pixels in the wiring diagram.

7. The intelligent detection method for field cabling schemes according to claim 1, characterized in that, The specific method for denoising the wiring image based on adjusting the threshold is as follows: The adjustment threshold of each pixel is used as the threshold value when denoising pixels, and the wavelet threshold denoising algorithm is used to denoise the wiring images from all viewpoints.

8. An intelligent detection device for field wiring schemes, implementing the method as described in claim 1, characterized in that, The intelligent detection device includes: The wiring image extraction module is used to acquire wiring videos from different perspectives and extract wiring images from different perspectives frame by frame. The salient pixel identification module is used to designate any pixel in the wiring image as the target pixel. Based on the difference in gradient direction between the target pixel and its neighboring pixels, it extracts the linear and curvature feature values ​​of the target pixel. Based on the grayscale distribution of the target pixel and its neighboring pixels, it extracts the scattered feature values ​​of the target pixel. Combining the linear and curvature feature values, it calculates the abnormal wiring feature value of the target pixel. Based on the difference between the abnormal wiring feature values ​​of all pixels in the same wiring image and the spatial distance between pixels, it identifies all salient pixels in the wiring image. The denoising image acquisition module is used to calculate the complexity of each significant pixel in each viewpoint based on all significant pixels in the wiring images from all different perspectives, as well as the linear, curvature, and scattered feature values ​​of pixels at the same position as the significant pixels. It calculates an adjustment threshold for each significant pixel based on its complexity, assigns adjustment thresholds to other non-significant pixels, and denoises the wiring images according to the adjustment thresholds to obtain denoised wiring images from all perspectives. The specific calculation of the complexity of each significant pixel in each viewpoint is as follows: The vector formed by sequentially arranging the linear, curvature, and scattered feature values ​​of pixels in the wiring image is denoted as the feature vector of the pixel; the wiring images from two different perspectives are denoted as the first wiring image and the second wiring image, respectively; any significant pixel in the first wiring image is denoted as the target significant pixel; the similarity between the feature vectors of pixels in the second wiring image with the same coordinates as the target significant pixel and the target significant pixel is denoted as the feature similarity of the target significant pixel determined by the first and second wiring images; the feature similarity of the target significant pixel determined by the first wiring image and the wiring images from all other perspectives is calculated. The mean of the feature similarity is denoted as the average feature similarity of the target salient pixels in the first wiring image. Based on the average feature similarity of all salient pixels in the first wiring image, an average feature similarity sequence of the first wiring image is established. The DTW distance between the average feature similarity sequences of the first and second wiring images is denoted as the first distance between the first and second wiring images. The average feature similarity of the corresponding target salient pixels in the average feature similarity sequences of the first and second wiring images is removed, and a first removed wiring sequence and a second removed wiring sequence are obtained respectively. The DTW distance between the first removed wiring sequence and the second removed wiring sequence is denoted as the second distance between the first and second wiring images. The absolute value of the difference between the first distance and the second distance is denoted as the third distance between the first and second wiring images. The normalized value of the mean of the first distance and the third distance is denoted as the feature complexity of the target salient pixels determined by the first and second wiring images. The mean of the feature complexity of the target salient pixels determined by the first wiring image and wiring images from all different viewpoints is denoted as the complexity of the target salient pixels. The cabling scheme detection module is used to perform image restoration, reflection suppression, and motion compensation on the denoised cabling image, extract the features of the denoised cabling image, and complete the detection of the cabling scheme on site.

9. An intelligent detection device for field cabling schemes, comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the steps of the method as claimed in any one of claims 1-7.

Images