Coin reference based crack real-time detection method and device

Abstract

The application discloses a kind of crack real-time detection method and device based on coin reference, the method is collected by smart phone crack image containing standard size coin, through coin detection, perspective transformation, crack analysis realizes accurate crack quantitative measurement.Main technical scheme includes:1) coin detection module, the position of reference coin, profile is accurately identified by improved ellipse detection algorithm combined with hand region shielding technology;2) perspective transformation module, based on coin position, image geometry correction is realized using perspective transformation;3) crack analysis module, "you only look once" algorithm YOLO (You Only Look Once) and skeletonization algorithm are used to realize sub-millimeter crack measurement.The application first proposes nondestructive testing scheme of smart phone combined with coin reference object, while ensuring measurement accuracy, significantly reduces hardware cost, and the detection accuracy can reach ±0.2mm, suitable for civil engineering detection and identification and other fields.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of computer vision and non-destructive testing, and in particular to a crack real-time detection method and device based on a coin reference. BACKGROUND

[0002] Wall surface cracks are a major feature of aging of residential, office buildings and other building structures. Cracks are one of the important inspection items for the safety of concrete structures and components. How to achieve accurate, fast and lightweight identification and quantitative detection is a big problem in the field of structural health monitoring.

[0003] Traditional crack detection techniques mainly include traditional contact detection techniques based on artificial crack rulers and crack width measuring instruments, and non-contact detection techniques based on computer vision. The former relies on manual visual observation, has low efficiency and strong subjectivity, and needs to be equipped with professional equipment. Ordinary people have high learning threshold and cannot meet the safety concerns and detection needs of ordinary people in their surrounding living / office environment. For the latter, the current academia and industry mainly focus on improving the accuracy of crack segmentation / detection algorithms and developing crack detection equipment for large civil engineering structures. Their effectiveness and convenience cannot meet the detection needs of ordinary people anytime and anywhere.

[0004] Many scholars have studied algorithms for measuring the width of building structure surface cracks using images. Some scholars recommend using a two-dimensional code as a calibration object for image rectification and size relationship conversion, and have achieved algorithm integration, indoor experimental verification, website platform construction and field measurement. Although using a two-dimensional code as a calibration object has certain effect in the case of regular inspection of specific cracks, however, the step of printing and pasting a two-dimensional code is too tedious for ordinary people and cannot meet the core needs of users: simplicity and convenience. Further, researchers have studied a road crack detection method without calibration objects based on the angle of the imaging plane and the camera, but the crack positioning is not accurate and the precision can only reach centimeters, which is difficult to popularize and apply.

[0005] As we all know, smartphones have become an indispensable part of people's lives, and coins are also common items that people carry with them. Using coins as calibration objects for crack detection has natural advantages. On the other hand, studies have shown that the number and denomination of coins can be accurately detected using convolutional neural networks. In view of this, it is feasible and meaningful to develop a crack real-time detection method based on a coin reference and establish a corresponding detection system to achieve accurate, fast and lightweight crack identification and quantitative detection. SUMMARY

[0006] This invention addresses two core problems in existing technologies: strong dependence on calibration, as current mobile detection schemes require manual placement of specialized calibration objects (such as QR codes), which is difficult to implement on complex building surfaces, and the calibration process accounts for more than 3 / 4 of the total detection time; and the specialized nature of the calibration objects, as the current calibration objects are not common objects used by ordinary people, lacking simplicity and convenience, making current vision-based crack detection schemes unsuitable for a wider range of users. This invention provides a real-time crack detection method and system based on a coin reference, applicable to the quantitative measurement of cracks in civil engineering, building maintenance, and other scenarios, especially addressing the need for rapid on-site detection when specialized measurement equipment is lacking.

[0007] The objective of this invention is achieved through the following technical solution: a real-time crack detection method based on a coin reference, comprising:

[0008] Capture an image of the test object, including a standard-sized coin and a cracked area, using a mobile imaging device (such as a smartphone);

[0009] The hand in the image to be tested is semantically segmented to mask the interference area of ​​the hand, and a grayscale image after masking the hand is obtained.

[0010] The grayscale image is sequentially processed using a feature enhancement module, an edge detection and morphological closure module, and a fitted coin detection module to obtain a set of valid fitted coins.

[0011] The coin with the largest radius is identified from the set of valid fitted coins and selected as the reference benchmark. The conversion ratio between pixels and actual size is determined. The image to be tested is corrected to the front view plane through the perspective transformation algorithm, and a front view image of the surface to be tested containing cracks is generated.

[0012] Cracks are segmented from the front view image of the surface to be tested, and the physical size parameters of the cracks are calculated.

[0013] Furthermore, the diameter of the coin Occupying the width of the image to be tested 5% to 10%.

[0014] Furthermore, semantic segmentation is performed on the hand in the image to be tested to mask the hand-related interference region, including:

[0015] The Hands hand detector from the MediaPipe library is used to generate a hand region mask for the image Image_test under test. .

[0016] Furthermore, the feature enhancement module applies contrast-limited adaptive histogram equalization and Gaussian blur algorithm to the grayscale image Image_non_hand to enhance the coin features, obtaining the coin feature-enhanced image Image_feat;

[0017] Edge detection and morphological closure module: Based on the occluded edge detection algorithm, locate the coin in the Image_feat image and obtain the coin boundary; perform morphological closure processing on the incomplete coin boundary to obtain the processed image Image_proc;

[0018] The coin fitting detection module uses a three-step process of ellipse fitting, Hough circle, and geometric verification to detect Image_proc and obtain a set of valid fitted coins.

[0019] Furthermore, the edge detection and morphological closure module includes:

[0020] Edge detection module: This module processes the grayscale matrix of the image Image_non_hand. Calculate the double-threshold Canny operator , Next, the Canny function from the OpenCV library is called to obtain the binarized edge image Edges.

[0021] Morphological Closure Module: Performs closure processing on the binarized edge image Edges. Using an elliptical structuring element S, it performs a "dilation-erosion" morphological closure operation on the edge image to obtain the processed image Image_proc.

[0022] Furthermore, the fitted coin detection module includes:

[0023] Ellipse fitting priority mode: Performs connected contour search on the processed image Image_proc, filtering contours with areas greater than a threshold. For incomplete elliptical regions, ellipse fitting is performed on incomplete ellipses with at least 5 contour feature points to obtain candidate regions. ,in , , , They are respectively The coordinates of the center and the radius of the circle. ;

[0024] right Calculate the ratio of major to minor axis , ,in , The first The lengths of the major and minor axes of each candidate contour ellipse; Let the roundness threshold be... ,when When a candidate contour ellipse is determined to be a valid fitted coin, a set of valid fitted coins is obtained, and the center coordinates of the ellipse are output. and equivalent radius ;

[0025] Hough circle backup mode: when the ellipse fitting priority mode fails to obtain an effective fitting coin, switch to the Hough circle fitting mode, call the HoughCircles function of the OpenCV library to locate the coin in the image Image_proc, and obtain the candidate region ;

[0026] For the candidate region , the candidate target with a vertical distance from the center of the circle to the image boundary less than its equivalent radius is screened out, and a new candidate region is obtained ;

[0027] For the candidate region , calculate its average radius , remove outliers with a radius deviation greater than , and obtain an effective fitting coin set , is a tolerance coefficient.

[0028] Further, a front view image of the surface to be tested containing cracks is generated, including:

[0029] Establish a mapping relationship between the imaging plane and the target plane to obtain a perspective transformation matrix , including:

[0030] Imaging plane reference point selection: select the center of the coin with the largest radius from the effective fitting coin set , and extract four extreme points on its circumference as the imaging plane coordinate point set ;

[0031] Target plane reference point selection: let , where is the conversion ratio of pixels to actual size, and the target plane coordinate system is established with four points , , and , and the four points form the target plane coordinate point set ;

[0032] Transformation matrix calculation: based on the and , the homography matrix is calculated by the least squares method, satisfying: , calculate the boundary coordinates of the four corner points of the test image Image_test after the transformation, so as to determine the size of the target image; construct a translation correction matrix , so that the upper left corner of the target image is aligned with the origin of the coordinate system, therefore, the perspective transformation matrix ;​

[0033] The OpenCV library's perspective transformation function warpPerspective is used to obtain the coordinates of each pixel in the image Image_test on the target plane.

[0034] Pixel resampling processing is used to obtain the pixel values ​​of the target plane, including:

[0035] The OpenCV library's bicubic interpolation function INTER_CUBIC is called to calculate the corresponding pixel value on the target plane by weighted combination of the 16 neighboring pixel values ​​of the sampling point on the imaging plane, thus generating a pixel matrix. For each neighboring pixel, the distance from the sampling point is considered. The weights are determined by a piecewise continuous cubic polynomial kernel function. This allows for the calculation of the corresponding pixel values ​​of the sampling points on the imaging plane within the target plane.

[0036] ;

[0037] in, Used to achieve optimal sharpness-smoothness balance;

[0038] The pixel values ​​of each pixel in the image under test Image_test on the target plane are obtained accordingly, thereby generating the front view image of the surface under test Image_pt containing the crack.

[0039] Furthermore, the front view image of the surface under test is segmented into cracks, and the physical size parameters of the cracks are calculated, including:

[0040] The YOLO model is used to segment the cracks in the front view image Image_pt. The physical size parameters of the cracks, including the maximum crack width, are calculated by combining skeletonization and distance transformation algorithms. Average width and length ,in Represents the first in the image A crack.

[0041] The present invention also provides a real-time crack detection device based on a coin reference, comprising one or more processors for implementing the above-described real-time crack detection method based on a coin reference.

[0042] The present invention also provides a computer-readable storage medium having a program stored thereon, which, when executed by a processor, is used to implement the above-described real-time crack detection method based on a coin reference.

[0043] The beneficial effects of the present application are that, by the innovative coin calibration method and image analysis technology, the sub-millimeter crack size measurement (±0.20mm) of a smart phone is realized for the first time, and the precision is higher than that of traditional manual detection. The coin detection algorithm for interference is adopted, and the coin recognition accuracy is still maintained above 90%. Combined with the lightweight YOLO model and parallel computing architecture, the processing time of 2.3s / frame is realized on a 4K resolution image. The natural calibration mode based on the coin greatly shortens the on-site operation time, and the hardware cost is reduced to 1 / 100 of that of professional equipment. The system has the following unique advantages: multi-scale fusion detection makes the detection rate of small cracks (<0.3mm) reach 91%. The present application provides a first intelligent mobile phone nondestructive testing solution with high precision, low cost and simple and convenient calibration object for the building detection field. BRIEF DESCRIPTION OF DRAWINGS

[0044] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings needed in the embodiment description will be briefly introduced. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creative labor.

[0045] Figure 1 The flowchart of the crack real-time detection method based on the coin reference provided by the present application.

[0046] Figure 2 The flowchart of removing hand interference and identifying coins in the present application.

[0047] Figure 3 The flowchart of correcting through the coin perspective in the present application.

[0048] Figure 4 The flowchart of crack parameter identification in the present application.

[0049] Figure 5 The crack identification result in the present application.

[0050] Figure 6 The technical roadmap of the crack detection system in the present application.

[0051] Figure 7 The schematic diagram of the crack identification smart phone platform in the present application.

[0052] Figure 8 The hardware structure schematic diagram provided by the embodiment of the present application. DETAILED DESCRIPTION

[0053] The present application will be described in detail below with reference to the drawings. In the case of no conflict, the features in the following embodiments and implementation manners can be combined with each other.

[0054] The core technology of the present application is to establish a complete coin calibration and crack measurement system, and to realize accurate physical size measurement through multi-level image processing algorithm. The system adopts multi-modal detection technology based on computer vision, combines deep learning with traditional image processing method, and realizes sub-millimeter level precision crack size measurement by using smart phone. The core technical features include: (1) adaptive coin calibration algorithm, which can accurately detect and identify standard size coins in complex background; (2) intelligent perspective transformation system, which eliminates the size error caused by shooting angle through perspective transformation; (3) crack analysis engine combining YOLO and skeleton, which realizes accurate quantification of crack geometric parameters.

[0055] The coin reference-based real-time crack detection method proposed by the present application comprises the following steps:

[0056] (1) Image acquisition stage: The image acquisition stage: through the smart phone, the image to be measured Image_test containing the standard size coin (radius ) and the crack area is photographed, the cv2.imread function in the OpenCV library is called to read the image to be measured, and the cv2.cvtColor function is called to obtain the gray matrix , the matrix size is , it is required that the hand-held coin and the crack are in the same plane (the thickness of the coin can be ignored) and are photographed in the same image, and the diameter of the coin needs to account for 5%~10% of the image width . In order to guarantee the imaging quality, adaptive shooting strategy is implemented: when the detected ambient illuminance is <100 lux, the flash compensation is automatically enabled, and the shooting distance is controlled within 30±5 cm range through the laser ranging module. For different mobile phone models, the built-in CMOS sensor parameter database is used to automatically correct the lens distortion coefficient .

[0057] (2) Hand shielding stage: based on the feature detection module of the MediaPipe hand recognition library, the hand in the image to be measured Image_test is semantically segmented to shield the hand interference area, and the gray image Image_non_hand after shielding the hand is obtained, and its matrix representation is , which specifically includes the following sub-steps:

[0058] The hand region mask of the image to be measured Image_test is generated by the Hands hand detector in the MediaPipe library, generally, the minimum confidence threshold for hand detection is set.

[0059] (3) Coin detection stage: the gray image Image_non_hand in step (2) is subjected to a feature enhancement module, an edge detection and morphological closing module, and a fitted coin detection module to obtain an effective fitted coin set , specifically comprising the following sub-steps:

[0060] (3.1) Feature enhancement module: applying a contrast limited adaptive histogram equalization (CLAHE) and a Gaussian blur algorithm to Image_non_hand to enhance the features of the coins, to obtain a coin feature enhanced image Image_feat, specifically comprising the following sub-steps:

[0061] (3.1.1) The parameters of the CLAHE algorithm include but are not limited to: contrast limit (inhibit noise amplification, generally set to 0.01-0.05, the larger the value, the stronger the contrast enhancement), block size (generally set to , , the smaller the block, the more obvious the local contrast);

[0062] (3.1.2) The Gaussian Blur algorithm parameters include but are not limited to: kernel parameters (generally set to no less than ), standard deviation parameters (generally set to 1-3, the larger the value, the smoother the image, but the details are obviously lost);

[0063] (3.2) Edge detection and morphological closing module: based on the occlusion edge detection (Edge detection) algorithm to locate the coins in the Image_feat image and obtain the non-complete coin boundary; the non-complete coin boundary is subjected to morphological closing processing to obtain a processed image Image_proc, specifically comprising the following sub-steps:

[0064] (3.2.1) Edge detection module: calculating a double-threshold Canny operator for the gray matrix of the image Image_non_hand , ,

[0065]

[0066] wherein is the average gray value of the non-hand region of the image, and 0.67 and 1.33 are fixed values for Canny edge detection;

[0067] The Canny function of the OpenCV library is called to obtain a binary edge image: ;

[0068] (3.2.2) Morphological closing module: the binary edge image Edges is closed,

[0069] An elliptical structural element S (kernel size of 3x3 pixels, to adapt to the geometric characteristics of circular or quasi-circular target objects) is used;

[0070] The morphological closing operation (Morphological Closing) of "first expansion and then corrosion" is performed on the edge image to obtain the processed image Image_proc:

[0071]

[0072] wherein, and represent expansion and corrosion operations, respectively.

[0073] (3.3) Coin detection module: through the three-step detection of ellipse fitting-Hough circle-geometric verification, the effective fitted coin set is obtained, which specifically includes the following sub-steps:

[0074] (3.3.1) Ellipse fitting priority mode:

[0075] The connected contour search is performed on the processed image Image_proc, and the non-complete ellipse region with an area greater than the threshold (default value of 100, generally set to 80-150 pixels) is screened out, and the non-complete ellipse with a contour feature point number not less than 5 (5 independent points determine an ellipse) is subjected to ellipse fitting to obtain a candidate region , wherein , , , are the center coordinates and radius of , respectively;

[0076] The length-to-short-axis ratio of is calculated , , wherein , are the lengths of the major axis and the minor axis of the th candidate contour ellipse, respectively; the roundness threshold is set to (generally set to 0.6-0.8), and when , the candidate contour ellipse is determined as an effective fitted coin, and the effective fitted coin set one is Output the coordinates of the center of the ellipse. and equivalent radius ;

[0077] (3.3.2) Hoff circle standby mode:

[0078] When the ellipse fitting mode cannot obtain a valid fitted coin (i.e.) Switching to Hough Circle fitting mode, the OpenCV library's HoughCircles function is called to locate the coin in the image and obtain candidate regions. Generally, the ratio of image pixel resolution to parameter space resolution is... (This indicates that the parameter space has the same resolution as the image space; it is generally set to 1. If the coin's edges are blurry, it can be adjusted.) To reduce parameter spatial resolution, minimum center distance (Set according to the coin diameter, generally larger than 24mm), Canny edge detection high threshold (Generally set to 100~200), center threshold (Generally set to 20~50, the smaller the value, the more circles are detected);

[0079] (3.3.3) Geometric constraint verification:

[0080] Boundary distance verification: for candidate regions Candidate targets whose vertical distance from the center of the circle to the image boundary is less than its equivalent radius are eliminated (i.e., those that satisfy the condition...). and ), obtain new candidate regions ;

[0081] Size consistency verification: for candidate regions Calculate its average radius Remove radius deviation ( )Exceed outliers (where (This is a tolerance coefficient, typically set to 0.2~0.4), to obtain a set of coins that fits effectively. .

[0082] (4) Perspective transformation and size calibration stage: Input the image to be tested Image_test, and from the set of valid fitted coins ( or Identify and select the coin with the largest radius from the list. As a reference benchmark, determine the conversion ratio between pixels and actual size. The Image_test is corrected to a frontal view plane using a perspective transformation algorithm, generating an image of the tested surface Image_pt (a coin) containing cracks. After perspective transformation This process eliminates geometric deformation caused by the shooting angle, ensuring the accuracy of crack size measurement; specifically, it includes the following sub-steps:

[0083] (4.1) Determine the imaging plane by effectively fitting the coin with the largest radius in the coin set, establish the mapping relationship between the imaging plane and the target plane (frontal view), and obtain the perspective transformation matrix. Thus, the coordinates of each pixel in the image under test, Image_test, on the target plane are:

[0084] Imaging plane reference point selection: from the set of effectively fitted coins Select the center of the coin with the largest radius. Centered on the image plane, four extreme points are extracted along its circumference at angles of 0°, 90°, 180°, and 270° to form the set of coordinate points on the imaging plane. ,in: Right extreme point , The upper extreme point , Left extreme point , The lower extreme point ;

[0085] Target plane reference point selection: Let ,in The conversion ratio between pixels and actual size, in , , as well as Establish a target plane coordinate system using four points, forming a set of target plane coordinate points. ;

[0086] Transformation matrix calculation: based on the above and The homography matrix is ​​calculated using the least squares method. ,satisfy: ,in The homography matrix is ​​calculated using the findHomography function from the OpenCV library; the four corner points of the image Image_test are calculated. , , , go through Transformed boundary coordinates , , , The size of the target image is determined based on the aforementioned corner points. ; construct translation correction matrix , so that the top left corner of the target image aligns with the origin of the coordinate system: , where and respectively refer to the minimum and coordinates in the target image; therefore, the perspective transformation matrix ;

[0087] Call the perspective transformation function warpPerspective of the OpenCV library to obtain the coordinates of each pixel point of the image to be tested Image_test on the target plane;

[0088] (4.2) Pixel resampling processing obtains the pixel value of the target plane :

[0089] Call the bicubic interpolation function INTER_CUBIC of the OpenCV library to calculate the corresponding pixel value of the target plane through the weighted combination of the adjacent 16 pixel values of the sampling point to generate the pixel matrix ; wherein, for each adjacent pixel, according to its distance ( direction distance , direction distance ) from the sampling point, determine its weight (horizontal weight or vertical weight ) through the piecewise continuous cubic polynomial kernel function:

[0090]

[0091] wherein, when the parameter , generally the optimal sharpness-smoothness balance can be achieved (generally set to -0.80 to -0.70, increasing the edge sharpness and reducing the smoothness);

[0092] The corresponding pixel value of the target plane is the bicubic weighted sum of the 16 adjacent pixels:

[0093]

[0094] In this way, the pixel value of each pixel point of the image to be tested Image_test on the target plane is obtained, thereby generating the front view image Image_pt of the surface to be tested containing the crack.

[0095] (5) Crack analysis stage: the YOLO (You Only Look Once, "you only look once" algorithm) model is used for crack segmentation on the front view image Image_pt, and the physical size parameters of the crack are calculated by combining the skeletonization and distance transformation algorithm, including the maximum width , average width and length of the crack, wherein c represents crack, represents the c-th crack in the image, and specifically includes the following sub-steps: (5.1) input Image_pt, and generate a crack mask by the YOLO model

[0096] (generally set the confidence threshold to 0.7);

[0097] (5.2) skeleton extraction: call the skeleton extraction function skeletonize of the scikit-image library to obtain a skeleton line image ;

[0098] (5.3) crack width calculation: call the distance transformation function distanceTransform of the OpenCV library to obtain the distance from all crack points ("1" value points) in the crack mask to the nearest non-crack point ("0" value point) , take the distance subset belonging to the position of the skeleton line image ; the width value of the c-th crack is twice the corresponding element value in the distance subset ; and the maximum width of the c-th crack is twice the maximum element value in the distance subset ;

[0099] (5.4) crack length calculation: using the 8-neighbor skeletonization algorithm, the length of the c-th crack is the quotient of the sum of non-zero elements in (which can be obtained by using ) and the conversion ratio ;

[0100] (5.5) average width calculation: call the countnozero function of the OpenCV library to obtain the area of each crack in the crack mask , and the average width of the c-th crack is the quotient of and the length of the c-th crack ; ​​​​​

[0101] (5.6) Crack contour extraction: The crack mask is obtained by calling the contour extraction function findCountours in the OpenCV library. Contours in the outline;

[0102] (5.7) Crack Feature Map Image_ft Drawing: Crack feature parameters are labeled on Image_pt. The maximum crack width is marked in red, and the crack outline and its interior are marked in green. The physical feature parameters of the crack, including the maximum crack width, are labeled on the lower right side of the maximum crack width. Average width and length ,in of Represents the first in the image A crack. The final output image is Image_crack.

[0103] In one embodiment, a real-time crack detection system is also included:

[0104] Principle of real-time crack detection system:

[0105] The backend architecture of the real-time crack detection system includes: servers, Alibaba Cloud, and AGL (Automotive Grade Linux). The servers, as the core of the system, are responsible for the operation of the entire detection system, while Alibaba Cloud is responsible for storing input and output images (including crack masks). (Including crack feature parameter maps), Python code for real-time crack detection based on a coin reference is deployed on AGL. The smartphone uploads the image to be tested, Image_test, to the server, which then calls Alibaba Cloud. (OSS Object Storage Service Application Programming Interface) saves the input image to a specified space on Alibaba Cloud and automatically sends commands to AGL to run the Python code: ① When the image resolution is >4K, it is automatically segmented (if the remaining part is less than 512×512, the remaining part is processed directly) into 512×512 sub-images for processing; when the image resolution is <4K, the entire image is processed; ② The output image Image_crack is obtained through the above-mentioned real-time crack detection method based on a coin reference; ③ The Python code calls Alibaba Cloud... Upload the output image (Image_crack) to Alibaba Cloud for storage and generate an information about Image_crack. (Image output application programming interface), and send it to the server; ④ The server automatically reads it. And then it is delivered to the smartphone, which then calls... The output image is displayed. In addition, when the crack detection is completed, if the output result is that the maximum width of the crack is greater than the early warning value (the early warning value is determined by historical data and specification limit value), the Python code sends an early warning signal to the server, and the server sends the early warning signal to the smart phone to prompt the user; if the output result is that the number of coins identified ≠ 1, the smart phone prompts the user to re-shoot the image.

[0106] User-detection system (mobile phone mobile terminal) interaction process:

[0107] The user enters the mobile phone mobile terminal platform. The user must register an account (WeChat can directly associate the account) for the first time, which facilitates the cloud platform to mark information management and privacy protection with a username (such as abc); after the user completes the registration, enter the measurement point (the measurement point is used to distinguish cracks in different positions, and images of the same crack must be in a measurement point according to the rules to accurately track the trend of the crack) selection interface. The measurement point is divided into a new measurement point and a historical measurement point; click the new measurement point to upload the image and identify the crack. At this time, a new measurement point number (such as 001) is automatically established; click the historical measurement point to select a measurement point. Select a measurement point to enter the detailed interface of the measurement point. The interface includes the crack mask of the measurement point identified by the latest time shooting , crack feature parameter diagram, and crack feature parameter (length, average width, maximum width) time curve, which can be selected. New images can be uploaded and cracks can be identified by clicking the new image in this interface.

[0108] Implementation example

[0109] The specific implementation of the application is realized on a detection workstation equipped with an Intel Core i9-14900HX central processor, an NVIDIA GeForce RTX 4060 graphics processor, and 32 GB of memory. All parameter configurations listed in the specific implementation method (COIN_DIAMETER_MM=24.0, MIN_COIN_RADIUS_PX=15, etc.) are adopted, and the experimental results shown in the drawings are obtained. The surface cracks of the detection target object are composed of multiple levels of geometric features, and each crack is converted into a continuous line segment composed of 20-25 key points by a skeletonization algorithm. In real-time analysis, these feature lines are mapped to the orthoview plane through the OpenCV geometric transformation module, and the measurement report is generated by fusing the YOLO segmentation result and the distance transformation data.

[0110] As Figures 1 to 4As shown, compared with the existing detection method, the method can accurately identify the coin edge under the hand shielding condition (the shielding rate of hand_mask is greater than 92%); the test on the linear crack defect form shown in the test shows that the system can stably process the input image with a resolution of 4096*2160; in the crack area of 100mm2, the width measurement accuracy is ±0.15mm, the length calculation error is less than 1.2%, and the area statistical deviation is less than 3%; wherein, Figure 2 The green circle in the figure is the detected coin boundary, and the red dot is the coin center point.

[0111] As shown in the crack identification result in Figure 5 The width of the part marked by the red circle in the figure is measured by a special crack ruler of a company, and the measurement data is shown in Table 1. The crack width of the marked point measured recently is 2.50mm, and the crack width of the corresponding point measured by the system is 2.62mm, with an error of 4.8%, which verifies the effectiveness of the method and system.

[0112]

[0113] As shown in the technical route in Figure 6 The system realizes efficient operation through a multi-stage pipeline processing architecture, and the average processing time on a 4K image is 2.3 seconds / frame (including the whole process of coin detection, perspective correction and crack analysis), which is 40 times higher than the efficiency of traditional manual measurement. Experimental data verifies that the present application has stable detection performance on the surface of different building materials (concrete, ceramic tile, metal, etc.).

[0114] As shown in the crack identification mobile phone mobile terminal platform schematic diagram in Figure 7 A real-time crack detection system based on coin reference is truly established, the cloud platform implements privacy management of crack information based on a username, users can newly create a measurement point or re-upload an image in a historical measurement point to perform crack identification, which facilitates users to monitor the trend of a specific crack and also facilitates the cloud platform to timely warn based on the trend.

[0115] Corresponding to the foregoing embodiment of the crack real-time detection method based on the coin reference, the present application also provides an embodiment of a crack real-time detection device based on the coin reference.

[0116] Referring to Figure 8 The crack real-time detection device based on the coin reference provided by the embodiment of the present application comprises one or more processors for realizing the crack real-time detection method based on the coin reference in the foregoing embodiment.

[0117] The processor can be a central processing unit (CPU), and can also be other general-purpose processors, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field-programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic device, discrete hardware component, etc. The general-purpose processor can be a microprocessor or the processor can also be any conventional processor.

[0118] An embodiment of the coin reference-based crack real-time detection device of the present application can be applied to any device with data processing capability, which can be a device or apparatus such as a computer. The device embodiment can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a logically meaningful device, it is formed by reading the corresponding computer program instructions in the non-volatile memory into the memory and running by the processor of the device with data processing capability where the device is located. From the hardware level, as shown in the figure, it is a hardware structure diagram of the device with data processing capability where the coin reference-based crack real-time detection device of the present application is located, in addition to the processor, the memory, the network interface, and the non-volatile memory shown in the figure, the device with data processing capability where the device is located usually includes other hardware according to the actual functions of the device with data processing capability, which will not be described here. Figure 8 Figure 8 From the hardware level, as shown in the figure, it is a hardware structure diagram of the device with data processing capability where the coin reference-based crack real-time detection device of the present application is located, in addition to the processor, the memory, the network interface, and the non-volatile memory shown in the figure, the device with data processing capability where the device is located usually includes other hardware according to the actual functions of the device with data processing capability, which will not be described here.

[0119] The implementation process of the functions and roles of each unit in the above device is specifically described in the implementation process of the corresponding steps in the above method, which will not be described here.

[0120] For the device embodiment, since it basically corresponds to the method embodiment, the related parts can be referred to the part of the method embodiment. The above described device embodiment is only schematic, and the units described as separate components can or can not be physically separated, and the components displayed as units can or can not be physical units, i.e. they can be located in one place, or distributed on multiple network units. According to the actual needs, part or all of the modules can be selected to achieve the purpose of the present application scheme. Those skilled in the art can understand and implement without creative labor.

[0121] ​The embodiment of the present application further provides a computer readable storage medium, which stores a program, and the program is executed by a processor to realize the coin reference based crack real-time detection method in the above embodiment.

[0122] The computer readable storage medium can be an internal storage unit of any data processing capable device, such as a hard disk or a memory, according to any of the above embodiments. The computer readable storage medium can also be any data processing capable device, such as a plug-in hard disk, a smart media card (SMC), an SD card, a flash card, etc. Further, the computer readable storage medium can include both an internal storage unit of any data processing capable device and an external storage device. The computer readable storage medium is used to store the computer program and other programs and data required by the data processing capable device, and can also be used to temporarily store data that has been output or will be output.

[0123] The above embodiments are only used for illustrating the design idea and characteristics of the present application, and the purpose is to enable those skilled in the art to understand the present application and implement it, and the protection scope of the present application is not limited to the above embodiments. Therefore, any equivalent changes or modifications made according to the disclosed principles and design ideas of the present application are within the protection scope of the present application.

Claims

1. A coin reference based real-time crack detection method, characterized by, The method comprises the following steps: Taking an image to be tested containing standard-size coins and a crack area by a mobile shooting device; Performing semantic segmentation on the hand in the image to be tested to shield the hand interference area and obtain a gray image after shielding the hand; Obtaining an effective fitting coin set by sequentially applying a feature enhancement module, an edge detection and morphological closing module, and a fitting coin detection module to the gray image; The feature enhancement module: applying limited contrast self-adaptive histogram equalization and Gaussian blur algorithm to enhance the features of the coins in the gray image Image_non_hand to obtain an image Image_feat after enhancing the features of the coins; The edge detection and morphological closing module: locating the coins in the image Image_feat based on an occlusion edge detection algorithm and obtaining the boundaries of the coins; Performing morphological closing processing on the boundaries of the non-intact coins to obtain a processed image Image_proc; The edge detection and morphological closing module comprises: Edge detection module: the gray matrix of the image Image_non_hand Compute the double threshold Canny operator , ; then call the Canny function of the OpenCV library to obtain the binary edge image Edges; The morphological closing module: performing closing processing on the binary edge image Edges, using an elliptical structural element S, performing a morphological closing operation of "first expansion and then corrosion" on the edge image to obtain a processed image Image_proc; The fitting coin detection module: obtaining an effective fitting coin set by performing an ellipse fitting-Hough circle-geometric verification three-step detection on Image_proc; The fitting coin detection module comprises: Ellipse fitting priority mode: perform connected contour search on the processed image Image_proc, screen non-complete ellipse regions with area greater than a threshold , perform ellipse fitting on non-complete ellipses with contour feature point number not less than 5 to obtain candidate regions , , , , respectively as the center coordinates and radius of the circle ; right Calculate the ratio of major to minor axis , ,in , The first The lengths of the major and minor axes of each candidate contour ellipse; Let the roundness threshold be... ,when When a candidate contour ellipse is determined to be a valid fitted coin, a set of valid fitted coins is obtained, and the center coordinates of the ellipse are output. and equivalent radius ; Hough circle backup mode: when the ellipse fitting priority mode cannot obtain an effective fitting coin, switch to the Hough circle fitting mode, call the HoughCircles function of the OpenCV library to locate the coin in the image Image_proc, and obtain the candidate region ; candidate regions The candidate regions are filtered to obtain new candidate regions ; for candidate regions compute their average radius , reject outliers with radius deviation exceeding , obtaining the set of valid fitted coins , is a tolerance coefficient; Identifying and selecting the coin with the largest radius from the effective fitting coin set as a reference benchmark, determining the conversion ratio of pixels to actual size, correcting the image to be tested to an orthographic plane by a perspective transformation algorithm to generate an orthographic image of the surface to be tested containing cracks; Performing crack segmentation on the orthographic image of the surface to be tested to calculate the physical size parameters of the cracks.

2. The method of claim 1, wherein, Coin diameter 5%~10% of the width of the image to be tested 5%~10% of the width of the image to be tested 3. The method of claim 1, wherein, The method for performing semantic segmentation on the hand in the image to be tested to shield the hand interference area comprises: Generate hand region mask for test image Image_test through Hands hand detector in MediaPipe library .

4. The method of claim 1, wherein, The method for generating an orthographic image of the surface to be tested containing cracks comprises: Establishing a mapping relationship between an imaging plane and a target plane to obtain a perspective transformation matrix , comprising: Imaging plane reference point selection: selecting the center of the coin with the largest radius from the set of valid fitted coins Taking the center as the center, four extreme points are extracted at four angular directions on the circumference as the imaging plane coordinate point set ;​ Target plane reference point selection: let wherein is the conversion ratio of pixels to actual size, so that , , and four points establish a target plane coordinate system, and the four points form a target plane coordinate point set ; Transformation matrix calculation: based on the above and The homography matrix is ​​calculated using the least squares method and satisfies: Calculate the distance from the four corner points of the image to be tested, Image_test. The transformed boundary coordinates are used to determine the size of the target image; a translation correction matrix is ​​then constructed. This aligns the top-left corner of the target image with the origin of the coordinate system; therefore, the perspective transformation matrix... ; Calling the perspective transformation function warpPerspective of the OpenCV library to obtain the coordinates of each pixel point of the image to be tested Image_test on the target plane; Performing pixel resampling processing to obtain the pixel values of the target plane, comprising: a bicubic interpolation function INTER_CUBIC of the OpenCV library is called to calculate the corresponding pixel value of the imaging plane sampling point in the target plane by weighted combination of 16 pixel values adjacent to the imaging plane sampling point to generate a pixel matrix ; wherein, for each adjacent pixel, its weight is determined by a piecewise continuous cubic polynomial kernel function according to its distance to the sampling point , so as to calculate the corresponding pixel value of the imaging plane sampling point in the target plane; ； wherein, for achieving an optimal sharpness-smoothness balance; Obtaining the pixel values of each pixel point of the image to be tested Image_test on the target plane to generate an orthographic image Image_pt of the surface to be tested containing cracks.

5. The method of claim 1, wherein, The method for performing crack segmentation on the orthographic image of the surface to be tested to calculate the physical size parameters of the cracks comprises: The YOLO model is used to segment the cracks in the front view image Image_pt. The physical size parameters of the cracks, including the maximum crack width, are calculated by combining skeletonization and distance transformation algorithms. Average width and length ,in Represents the first in the image A crack.

6. A coin reference based crack real-time detection apparatus, characterized by, The device comprises one or more processors for implementing the method for real-time crack detection based on a coin reference according to any one of claims 1-5.

7. A computer-readable storage medium having stored thereon a program, characterized in that, The program is executed by the processor to implement the method for real-time crack detection based on a coin reference according to any one of claims 1-5.

Images