Intelligent crack monitoring system based on machine vision and cloud platform

Abstract

The invention relates to an intelligent crack monitoring system based on machine vision and a cloud platform. The intelligent crack monitoring system comprises a front-end acquisition and transmission device and the cloud platform, the front-end acquisition and transmission device is used for acquiring a crack image of a target area through a self-adaptive light supplement technology, encrypting the crack image and transmitting the encrypted crack image to the cloud platform through a mobile communication network; and the cloud platform is used for carrying out preprocessing, image segmentation and identification on the crack image, calculating the width and length of the crack, obtaining data after intelligent processing, and storing the data after intelligent processing in a cloud server and a database, the image recognition comprises the step of calling a pre-trained recognition model to recognize the crack image according to a user preset condition and crack image quality. According to the invention, comprehensive, accurate and efficient processing of crack image data acquired in intelligent crack monitoring can be realized.

Description

Technical Field

[0001] The present invention relates to the technical field of building engineering structure quality monitoring, and particularly to an intelligent crack monitoring system based on machine vision and cloud platform. Background Art

[0002] In the maintenance and inspection of civil engineering, especially large-scale infrastructure such as bridges and tunnels, the monitoring of surface cracks is crucial. Cracks are one of the important factors affecting the structural safety of buildings. The early cracks on buildings are all small cracks, which will continuously develop and extend into thicker cracks after long-term growth and change. When the width of the cracks exceeds a certain limit, it will seriously affect the safety of the building structure. Therefore, while detecting the crack width, the change of the crack width should also be monitored to master the development of the building cracks. At present, the most commonly used crack detection method is mainly manual detection. The detection personnel use instruments to detect cracks or directly observe them with the naked eye. This method must be in close contact with the cracks, and it is easy to be affected by the subjectivity of the detection personnel during the detection process, resulting in inaccurate measurement data. Moreover, there are problems of low efficiency, time-consuming and laborious, and it is not conducive to effectively monitoring the change of the crack width in the later stage. With the increase and aging of urban infrastructure, the task of monitoring and maintaining surface cracks of infrastructure such as buildings, bridges and roads is becoming increasingly heavy. The traditional detection method relies on manual inspection, which is inefficient, costly and cannot achieve real-time monitoring. The purpose of the present invention is to provide an automated, efficient and real-time crack safety monitoring solution by integrating machine vision and cloud platform technology to realize the real-time monitoring of cracks on the surfaces of infrastructure such as bridges, dams, roads and houses. Summary of the Invention

[0003] The purpose of the present invention is to provide an intelligent crack monitoring system based on machine vision and cloud platform. The whole system realizes the automated acquisition, data transmission, intelligent identification and measurement management of crack monitoring through the front-end acquisition and transmission device and the cloud platform intelligent processing center, and finally realizes the intelligent monitoring of cracks on the surfaces of building facilities such as bridges, dams, roads and houses.

[0004] To achieve the above purpose, the present invention provides the following solutions:

[0005] An intelligent crack monitoring system based on machine vision and cloud platform, comprising: a front-end acquisition and transmission device and a cloud platform;

[0006] The front-end acquisition and transmission device is used to collect crack images of the target area through adaptive light compensation technology, and encrypt and transmit the crack images to the cloud platform through a mobile communication network;

[0007] The cloud platform is used to preprocess the crack image, segment the image, identify and calculate the width and length of the crack, obtain the data after intelligent processing, and store the data after intelligent processing in the cloud server and database. Among them, the image recognition includes calling a pre-trained recognition model to recognize the crack image according to the user's preset conditions and the quality of the crack image.

[0008] Optionally, the front-end acquisition and transmission device includes: an image acquisition module, an LED autonomous control lighting compensation module, an MCU main control module, a 4G-Cat1 transmission module, and a micro-power lithium battery power module;

[0009] The image acquisition module is used to carry a high-resolution camera and a macro lens, combine the macro lens with the adaptive lighting compensation technology of the LED autonomous control lighting compensation module, and acquire the crack image;

[0010] The LED autonomous control lighting compensation module is used to automatically adjust the LED brightness according to the ambient light intensity and eliminate shadow interference through diffuse reflection lighting compensation;

[0011] The MCU main control module is used to control the image acquisition module, the LED autonomous control lighting compensation module, the 4G-Cat1 transmission module, and the micro-power lithium battery power module, perform image noise and interference signal elimination processing on the crack image, and compress the denoised crack image and send it to the 4G-Cat1 transmission module;

[0012] The 4G-Cat1 transmission module is used to transmit the acquired crack image to the cloud platform through the network using the TCP / IP protocol;

[0013] The micro-power lithium battery power module is used to supply power to the image acquisition module, the LED autonomous control lighting compensation module, the MCU main control module, and the 4G-Cat1 transmission module by combining the power supply with a low-power power management unit.

[0014] Optionally, the cloud platform includes: an image preprocessing module, a crack image segmentation and recognition module, a crack image library, a recognition result library, and a recognition model library;

[0015] The image preprocessing module is used to perform grayscale conversion, threshold segmentation, and morphological processing on the crack image to obtain the crack image after preprocessing;

[0016] The crack image segmentation and recognition module is used to extract the crack skeleton according to the threshold segmentation algorithm to obtain a binary image, calculate the crack width and length, call a pre-trained recognition model from the recognition model library to recognize the crack image after preprocessing, obtain the crack recognition result, and store the recognition result and the calculated value in the recognition result library;

[0017] The crack image library is used to store crack image samples. When the number of newly added crack image samples exceeds a preset value, the newly added crack image samples are extracted to train the models in the recognition model library;

[0018] The recognition result library is used to store recognition results and calculation results;

[0019] The recognition model library is used to store pre-trained recognition models.

[0020] Optionally, calculating the crack width and length includes:

[0021] Obtaining the area of the crack according to the binary image, measuring the crack length using the center line method according to the area of the crack, and calculating the center line length through the position of the center line of the target crack as the length of the crack;

[0022] Calculating the average width of the crack according to the length of the crack and the distance between two adjacent center points, and obtaining the minimum width and maximum width of the crack according to the binary image.

[0023] Optionally, obtaining the minimum width and maximum width of the crack according to the binary image includes:

[0024] Obtaining the contour of the crack according to the binary image;

[0025] Calculating the minimum circumscribed matrix of each contour according to the contour of the crack;

[0026] Rotating the contour of the crack until the contour is parallel to the width direction of the minimum circumscribed matrix;

[0027] Calculating the number of non-zero pixels of the rotated contour in the width direction to obtain the minimum width and maximum width of the crack.

[0028] Optionally, the crack image library includes: a measuring point information table, a crack original acquisition image table, and a preprocessed crack image table;

[0029] The measuring point information table records the basic information and deployment parameters of the monitoring points, and the attributes include the project where it is located, the measuring point number, the installation position coordinates, and the installation drawing;

[0030] The crack original acquisition image table stores the original crack image data that has not been processed, and the attributes include the image number, the image format type, the image size, and the storage path;

[0031] The preprocessed crack image table stores the preprocessed intermediate image files, and the attributes include the grayscale file, the threshold segmentation result file, and the morphological processing file.

[0032] Optionally, the recognition result library includes a crack recognition result table and a calculation result table;

[0033] The attributes of the crack recognition result table include crack type and recognition confidence value;

[0034] The attributes of the calculation result table include crack length value, crack width value, length increment, and width increment.

[0035] Optionally, the recognition model library includes: a model information table and a training information table;

[0036] The attributes of the model information table include model type, model number, recognition accuracy, and computing power requirement value;

[0037] The attributes of the training information table include training sample size, training model number, and training history information.

[0038] The beneficial effects of the present invention are as follows: (1) The present invention uses a mobile network to connect to a cloud platform, and data is sent from a wireless module to the cloud platform through the mobile network, realizing a high degree of integration of on-site data and computing resources. The system utilizes the powerful computing power of the cloud platform to process large-scale data and provides flexible scalability.

[0039] (2) The system realizes comprehensive, accurate, and efficient processing of the crack image data collected in intelligent crack monitoring through multi-level image analysis and processing, combined with model training optimization, and an efficient data storage and visualization display mechanism.

[0040] (3) The crack image segmentation, recognition, and width calculation provided by the present invention are the core of the system's intelligent recognition. Deep learning algorithms are applied on the cloud platform to accurately locate the crack position and width, ensuring the accuracy and efficiency of crack detection.

[0041] (4) The web server of the cloud platform provided by the present invention can perform data analysis, compare historical data, predict the development trend of future crack detection, and support accurate maintenance decisions.

[0042] (5) The present invention stores the detection results and the original images in the database on the cloud platform, ensuring data security and supporting the traceability and analysis of data history.

[0043] (6) Through the web server on the cloud platform, the present invention intuitively presents the monitoring data and analysis results. The change of the crack situation and its historical data can be viewed in real time through the web client, facilitating timely maintenance and decision-making. Description of the Drawings

[0044] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the accompanying drawings required for use in the embodiments. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For those of ordinary skill in the art, without creative efforts, other accompanying drawings can be obtained based on these drawings.

[0045] Figure 1 Schematic diagram of an intelligent crack monitoring system based on machine vision and cloud platform according to an embodiment of the present invention;

[0046] Figure 2 Overall working flowchart of an intelligent crack monitoring system based on machine vision and cloud platform according to an embodiment of the present invention;

[0047] Figure 3 Composition schematic diagram of the front-end acquisition and transmission device according to an embodiment of the present invention;

[0048] Figure 4 Working flowchart of the front-end acquisition and transmission device according to an embodiment of the present invention;

[0049] Figure 5 Mechanical structure diagram of the front-end acquisition and transmission device according to an embodiment of the present invention;

[0050] Figure 6 Composition schematic diagram of the cloud platform according to an embodiment of the present invention;

[0051] Figure 7 Working flowchart of the cloud platform according to an embodiment of the present invention;

[0052] Figure 8 Design diagram of the composition of the cloud platform data center according to an embodiment of the present invention. Specific implementation manners

[0053] The following will clearly and completely describe the technical solutions in the embodiments of the present invention with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, rather than all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts belong to the scope of protection of the present invention.

[0054] To make the above objects, features, and advantages of the present invention more clearly understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific implementation manners.

[0055] As Figure 1 shown, this embodiment provides an intelligent crack monitoring system based on machine vision and cloud platform, including: a front-end acquisition and transmission device and a cloud platform;

[0056] The front-end acquisition and transmission device is used to collect crack images of the target area through adaptive light compensation technology and encrypt and transmit the crack images to the cloud platform through the mobile communication network;

[0057] The cloud platform is used to preprocess, segment, identify and calculate the width and length of the cracks in the crack images, obtain the intelligently processed data, and store the intelligently processed data in the cloud server and the database. Among them, image recognition includes calling a pre-trained recognition model to recognize the crack images according to user preset conditions and crack image quality.

[0058] Specifically, the system mainly consists of the front-end acquisition device and transmission part, the cloud platform, and the connection between the two through the mobile network, including the front-end acquisition device and transmission module, machine vision crack image acquisition, wireless network transmission module, main control module, cloud platform, crack image segmentation, recognition, width calculation module, web server and database storage module. The front-end acquisition device includes a machine vision crack image acquisition module with a high-resolution camera as the core, a wireless network transmission module with 4G-CAT1 as the core, and a main control module with a microprocessor as the core, mainly completing the acquisition of high-definition image data of building surface cracks and transmitting the crack image data to the cloud platform through the wireless network transmission module. The cloud platform constructs an intelligent processing center for crack image data, uses a switchable intelligent recognition model to perform processing such as segmentation, recognition and width calculation on the received crack images, stores the relevant data in the database, and at the same time provides data visualization display by the web server.

[0059] Furthermore, the front-end acquisition and transmission device includes: an image acquisition module, an LED autonomous control light compensation module, an MCU main control module, a 4G-Cat1 transmission module, and a micro-power lithium battery power module;

[0060] The image acquisition module is used to carry a high-resolution camera and a macro lens, combine the macro lens with the adaptive light compensation technology of the LED autonomous control light compensation module, and collect crack images;

[0061] The LED autonomous control light compensation module is used to automatically adjust the LED brightness according to the ambient light intensity and eliminate shadow interference through diffuse reflection light compensation;

[0062] The MCU main control module is used to control the image acquisition module, the LED autonomous control light compensation module, the 4G-Cat1 transmission module, and the micro-power lithium battery power module, perform image noise and interference signal elimination processing on the crack images, and compress and send the denoised crack images to the 4G-Cat1 transmission module;

[0063] The 4G-Cat1 transmission module is used to transmit the collected crack images to the cloud platform through the network using the TCP / IP protocol;

[0064] A micro-power lithium battery power module is used to supply power to an image acquisition module, an LED independent control supplementary lighting module, an MCU main control module, and a 4G-Cat1 transmission module by combining a power supply with a low-power power management unit.

[0065] Specifically, the front-end acquisition device is responsible for capturing the crack images on the building surface and transmitting the data to the cloud platform through the wireless network transmission module. It uses a high-resolution camera and wireless communication equipment to ensure image quality and communication reliability. As Figure 3 shown, the front-end acquisition and transmission device includes a machine vision crack image acquisition module with a macro high-definition image sensor as the core, an LED independent control supplementary lighting module, an MCU main control module, a 4G Cat1 transmission module, and a micro-power lithium battery power module. Each component works together. Among them, the machine vision acquisition module is equipped with a high-resolution camera and a macro lens. The macro lens is combined with the adaptive supplementary lighting technology to ensure clear imaging of millimeter-level cracks. The crack image data is collected by the high-definition cameras deployed in key areas according to the formulated rules. The LED independent control supplementary lighting module automatically adjusts the LED brightness according to the ambient light intensity, eliminates shadow interference through diffuse reflection supplementary lighting, and ensures image clarity in low-light or backlight scenarios. The MCU main control module, as the core control module of the entire front-end acquisition and transmission, coordinates the operation of the entire system and processes and controls the work processes of each component. The 4G Cat1 transmission module transmits the collected crack image data to the cloud platform system at the back end through the 4G network. The micro-power lithium battery power module provides power support for the entire front-end acquisition and transmission module, ensures the stable operation of the system in a low-power state, and is paired with a dynamic power consumption management strategy. Through the sleep-wake mechanism, the standby power consumption of the system is guaranteed to be lower than 10 μA, and the battery life can reach 6 - 12 months.

[0066] As Figure 4The working process of the front-end acquisition and transmission device is as follows: First, the micro-power consumption power management system is powered on and starts to supply power to the main control unit, the image acquisition unit, the supplementary lighting system, and the wireless transmission unit. The main control unit controls each subsystem to complete initialization according to the preset parameters, loads the preset parameters (such as acquisition frequency, supplementary lighting threshold, communication protocol, etc.), and completes the configuration and self-check of each subsystem. The main control unit first controls the image acquisition unit to collect macro high-definition images of the cracks. Based on the ambient light sensor data, it dynamically adjusts the LED brightness and controls the LED supplementary lighting system to automatically assist macro high-definition imaging according to the environmental conditions. The main control unit temporarily stores the machine vision crack image acquisition data in the internal RAM storage area, runs a lightweight algorithm to eliminate image noise and interference signals, and completes the preliminary noise reduction process. The main control unit extracts the denoised crack image data from the RAM, compresses it, and sends it to the send buffer of the 4G-Cat1 transmission module. At the same time, it encapsulates it according to the communication protocol format, adds a timestamp, a measurement point number, etc., and completes the data transmission preparation. The 4G-Cat1 transmission module encrypts and transmits the crack image data to the cloud platform through the mobile communication network using the TCP / IP protocol, and at the same time feedbacks the completion flag. After receiving the "transmission completed" confirmation, the main control unit turns off the unnecessary modules and makes the system enter the low-power sleep mode. The main control module coordinates all operations of the front-end acquisition unit, including the deployment of the camera, the timing of image acquisition, and the setting of parameters (such as exposure time, resolution, etc.). It controls the data acquisition process by sending instructions to the acquisition device to ensure the orderly operation of each module. At the same time, it coordinates the LED autonomous control supplementary lighting system to perform the supplementary lighting operation; after the crack image data acquisition is completed, it is processed and controlled by the MCU main control unit and the acquired image data is transmitted through the 4G network via the 4G-Cat1 module.

[0067] Furthermore, the front-end acquisition and transmission device is divided into an upper chamber and a lower chamber;

[0068] The upper chamber includes a battery chamber for setting the micro-power consumption lithium battery power module;

[0069] The upper chamber also includes a photosensitive sensor installation chamber and a photosensitive chamber for ambient light monitoring. In the central area of the lower chamber, a main control unit installation chamber is set for installing the MCU main control module; a 4G transmission module installation chamber is set for installing the 4G communication module; a crack image acquisition area is set for installing the image acquisition module, the LED autonomous control supplementary lighting module, and the image light-transmitting plate.

[0070] Specifically, the mechanical structure diagram of the front-end acquisition and transmission device is as Figure 5 shown, which shows the structural layout of the "front-end integrated crack image acquisition and transmission device". The overall design is divided into an upper chamber and a lower chamber, and each functional module is set in partitions according to the actual installation requirements. The overall adopts an integrated design and a modular layout, and each unit is independently installed, which is convenient for maintenance and upgrade.

[0071] The upper bin mainly houses the power supply and basic function modules: The power management includes a battery compartment and a low-power power management unit, which provides stable power supply for the device, supports long-term operation. The battery compartment has an in-built power supply, paired with a low-power power management unit to optimize energy consumption; The auxiliary module contains a photosensitive sensor installation bin and a photosensitive bin, which are used for ambient light monitoring. The photosensitive sensor installation bin is fixed inside the upper bin through studs passing through the stud holes. The bottom of the photosensitive bin is designed with square light-transmitting holes. A light-transmitting plate is used to separate the photosensitive sensor from the light-transmitting holes. The photosensitive bin and the light-transmitting plate work together to assist in the adjustment of ambient light and enhance image quality. The top of the photosensitive sensor installation bin and the battery compartment are designed with light-transmitting plates and a battery compartment cover, taking into account both protection and functionality. The design of the battery compartment cover facilitates maintenance and replacement.

[0072] The lower bin contains the core control and image processing modules: The main control module installation bin is located in the central area at the bottom inside the lower bin, responsible for coordinating the operation of the device, integrating the core control functions of the device, and coordinating the work of sensors and transmission modules; The 4G transmission module installation bin is connected to the main control module and is used to transmit the collected crack image data in real time; The image acquisition core area includes a crack image acquisition area and a micro-distance clear image acquisition sensor installation bin, which are responsible for high-precision acquisition of crack images. The micro-distance clear image acquisition sensor installation bin is fixed to the crack image acquisition area through studs, equipped with an LED supplementary lighting system and an image light-transmitting plate. It can provide uniform lighting through the LED supplementary light lamp light-transmitting holes in low-light environments, ensuring high-definition imaging in the crack image acquisition area. The front-end crack image acquisition area integrates sensors and a supplementary lighting system with LED supplementary light lamp light-transmitting holes and an image light-transmitting plate. By providing uniform supplementary lighting, it improves image clarity. Combined with the back-end control and transmission modules, it realizes the full-process functions from image capture, processing to remote transmission, and is applicable to crack monitoring scenarios in complex environments.

[0073] Furthermore, the cloud platform includes: an image preprocessing module, a crack image segmentation and recognition module, a crack image library, a recognition result library, and a recognition model library;

[0074] The image preprocessing module is used to perform grayscale conversion, threshold segmentation, and morphological processing on the crack images to obtain the preprocessed crack images;

[0075] The crack image segmentation and recognition module is used to extract the crack skeleton according to the threshold segmentation algorithm to obtain a binary image, calculate the crack width and length, and retrieve the pre-trained recognition model from the recognition model library to recognize the preprocessed crack images, obtain the crack recognition results, and store the recognition results and calculated values in the recognition result library.

[0076] The crack image library is used to store crack image samples. When the number of newly added crack image samples exceeds the preset value, the newly added crack image samples are extracted to train the models in the recognition model library;

[0077] An identification result library for storing identification results and calculation results;

[0078] An identification model library for storing pre-trained identification models.

[0079] Furthermore, calculating the crack width and length includes:

[0080] Obtaining the area of the crack from the binary image, measuring the crack length using the centerline method based on the area of the crack, and calculating the centerline length as the crack length through the centerline position of the target crack;

[0081] Calculating the average width of the crack based on the crack length and the distance between two adjacent center points, and obtaining the minimum width and maximum width of the crack from the binary image.

[0082] Furthermore, obtaining the minimum width and maximum width of the crack from the binary image includes:

[0083] Obtaining the contour of the crack from the binary image;

[0084] Calculating the minimum circumscribed matrix of each contour based on the contour of the crack;

[0085] Rotating the contour of the crack until the contour is parallel to the width direction of the minimum circumscribed matrix;

[0086] Calculating the number of non-zero pixels of the rotated contour in the width direction to obtain the minimum width and maximum width of the crack.

[0087] Specifically, such as Figure 6The figure shows the schematic composition of the cloud platform. As the data processing and storage center of the system, the cloud platform receives the image data transmitted from the front end and performs processing such as segmentation, recognition, and width calculation of crack images using advanced machine learning algorithms. It includes crack image recognition and width calculation, image preprocessing, crack feature extraction, crack image segmentation and recognition, pre-trained image recognition model, web server and data storage, crack image library, recognition result library, web server and display. Among them, the crack image recognition and width calculation module is mainly responsible for initially recognizing the crack images collected and transmitted from the front end and calculating their width information. The image preprocessing module performs preprocessing operations on the images after crack image recognition and width calculation. The crack feature extraction module extracts the feature information in the crack images based on the preprocessed image data. The extracted features can be some characteristics such as shape and texture features that are helpful for subsequent accurate crack image segmentation and recognition. The crack image segmentation and recognition module uses the extracted crack feature information to segment the crack images, further determine the specific shape, position and other details of the cracks, divide the cracks from other background areas, make the crack information clearer and more definite, and retrieve the pre-trained recognition model from the recognition model library to recognize the preprocessed crack images. The pre-trained image recognition model is pre-trained on the model through an existing large-scale crack and non-crack image dataset, enabling it to learn various patterns and feature representations of crack images. Then, in actual processing, the model is continuously fine-tuned and optimized according to the collected crack image data to improve its recognition ability for various types of crack images. The crack image library stores the crack image data collected by machine vision at various times, which is the collection place of the original image information in the intelligent crack monitoring system, covering the crack images that appear on the surfaces of different types of structures under different environmental conditions. The recognition result library stores the final analysis results after crack image recognition, including quantitative data such as the position, width, length, and development trend of the cracks, as well as qualitative conclusions such as the assessment of the danger level of the cracks. It is the key information source for the entire monitoring system to provide support for subsequent engineering decisions. The web server and data storage, as the hub of data storage and external interaction, undertakes two important tasks. On the one hand, it stores a large amount of crack image information after crack image segmentation and recognition into the crack image library; on the other hand, it stores the crack recognition results obtained after a series of processes such as crack image recognition and width calculation into the recognition result library. The web server and display display the crack-related recognition results stored in the recognition result library in a visual manner on the web page through the display function of the web server, so that relevant personnel (such as engineering monitoring personnel, maintenance personnel, etc.) can conveniently and intuitively view and analyze the relevant information of the cracks.

[0088] Such as Figure 7The figure shows the workflow diagram of the cloud platform. The workflow of the cloud platform specifically includes: the cloud platform processing center receives the crack image monitoring data, and first preprocesses the image, including grayscale conversion, threshold segmentation, and morphological processing; establishes a crack image library, an identification result library, and a crack image recognition model library to store crack image samples, identification results, and pre-trained crack image recognition models respectively; retrieves a suitable recognition model from the recognition model library to extract features and identify the preprocessed crack image, extracts the crack recognition result, segments the crack image through the threshold segmentation algorithm to extract the crack skeleton (i.e., the segmentation result), and completes the calculation of the crack width and length; stores the recognition result, segmentation result, and calculated value output by the recognition model in the database, updates the original crack image library, and extracts 10% from it as new sample data for the retraining of the recognition model, where the new samples are extracted from multiple original images collected during the monitoring process; the web server extracts the recognition result and calculated value from the recognition result library and displays the data on the web page and mobile terminal through the visualization interface. Starting from crack image recognition and width calculation, it successively goes through core processing steps such as image preprocessing, crack feature extraction, and crack image segmentation and recognition. In this process, the pre-trained image recognition model provides training support and optimization for crack image segmentation and recognition, and updates and optimizes the crack images stored in the crack image library. The corresponding recognition results will be recorded in the recognition result library; the web server and data storage are responsible for establishing a data storage mechanism to store the intermediate and final results in an orderly manner in different libraries; finally, the final processing result is presented through the web server and display; thus forming a complete closed-loop process from data collection input to processing, storage, optimization, and then to output display.

[0089] The method for preprocessing the image includes grayscale conversion, threshold segmentation, and morphological processing. Among them, the grayscale conversion of the image adopts the weighted average method, and selects the weighted average brightness of the red, green, and blue components in the pixel as the grayscale value of the pixel. The calculation formula is as follows:

[0090] Gray(x,y)=0.299R(x,y)+0.587G(x,y)+0.114B(x,y)

[0091] Among them, Gray(x,y) represents the grayscale value of the image after grayscale conversion at (x,y); R(x,y), G(x,y), B(x,y).

[0092] respectively represent the red, green, and blue color component values of the color image at (x,y).

[0093] Meanwhile, in order to reduce noise and improve the quality of crack images, this embodiment performs filtering processing on the images to enhance the images of cracks and effectively reduce interference. The present invention provides three filtering methods including mean filtering, Gaussian filtering, and median filtering for filtering crack images, which can be selected according to different imaging environments and image qualities to achieve the best filtering effect.

[0094] In the threshold segmentation of crack images, this embodiment adopts three threshold calculation methods, including the iterative method, the Otsu method, and the maximum entropy method, and calls the calculation methods according to the following principles: the iterative method is selected for targets with small areas and low requirements for distinguishing image details, the Otsu method is used for real-time threshold segmentation with high stability requirements and low algorithm time complexity, and the maximum entropy method is used for crack images with high requirements for retaining image details and large noise and uneven illumination. The present invention improves the threshold segmentation effect by flexibly selecting the threshold calculation method for crack images with different qualities and requirements.

[0095] The pixel point method is used to calculate the length and average width values of cracks:

[0096] After the crack image is segmented, a binary image is formed, which only contains pixel points with gray values of 1 and 0. The pixel points with a gray value of 1 represent the target crack pixel points, and the pixel points with a gray value of 0 represent non-crack pixel points.

[0097] First, count the number of pixel points where CRack(x, y) = 1 to obtain the area S of the crack crack . The calculation formula is:

[0098]

[0099] Secondly, the center line method is used to measure the crack length, and the center line length is calculated through the position of the center line of the target crack as the crack length. The specific steps are as follows:

[0100] (1) Store the crack binary image information in the matrix Z of M*N, scan the matrix row by row to determine the leftmost edge pixel coordinates and the rightmost edge pixel coordinates of the crack in each row, and record them as (x mn , y i ) and (x i , y j , y i ).

[0101] (2) Calculate the center point coordinates of each row of the matrix Z mn ((x i + x j ) / 2, y i ).

[0102] (3) Calculate the distance Li between two adjacent center points in sequence. The calculation formula is:

[0103] (4) Calculate the sum of the distances between adjacent points, which is the length L of the crack crack , and the calculation formula is:

[0104] Finally, calculate the average width W of the crack crack , and the calculation formula is: W crack = S crack / L i .

[0105] This embodiment also provides a method and steps for calculating the minimum width and maximum width of the crack and determining their positions:

[0106] (1) Find the contour of the crack from the segmented mask image.

[0107] (2) Calculate the minimum bounding rectangle of each contour.

[0108] (3) Rotate each contour to make it parallel to the width direction of the minimum bounding rectangle.

[0109] (4) Calculate the number of non-zero pixels of the rotated contour along the width direction to determine the minimum width and maximum width.

[0110] (5) Determine the center point positions of the minimum width and maximum width of the crack in the original image.

[0111] (6) Finally, mark these center points on the original image.

[0112] Finally, save the marked image into the recognition result library.

[0113] This embodiment provides a crack recognition model library, which stores multiple pre-trained crack recognition models, such as YOLOv5, YOLOv8, YOLOv9, YOLOv10, and YOLOv11 and their corresponding improved versions. The performance parameters of each version are different. The system calls the most suitable model according to the quality of the image. At the same time, it stores the crack detection performance indicators corresponding to each recognition model. The system automatically calls the appropriate recognition model according to the performance indicators of the model, user settings, and the quality of the crack image to complete the recognition of the crack image.

[0114] When the new sample volume in the crack image sample library in this embodiment exceeds 30%, 10% of the new sample data will be extracted from the new samples for retraining the recognition model, so as to improve the recognition rate of the recognition model for the new image samples.

[0115] Such as Figure 8The figure shows the design diagram of the cloud platform data center of the present invention. The data center of the intelligent processing unit of the cloud platform consists of three sub-databases, including a crack image library, an identification model library, and a processing result library. It supports the full-process management from data collection, preprocessing, model training to result storage. The functions of each data table are clear and the levels are distinct. Data association and traceability are realized through key fields, providing reliable data support for the automatic identification, quantitative analysis, and long-term tracking of cracks.

[0116] Furthermore, the data tables in the crack image library include a measuring point information table, a raw crack acquisition image table, and a preprocessed crack image table.

[0117] Among them, the measuring point information table records the basic information and deployment parameters of the monitoring points. The attributes include the project where it is located, the measuring point number, the installation location coordinates, and the installation drawing. The project where it is located describes the name of the engineering project to which the measuring point belongs (foreign key, associated with the project management system);

[0118] The measuring point number is the unique identifier of the table (primary key); the installation location coordinates describe the longitude and latitude or three-dimensional coordinates, which are used to locate the monitoring point; the installation drawing describes the storage path or file link of the measuring point layout drawing. The measuring point information table is associated with the raw crack acquisition image table through the "measuring point number" to ensure that the image data corresponds one by one with the physical measuring points.

[0119] The raw crack acquisition image table stores the unprocessed raw crack image data. The attributes include the image number, the image format type, the image size, and the storage path. The image number is the unique identifier of the table (primary key); the image format type is such as JPG, PNG, etc.; the image size describes the resolution and the storage volume; the storage path describes the physical address of the image in the server or cloud storage. The raw crack acquisition image table is associated with the preprocessed crack image table through the "image number" to form a link between the raw data and the preprocessed data.

[0120] The preprocessed crack image table stores the preprocessed intermediate image files to support subsequent model analysis. The attributes include the grayscale file, the threshold segmentation result file, and the morphological processing file. The grayscale file describes the image path after grayscale processing; the threshold segmentation result file describes the binary image path based on threshold segmentation; the morphological processing file describes the image path after morphological operations (such as dilation, erosion). The preprocessed crack image table establishes a one-to-one relationship with the raw crack acquisition image table through the "image number" to ensure data traceability.

[0121] Furthermore, the recognition model library manages the crack recognition model and its training process. Its data tables include a model information table and a training information table. The attributes of the model information table include model type, model number, recognition accuracy, and computing power requirement value. Model types such as convolutional neural network (CNN), support vector machine (SVM), etc.; the model number is the unique identifier (primary key) of the table; the recognition accuracy describes the accuracy rate of the model on the validation set; the computing power requirement value describes the computing resources required to run the model (such as GPU video memory, CPU core count).

[0122] The attributes of the training information table include training sample capacity, training model number, and training history information. The training sample capacity describes the number of training set images; the training model number is used as a foreign key to associate with the "model information table"; the training history information describes logs such as training time, loss function curve, hyperparameter configuration, etc. The training information table realizes the dynamic binding of the model and training records through the "model number", facilitating version management and performance comparison.

[0123] Furthermore, the data tables in the processing result library store the crack types identified by the model and the calculation results of the dimensions, including a crack recognition result table and a calculation result table.

[0124] The attributes of the crack recognition result table include crack type and recognition confidence value. The crack type describes the classification result (horizontal, vertical, reticular, etc.); the recognition confidence value describes the probability value of the model output result.

[0125] The attributes of the calculation result table include crack length value, crack width value, length increment, and width increment. The crack length value is in millimeters (mm); the crack width value is in millimeters (mm); the length increment describes the length change amount compared with historical data;

[0126] The width increment describes the width change amount compared with historical data. The calculation result table is associated with the original data table through the "image number" or "measurement point number", supporting trend analysis in the spatio-temporal dimension.

[0127] The overall architecture of the database and the working process of the data flow: First, at the data acquisition layer, the original images are obtained through measurement point deployment and stored in the "crack original acquisition image table"; second, at the preprocessing layer, the original images are grayscaled, segmented, etc., and intermediate files are generated and stored in the "preprocessed crack image table"; third, at the model processing layer, the algorithm model in the "model information table" is called, and the parameters are optimized in combination with the training history to output the recognition result; then, at the result storage layer, the classification and quantification results are written into the "crack recognition result table" and the "calculation result table"; finally, at the application layer, a monitoring report is generated based on the calculation results, and the early warning mechanism is triggered.

[0128] The database of this system has the following key design features: (1) Through primary keys (such as measurement point numbers, image numbers) and foreign key constraints, data relevance and consistency are ensured, guaranteeing data integrity; (2) Each table is designed independently. When adding new model types or monitoring items, there is no need to reconstruct the overall structure, achieving modular expansion; (3) Indexes are established for fields such as "crack type" and "confidence value" to achieve efficient querying and improve analysis efficiency; (4) From the original image to the final calculation result, layer-by-layer backtracking and problem location are supported, realizing full-link traceability of data.

[0129] As Figure 2 shown is the overall working process of this system, including:

[0130] First, determine the positions of the crack monitoring points on the surface of the building to be measured, and install the integrated front-end low-power machine vision image acquisition and data wireless transmission equipment of this system. The front-end low-power machine vision collector is used to collect high-definition crack images. The main control module controls the wireless transmission module to transmit the collected crack image data to the intelligent processing center of the cloud platform through the wireless mobile network. The intelligent processing center of the cloud platform preprocesses and segments the crack images, calls the built-in switchable deep learning crack recognition model for recognition, and calculates the width and length of the cracks. The recognition results and the calculation results of the width and length of the cracks are stored in the database of the cloud platform. At the same time, an original sample library is established for the recognition model to perform secondary training. Finally, the web server of the cloud platform visualizes the crack recognition results.

[0131] The embodiments described above are only descriptions of the preferred embodiments of the present invention, and do not limit the scope of the present invention. Without departing from the design spirit of the present invention, various deformations and improvements made by those of ordinary skill in the art to the technical solutions of the present invention shall fall within the protection scope determined by the claims of the present invention.

Claims

1. An intelligent crack monitoring system based on machine vision and cloud platform, characterized in that: include: Front-end acquisition and transmission devices and cloud platforms; The front-end acquisition and transmission device is used to acquire crack images of the target area by using the adaptive fill light technology, and encrypt and transmit the crack images to the cloud platform via the mobile communication network; The cloud platform is used to preprocess the crack image, segment the image, identify and calculate the width and length of the crack, obtain the data after intelligent processing, and store the data after intelligent processing in the cloud server and database, wherein image recognition includes calling a pre-trained recognition model to identify the crack image according to user preset conditions and crack image quality.

2. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 1 is characterized in that: The front-end acquisition and transmission device includes: an image acquisition module, an LED autonomous control fill light module, an MCU main control module, a 4G-Cat1 transmission module, and a micro-power lithium battery power module; The image acquisition module is used to carry a high-resolution camera and a macro lens, and to combine the macro lens with the adaptive fill light technology of the LED autonomous control fill light module to acquire the crack image; The LED self-controlled fill light module is used to automatically adjust the LED brightness according to the ambient light intensity and eliminate shadow interference through diffuse reflection fill light; The MCU main control module is used to control the image acquisition module, the LED autonomous control fill light module, the 4G-Cat1 transmission module, and the micro-power lithium battery power module, and to eliminate image noise and interference signals from the crack image, and to compress the denoised crack image and send it to the 4G-Cat1 transmission module; The 4G-Cat1 transmission module is used to transmit the collected crack image to the cloud platform via the network using the TCP / IP protocol; The micro-power consumption lithium battery power module is used to combine the power supply with the low-power consumption power management unit to power the image acquisition module, the LED autonomous control fill light module, the MCU main control module, and the 4G-Cat1 transmission module.

3. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 1 is characterized in that: The cloud platform includes: an image preprocessing module, a crack image segmentation and recognition module, a crack image library, a recognition result library and a recognition model library; The image preprocessing module is used to perform grayscale conversion, threshold segmentation and morphological processing on the crack image to obtain the crack image after preprocessing; The crack image segmentation and recognition module is used to extract the crack skeleton according to the threshold segmentation algorithm to obtain a binary image, calculate the crack width and length, and call the pre-trained recognition model from the recognition model library to recognize the pre-processed crack image, obtain the crack recognition result, and store the recognition result and the calculated value in the recognition result library; The crack image library is used to store crack image samples. When the number of newly added crack image samples exceeds a preset value, the newly added crack image samples are extracted to train the model in the recognition model library; The recognition result library is used to store recognition results and calculation results; The recognition model library is used to store pre-trained recognition models.

4. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 3 is characterized in that: Calculation of crack width and length includes: Acquire the area of ​​the crack according to the binary image, measure the length of the crack by using the centerline method according to the area of ​​the crack, and calculate the centerline length as the length of the crack by the centerline position of the target crack; The average width of the crack is calculated according to the length of the crack and the distance between two adjacent center points, and the minimum width and the maximum width of the crack are obtained according to the binary image.

5. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 3 is characterized in that: Obtaining the minimum width and the maximum width of the crack according to the binary image includes: Acquire the outline of the crack according to the binary image; According to the contours of the crack, a minimum circumscribed matrix of each contour is calculated; Rotating the outline of the crack until the outline is parallel to the width direction of the minimum circumscribed matrix; The number of non-zero pixels of the rotated contour along the width direction is calculated to obtain the minimum width and the maximum width of the crack.

6. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 3 is characterized in that: The crack image library includes: a measuring point information table, a crack original acquisition image table and a pre-processed crack image table; The measuring point information table records the basic information and deployment parameters of the monitoring point, and the attributes include the project, measuring point number, installation location coordinates, and installation drawings; The crack original acquisition image table stores unprocessed original crack image data, and the attributes include image number, image format type, image size and storage path; The pre-processed crack image table stores the pre-processed intermediate image file, and the attributes include grayscale file, threshold segmentation result file, and morphological processing file.

7. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 3 is characterized in that: The identification result library includes a crack identification result table and a calculation result table; The attributes of the crack identification result table include crack type and identification confidence value; The attributes of the calculation result table include crack length value, crack width value, length increment, and width increment.

8. The intelligent crack monitoring system based on machine vision and cloud platform according to claim 3 is characterized in that: The recognition model library includes: a model information table and a training information table; The attributes of the model information table include model type, model number, recognition accuracy, and computing power requirement value; The attributes of the training information table include training sample capacity, training model number, and training history information.

Images