Bridge construction quality inspection method, system and platform supported by machine vision

Abstract

The present invention relates to a bridge construction quality inspection method, system and platform supported by machine vision, and relates to the field of quality inspection, including: performing first-order crack recognition to obtain a target area; collecting an image of the target area, performing image significant area recognition, obtaining image significance and image significant area, and dividing the significant area image of the image significant area; configuring the number of recognition nodes according to the image significance, performing second-order crack recognition on the significant area image, and obtaining crack feature information; configuring a bridge deformation detection range according to the crack feature information and image significance to perform deformation detection, obtain deformation detection results, and combine the crack feature information as a bridge quality inspection result. The present invention can solve the technical problems that the traditional bridge construction quality inspection method cannot efficiently identify bridge crack defects and deformation defects, has a high probability of missed detection, low detection accuracy and low detection efficiency; and can significantly improve the efficiency, accuracy and reliability of bridge quality inspection.

Description

Technical Field

[0001] The present invention relates to the field of quality inspection, and in particular to a method, system and platform for bridge construction quality inspection supported by machine vision. Background Art

[0002] During the bridge construction process, construction quality inspection is a key link to ensure the safety, durability and stability of the bridge structure. As bridge structures become increasingly complex, defects such as cracks and deformations often appear at different construction stages, and these defects may exist in hidden parts of the bridge (such as the inside of the beam, under the bridge deck, or in hard-to-reach corners such as the pier base), making them difficult to detect.

[0003] At present, the traditional bridge quality inspection method mainly relies on visual inspection and basic manual measurement tools to identify cracks and deformations. It cannot efficiently identify bridge crack defects and deformation defects. It has technical problems such as high probability of missed detection, low detection accuracy and low detection efficiency, and cannot meet the quality inspection needs of modern bridge construction. Summary of the invention

[0004] The present invention aims to solve the technical problems that traditional bridge construction quality inspection methods cannot efficiently identify bridge crack defects and deformation defects, and have a high probability of missed detection, low detection accuracy and low detection efficiency. A machine vision-supported bridge construction quality inspection method, system and platform are provided to solve the problems.

[0005] The technical solution of the present invention to solve the above technical problems is as follows:

[0006] In a first aspect, the present invention provides a bridge construction quality inspection method supported by machine vision, comprising: acquiring a surface image of a target bridge through an image acquisition device, performing first-order crack recognition through machine vision to obtain a target area; locating and acquiring a target area image of the target area through the image acquisition device, performing image significant area recognition to obtain image significance and image significant area, and dividing the significant area image of the image significant area; configuring the number of recognition nodes for crack recognition on the significant area image according to the image significance, performing second-order crack recognition on the significant area image through machine vision to obtain crack feature information; configuring a bridge deformation detection range according to the crack feature information and image significance, performing deformation detection on the bridge deformation detection range adjacent to the target area, obtaining a deformation detection result, and combining the crack feature information as a bridge quality inspection result.

[0007] In a second aspect, the present invention provides a bridge construction quality inspection system supported by machine vision, including: a first-order crack recognition module, which is used to capture the surface image of the target bridge through an image acquisition device, perform first-order crack recognition through machine vision, and obtain a target area; an image significant area recognition module, which is used to locate and capture the target area image of the target area through the image acquisition device, perform image significant area recognition, obtain image significance and image significant area, and divide the significant area image of the image significant area; a second-order crack recognition module, which is used to configure the number of recognition nodes for crack recognition on the significant area image according to the image significance, perform second-order crack recognition on the significant area image through machine vision, and obtain crack feature information; a bridge deformation detection module, which is used to configure the bridge deformation detection range according to the crack feature information and image significance, perform deformation detection on the bridge deformation detection range adjacent to the target area, obtain deformation detection results, and use them in combination with the crack feature information as bridge quality inspection results.

[0008] In a third aspect, the present invention also provides a machine vision-supported bridge construction quality inspection platform, comprising: at least one processor; a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the steps of the method in the first aspect above.

[0009] In a fourth aspect, a computer-readable storage medium is provided, wherein a computer program is stored on the computer-readable storage medium, and when the computer program is executed, the steps of the method in the first aspect are implemented.

[0010] The beneficial effects of the present invention are as follows: the surface image of the target bridge is acquired by an image acquisition device, and first-order crack recognition is performed by machine vision to obtain the target area; then, the target area image of the target area is located and acquired by the image acquisition device, and image significant area recognition is performed to obtain image significance and image significant area, and the significant area image of the image significant area is divided; further, according to the image significance, the number of recognition nodes for crack recognition of the significant area image is configured, and second-order crack recognition is performed on the significant area image by machine vision to obtain crack feature information; on the other hand, according to the crack feature information and image significance, the bridge deformation detection range is configured, and deformation detection is performed on the bridge deformation detection range adjacent to the target area to obtain deformation detection results; finally, the deformation detection results and crack feature information are used as bridge quality detection results; that is, by integrating machine vision technology, image significance analysis and intelligent recognition node configuration, automatic and accurate detection of bridge crack defects can be achieved, the probability of missed detection can be effectively reduced, and the crack detection accuracy can be improved; at the same time, the deformation detection range can be intelligently configured based on crack features, which can reduce the blindness of deformation detection, reduce unnecessary detection, and improve detection efficiency, thereby significantly improving the efficiency, accuracy and reliability of bridge quality detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] Figure 1 A schematic diagram of a flow chart of a bridge construction quality inspection method supported by machine vision provided by the present invention;

[0012] Figure 2 A schematic diagram of the structure of a bridge construction quality inspection system supported by machine vision provided by the present invention;

[0013] Figure 3 A schematic diagram of the structure of a computer-readable storage medium provided by the present invention.

[0014] In the accompanying drawings, the components represented by the reference numerals are described as follows:

[0015] A first-order crack recognition module 01 , an image salient region recognition module 02 , a second-order crack recognition module 03 , a bridge deformation detection module 04 , a computer-readable storage medium 600 , and a second computer program 611 . DETAILED DESCRIPTION

[0016] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative work are within the scope of protection of the present invention.

[0017] In the description of the present invention, the terms "first" and "second" are used for descriptive purposes only and should not be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined as "first" and "second" may explicitly or implicitly include one or more of the features. In the description of the present invention, the meaning of "plurality" is two or more, unless otherwise clearly and specifically defined.

[0018] In the description of the present invention, the term "for example" is used to mean "used as an example, illustration or explanation". Any embodiment described as "for example" in the present invention is not necessarily to be construed as being more preferred or advantageous than other embodiments. The following description is given to enable any technician in the field to implement and use the present invention. In the following description, details are listed for the purpose of explanation. It should be understood that a person of ordinary skill in the art can recognize that the present invention can be implemented without using these specific details. In other examples, well-known structures and processes will not be elaborated in detail to avoid obscuring the description of the present invention with unnecessary details. Therefore, the present invention is not intended to be limited to the embodiments shown, but is consistent with the widest scope consistent with the principles and features disclosed in the present invention.

[0019] Embodiment 1, as Figure 1 As shown, the embodiment of the present invention provides a bridge construction quality detection method supported by machine vision, which specifically includes the following steps:

[0020] S100: The surface image of the target bridge is collected by an image acquisition device, and first-order crack recognition is performed by machine vision to obtain a target area.

[0021] Furthermore, step S100 of the present invention further includes:

[0022] S110: Collect a set of sample surface images of similar bridges, and identify the crack area in each sample surface image to obtain a set of sample crack areas; S120: Based on the convolutional neural network in machine vision, use the sample surface image set and the sample crack area set as first-order machine vision training data to train a crack area identifier; S130: Collect the surface image of the target bridge through an image acquisition device; S140: Input the surface image into the crack area identifier, identify and output the crack area as the target area.

[0023] Specifically, first, query the bridge construction records, select multiple bridge construction or built samples, and collect a set of sample surface images of similar bridges; then, accurately identify the crack area in each surface image through manual or semi-automatic labeling to obtain a set of sample crack areas, in which the sample surface images and sample crack areas correspond one to one.

[0024] Convolutional neural network is a deep learning model, which is particularly good at processing image data. Using convolutional neural network to build a crack area identifier can improve the detection accuracy and automation level of crack defects. Then, a crack area identifier is built based on the convolutional neural network in machine vision. The crack area identifier is used to accurately identify and locate the crack area in the bridge surface image in an automated manner, including an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer. The input data of the input layer is the bridge surface image, and the output data of the output layer is the crack area. The convolution layer is used to perform convolution operations on the input image using multiple convolution kernels to extract local features (such as edges, textures, etc.) in the image; the pooling layer is used to reduce the size of the feature map while maintaining important information in the image; the fully connected layer is used to integrate the extracted features into a one-dimensional vector to obtain the final output.

[0025] Furthermore, the sample surface image is used as input, the sample crack area is used as supervision, and the sample surface image set and the sample crack area set are used as first-order machine vision training data to supervise the crack area identifier. First, the sample surface image is input, and forward propagation is performed through each layer of the network to calculate the output result (i.e., the crack area); then the error between the predicted value of the model and the actual labeled value is calculated by the loss function; the back propagation algorithm is further used to update the parameters (weight and bias) of the network according to the loss value, and the training error is reduced by the optimization algorithm (such as Adam or SGD); then the training data is used for multiple iterations of training, and the network weight is optimized each time to continuously improve the accuracy of the model until the expected convergence index is met (i.e., the loss function is less than a predetermined threshold or reaches a predetermined number of training times), and a crack area identifier that has been trained is obtained. By constructing a crack area identifier based on a convolutional neural network, the crack area of ​​the bridge can be identified efficiently and accurately, the probability of missed detection can be reduced, and the efficiency and accuracy of crack area detection can be improved.

[0026] Then, a high-resolution image acquisition device (such as a digital camera, a high-definition camera or a drone, etc.) is used to obtain the surface image of the target bridge. These images should cover various parts of the bridge (such as the bridge deck, the bridge bottom, the beam body, etc.) and be able to capture various features of the cracks in detail to obtain the surface image of the target bridge; then, the surface image is input into the crack area identifier for identification, the crack area is output, and the crack area is used as the target area.

[0027] S200: Using the image acquisition device, locate and acquire a target area image of the target area, perform image salient area recognition, obtain image saliency and image salient areas, and divide salient area images of the image salient areas.

[0028] Furthermore, step S200 of the present invention further includes:

[0029] S210: Perform pixel saliency analysis on all pixels in the target area image to obtain all pixel saliencies.

[0030] Furthermore, step S210 of the present invention further includes:

[0031] S211: grayscale the target area image to obtain a target area grayscale image; S212: select a first grayscale value of a first pixel in the target area grayscale image, extract multiple first neighboring grayscale values ​​of multiple first neighboring pixels of the first pixel, and randomly extract multiple first random grayscale values ​​in the target area grayscale image; S213: calculate the deviation ratio between the mean of the multiple first neighboring grayscale values ​​and the mean of the multiple first random grayscale values ​​as the first pixel significance of the first pixel; S214: continue calculating and processing to obtain all pixel significances of all pixel points in the target area image.

[0032] Specifically, firstly, the image acquisition device is used to locate and acquire a high-resolution image of the target area to ensure that minute details of the crack can be captured, especially when the crack width or depth is small, as the target area image.

[0033] In the image processing process, grayscale processing is the process of converting a color image into a grayscale image. Especially for crack detection, grayscale images can simplify the subsequent feature extraction and analysis process and improve detection efficiency and accuracy. Grayscale images usually have only one channel, and the pixel value is between 0 and 255, where 0 represents black and 255 represents white. Next, the target area image is grayscaled, such as using a weighted average method to process the image grayscale, that is, the color information of the three channels of red, green, and blue (RGB) of each pixel of the image is weighted according to a certain weight to obtain a grayscale value, and obtain a grayscale image of the target area. After grayscale processing, the color information in the image is converted into brightness information, making the crack area (usually in obvious contrast with the surrounding area) more prominent, which is convenient for subsequent crack detection; at the same time, after being converted into a grayscale image, the amount of information in the image is reduced, and the amount of calculation is greatly reduced.

[0034] Then, randomly select any pixel point in the grayscale image of the target area as the first pixel point, and obtain the first grayscale value of the first pixel point; then, extract multiple first neighboring grayscale values ​​of multiple first neighboring pixel points of the first pixel point, where the neighboring pixel points refer to the neighboring pixels around the first pixel point, and are usually selected in a local area manner, such as a 3×3, 5×5 or 7×7 neighborhood; and randomly extract the grayscale values ​​of multiple equal numbers of pixels in the grayscale image of the target area to obtain multiple first random grayscale values.

[0035] Next, the mean of the multiple first neighboring grayscale values ​​is calculated to obtain a first neighboring grayscale mean; the mean of the multiple first random grayscale values ​​is calculated to obtain a first random grayscale mean; then the ratio of the first neighboring grayscale mean to the first random grayscale mean is calculated as the deviation ratio, and the deviation ratio is set as the first pixel significance of the first pixel point, wherein cracks usually appear as darker areas, whose grayscale values ​​are significantly lower than the grayscale values ​​of the surrounding normal areas, while the grayscale values ​​of the concrete surface area are usually higher and do not change much, so the grayscale difference between it and the neighboring areas is small; when the deviation ratio is large, it means that the change in the grayscale value of the area is more significant, and it may be a crack area.

[0036] The same method for calculating the first pixel significance is further used to continue calculating the pixel significance of other pixels in the target area image to obtain the total pixel significance of all pixels. By calculating the pixel significance, the significance of each pixel in the image can be comprehensively evaluated, thereby providing more accurate data support for the automatic identification of crack areas.

[0037] S220: Filter the pixel prominences greater than the prominence threshold among all the pixel prominences to obtain multiple sensitive pixel prominences, and mark the pixel points corresponding to the multiple sensitive pixel prominences as multiple sensitive pixel points; S230: Calculate the sum of the multiple sensitive pixel points to obtain image prominence; S240: Select the multiple sensitive pixel points in the target area image according to a preset anchor frame shape to obtain an image salient area; S250: Divide the image of the image salient area in the target area image to obtain a salient area image.

[0038] Specifically, first, a significance threshold is set according to the training data and image features. The threshold is used to filter out pixels with higher significance. Usually, these pixels represent the parts with larger changes in the image and may belong to the crack area. Then, the significance of all pixels is judged according to the significance threshold, and the significance of pixels with significance greater than the significance threshold is set as the sensitive pixel significance, and multiple sensitive pixel significances are obtained. The pixels corresponding to the sensitive pixel significance are marked as sensitive pixels. These pixels are usually located in the crack area or possible defect area in the image, and multiple sensitive pixels are obtained. Then, the multiple sensitive pixels are summed to obtain the number of sensitive pixels, which is set as the image significance. The image significance can reflect the significance of cracks in the entire target area. Generally speaking, images with higher significance may contain larger cracks or structural defects.

[0039] Then, a preset anchor frame shape is obtained. The preset anchor frame shape is a preset rectangular area, which is used to select the sensitive pixels in the target area. In this solution, the anchor frame shape is a rectangle, that is, a minimum rectangular frame is selected to surround all sensitive pixels. The size and position of the anchor frame can be dynamically adjusted according to the distribution of sensitive pixels to ensure that all sensitive pixels are included. Further, according to the preset anchor frame shape, the multiple sensitive pixels in the target area image are selected, that is, according to the coordinates of all sensitive pixels, a rectangular frame is calculated, and the rectangular frame can completely contain all sensitive pixels. According to the calculated boundary value, a rectangular frame is generated, and the rectangular frame is used as the salient area of ​​the image, which contains the area where all sensitive pixels are located, and the image salient area is obtained. Next, the image of the salient area of ​​the image in the target area image is divided, that is, the salient area is extracted separately, and the part in the rectangular frame is cropped from the target area image through the boundary of the salient area to obtain the salient area image.

[0040] By selecting and dividing the salient areas in the target area image, the sensitive parts related to the cracks can be extracted to form a salient area image, thereby providing a focused area for subsequent crack identification, which helps to improve the efficiency and accuracy of crack detection.

[0041] S300: configuring the number of recognition nodes for crack recognition on the image in the significant region according to the image saliency, performing second-order crack recognition on the image in the significant region through machine vision, and obtaining crack feature information.

[0042] Furthermore, step S300 of the present invention further includes:

[0043] S310: training a second-order crack identification channel, wherein the second-order crack identification channel includes second-order crack identification nodes having a total number of nodes.

[0044] Furthermore, step S310 of the present invention further includes:

[0045] S311: According to the significant area image processing records in the historical time, a sample significant area image set is collected, and the crack width and crack length in each sample significant area image are marked to obtain a sample crack feature information set; S312: According to a preset division ratio, the first crack feature recognition training data is randomly divided with replacement in the sample significant area image set and the sample crack feature information set, and the first and second order crack recognition nodes are trained until convergence; S313: Continue to divide the crack feature recognition training data according to the preset division ratio, with replacement in the sample significant area image set and the sample crack feature information set, and train the second order crack recognition nodes until the training obtains the second order crack recognition nodes of the total number of nodes that converge; S314: Integrate the second order crack recognition nodes of the total number of nodes to obtain a second order crack recognition channel.

[0046] Specifically, first, representative images are selected from the historical records using the previously processed salient area image records to construct a sample salient area image set, where each image contains salient images of a specific construction stage or a specific defect area. Next, for each sample salient area image, the width and length of the cracks are manually annotated. The crack width refers to the maximum width of the cracks in the salient area, which usually represents the width of the pixels occupied by the cracks in the image; the crack length refers to the maximum length of the cracks in the salient area, which represents the distance from the beginning to the end of the crack, usually the maximum span of the crack along the upper or lower edge of the image; by annotating the cracks in each sample salient area image, the corresponding crack width, length and other feature information are obtained to form sample crack feature information, and a sample crack feature information set is obtained.

[0047] Obtain a preset division ratio of the sample data, and the preset division ratio can be set according to the actual amount of sample data, for example, the sample data is divided into 10 equal parts; then, the sample salient area image set and the sample crack feature information set are used as sample data sets, and the sample data sets are divided according to the preset division ratio to obtain N data sets, wherein N is the total number of preset division ratios, such as 10; further, N times of selection are made from the N data sets with replacement to construct the first crack feature recognition training data.

[0048] Next, a first and second order crack recognition node is constructed based on a convolutional neural network. The first and second order crack recognition node includes an input layer, a convolution layer, a pooling layer, a fully connected layer and an output layer, wherein the input data of the input layer is a significant area image, and the output data of the output layer is crack feature information (crack width and crack length); then, the sample significant area image is used as input, the sample crack feature information is used as supervision, and the first crack feature recognition training data is used to perform supervised training on the first and second order crack recognition nodes. First, the training sample (significant area image) is input into the network, and after being processed by the convolution layer, the pooling layer and the fully connected layer, the output of the network (predicted crack width and length) is finally obtained; then, a loss function (such as mean square error MSE) is used to calculate the gap between the predicted result and the actual label, and the gradient of each layer is calculated by the back propagation algorithm, and the weight of the network is updated to minimize the loss function; the above process is repeated, and the network weight is gradually updated until the loss function converges or reaches a predetermined number of iterations, and the first and second order crack recognition nodes that have been trained are obtained.

[0049] Using the same method as training the first second-order crack recognition node, continue to randomly divide the crack feature recognition training data in the sample significant area image set and the sample crack feature information set according to the preset division ratio, and train other second-order crack recognition nodes until the training obtains the second-order crack recognition nodes with the total number of nodes that converge, wherein the number of summary points can be set according to actual needs, such as setting the number of summary points to 10. Finally, the second-order crack recognition nodes with the total number of nodes are integrated in parallel, that is, all the second-order crack recognition nodes process the input significant area image at the same time, and finally merge the output results of each node (such as by weighted average, maximum voting, etc.) to obtain a second-order crack recognition channel. Through the integrated multiple second-order crack recognition nodes, a channel for processing more complex and diversified crack features is formed, and the output of each node will provide the width and length features of the crack. The second-order recognition channel finally outputs more accurate crack information by fusing these features.

[0050] S320: Obtain the maximum image significance based on the crack image recognition data in the historical time; S330: Calculate the ratio of the image significance and the maximum image significance, multiply it by the total number of nodes and round it to obtain the number of recognition nodes; S340: Randomly select the second-order crack recognition nodes of the recognition node number from the second-order crack recognition nodes of the total number of nodes, perform crack recognition on the significant area image respectively, and obtain the node crack feature information of the recognition node number, wherein each node crack feature information includes the crack width and the crack length; S350: Calculate the mean of the node crack feature information of the recognition node number to obtain the crack feature information.

[0051] Specifically, according to the crack image recognition data in the historical time, the maximum image saliency is obtained, and the maximum image saliency represents the most significant crack area feature in the past; then the ratio of the image saliency and the maximum image saliency is calculated to obtain the saliency ratio, which reflects the strength of the saliency of the current image relative to the historical maximum saliency, and the saliency ratio is multiplied by the total number of nodes and rounded to obtain the number of recognition nodes. By configuring the number of recognition nodes according to the image saliency, the adaptability of the number of nodes to the image state can be improved, and the computing resources can be reasonably configured to ensure the efficient use of computing resources. Through this adjustment mechanism, when the image saliency is high, the system will automatically increase the number of recognition nodes, thereby improving the precision and accuracy of the detection; when the image saliency is low, the number of nodes is reduced to save computing resources.

[0052] Next, randomly select the second-order crack identification nodes of the identification node number from the second-order crack identification nodes of the total node number. For example, assuming that the total node number is 10 and the identification node number is 5, randomly select 5 second-order crack identification nodes from the 10 second-order crack identification nodes. Then, the cracks of the salient region image are respectively identified through the second-order crack identification nodes of the identification node number to obtain the node crack feature information of the identification node number, wherein each node crack feature information includes crack width and crack length. Finally, the node crack feature information of the identification node number is averaged to obtain crack feature information (including crack width mean and crack length mean).

[0053] S400: According to the crack feature information and image prominence, a bridge deformation detection range is configured, deformation detection is performed on the bridge deformation detection range adjacent to the target area, and a deformation detection result is obtained, which is combined with the crack feature information as a bridge quality detection result.

[0054] Furthermore, step S400 of the present invention further includes:

[0055] S410: Obtain crack feature information threshold, wherein the crack feature information threshold includes a crack width threshold and a crack length threshold; S420: Calculate the ratio of the crack width and crack length in the crack feature information to the crack width threshold and crack length threshold respectively, and calculate the mean to obtain a crack scale coefficient; S430: Obtain the average image prominence of the crack area image formed by the crack of the crack feature information threshold; S440: Calculate the ratio of the image prominence and the average image prominence to obtain a significant scale coefficient; S450: Calculate the mean of the crack scale coefficient and the significant scale coefficient to obtain a potential deformation scale coefficient; S460: Obtain a preset deformation detection range, and multiply the potential deformation scale coefficient by the preset deformation detection range to obtain a bridge deformation detection range; S470: Perform deformation detection on the bridge deformation detection range adjacent to the target area to obtain a deformation detection result, and combine it with the crack feature information as a bridge quality detection result.

[0056] Specifically, first, the crack characteristic information threshold is obtained, wherein the crack characteristic information threshold includes a crack width threshold and a crack length threshold, wherein the crack width threshold and the crack length threshold are set according to historical data or engineering standards, and represent the maximum width (such as 1 mm) and maximum length (such as 5 mm) of the crack that are considered acceptable during the construction process. Next, the ratio of the crack width in the crack characteristic information to the crack width threshold is calculated to obtain the crack width ratio; the ratio of the crack length in the crack characteristic information to the crack length threshold is calculated to obtain the crack length ratio, and the crack width ratio and the crack length ratio are averaged to obtain the crack scale coefficient, wherein the crack scale coefficient represents the relative severity of the crack, and the larger the value, the larger the crack, which may have a greater impact on the structural safety of the bridge.

[0057] Next, the average image prominence of the crack region image formed by the cracks of the crack feature information threshold is obtained. The average image prominence represents the average prominence of all pixels in the crack region image, and is used to measure the prominence of the crack in the image. Then, the ratio of the image prominence to the average image prominence is calculated, and the ratio is set as the prominence scale coefficient, wherein the prominence scale coefficient is used to measure the relative size of the prominence of the current crack region relative to the reference prominence. The larger the ratio, the stronger the prominence of the current crack region, which may indicate that the crack is more serious or more prominent.

[0058] The crack scale coefficient and the significant scale coefficient are further averaged, and the coefficient average is set as the potential deformation scale coefficient, where the potential deformation scale coefficient is a deformation potential indicator after comprehensively considering the crack scale and the significant scale, and is used to evaluate the potential structural deformation scale caused by the crack. The potential deformation scale coefficient reflects the size of the crack and the degree of its influence in the structure. A higher value indicates that the crack has a higher potential deformation risk, which may cause structural deformation.

[0059] Then, a preset deformation detection range is obtained, and the preset deformation detection range can be set according to the actual scenario (such as a fixed area size or a maximum detectable range); then, the potential deformation scale coefficient is multiplied by the preset deformation detection range to obtain the bridge deformation detection range, that is, when the potential deformation scale coefficient is large, the deformation detection range will be expanded accordingly to ensure that the deformation can be fully detected, which can effectively avoid local missed detection; when the potential deformation scale coefficient is small, the deformation detection range will be reduced accordingly to concentrate resources for more detailed detection, thereby improving the detection efficiency. This method of dynamically adjusting the deformation detection range can intelligently configure the detection area according to the scale and significance of the cracks, avoiding the waste of resources and inefficiency caused by comprehensive detection of all areas in the traditional method.

[0060] Finally, deformation detection is performed on the deformation detection range of the bridge adjacent to the target area. Common detection methods include displacement sensors, laser scanning, structural health monitoring systems, etc., to obtain deformation detection results; and the deformation detection results and crack feature information are used as bridge quality detection results.

[0061] The machine vision-supported bridge construction quality inspection method provided in the embodiment of the present invention has at least the following technical effects:

[0062] By integrating machine vision technology, image significance analysis and intelligent recognition node configuration, automatic and accurate detection of bridge crack defects can be achieved, effectively reducing the probability of missed detection and improving crack detection accuracy. At the same time, the deformation detection range can be intelligently configured based on crack characteristics, which can reduce the blindness of deformation detection, reduce unnecessary detection, and improve detection efficiency, thereby significantly improving the efficiency, accuracy and reliability of bridge quality inspection.

[0063] Embodiment 2, as Figure 2 As shown, based on the same inventive concept of the machine vision-supported bridge construction quality inspection method provided in Embodiment 1, the embodiment of the present invention also provides a machine vision-supported bridge construction quality inspection system, including:

[0064] The first-order crack recognition module 01 is used to capture the surface image of the target bridge through an image acquisition device, perform first-order crack recognition through machine vision, and obtain the target area; the image significant area recognition module 02 is used to locate and capture the target area image of the target area through the image acquisition device, perform image significant area recognition, obtain image significance and image significant area, and divide the significant area image of the image significant area; the second-order crack recognition module 03 is used to configure the number of recognition nodes for crack recognition on the significant area image according to the image significance, perform second-order crack recognition on the significant area image through machine vision, and obtain crack feature information; the bridge deformation detection module 04 is used to configure the bridge deformation detection range according to the crack feature information and image significance, perform deformation detection on the bridge deformation detection range adjacent to the target area, obtain deformation detection results, and combine the crack feature information as the bridge quality detection result.

[0065] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: collecting a set of sample surface images of similar bridges, and identifying the crack area in each sample surface image to obtain a set of sample crack areas; based on the convolutional neural network in the machine vision, using the sample surface image set and the sample crack area set as first-order machine vision training data to train a crack area identifier; through an image acquisition device, collecting a surface image of the target bridge; inputting the surface image into the crack area identifier, identifying and outputting the crack area to obtain the target area.

[0066] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: performing pixel significance analysis on all pixels in the target area image to obtain all pixel significances; screening pixel significances greater than a significance threshold in all pixel significances to obtain multiple sensitive pixel significances, and marking the pixel points corresponding to the multiple sensitive pixel significances as multiple sensitive pixel points; calculating the sum of the multiple sensitive pixel points to obtain image significance; selecting the multiple sensitive pixel points in the target area image according to a preset anchor frame shape to obtain an image salient area; dividing the image of the image salient area in the target area image to obtain a salient area image.

[0067] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: grayscale processing the target area image to obtain a target area grayscale image; selecting a first grayscale value of a first pixel in the target area grayscale image, extracting multiple first neighboring grayscale values ​​of multiple first neighboring pixels of the first pixel, and randomly extracting multiple first random grayscale values ​​in the target area grayscale image; calculating the deviation ratio between the mean of the multiple first neighboring grayscale values ​​and the mean of the multiple first random grayscale values ​​as the first pixel significance of the first pixel; and continuing the calculation and processing to obtain all pixel significances of all pixels in the target area image.

[0068] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: training a second-order crack recognition channel, wherein the second-order crack recognition channel includes second-order crack recognition nodes of a total number of nodes; obtaining a maximum image significance based on crack image recognition data in a historical time period; calculating a ratio of the image significance to the maximum image significance, multiplying the ratio by the total number of nodes and rounding the ratio to obtain the number of recognition nodes; randomly selecting second-order crack recognition nodes of the number of recognition nodes from the second-order crack recognition nodes of the total number of nodes, performing crack recognition on the significant area image respectively, and obtaining node crack feature information of the number of recognition nodes, wherein each node crack feature information includes a crack width and a crack length; calculating the mean of the node crack feature information of the number of recognition nodes to obtain crack feature information.

[0069] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: collecting a set of sample significant area images based on the significant area image processing records in the historical time, and marking the crack width and crack length in each sample significant area image to obtain a sample crack feature information set; randomly dividing the first crack feature recognition training data in the sample significant area image set and the sample crack feature information set with replacement according to a preset division ratio, and training the first and second-order crack recognition nodes until convergence; continuing to randomly divide the crack feature recognition training data in the sample significant area image set and the sample crack feature information set according to the preset division ratio with replacement, and training the second-order crack recognition nodes until the training obtains the second-order crack recognition nodes with the total number of nodes that converge; integrating the second-order crack recognition nodes with the total number of nodes to obtain a second-order crack recognition channel.

[0070] Furthermore, the machine vision-supported bridge construction quality inspection system also includes: obtaining a crack feature information threshold, wherein the crack feature information threshold includes a crack width threshold and a crack length threshold; respectively calculating the ratio of the crack width and crack length in the crack feature information to the crack width threshold and the crack length threshold, and calculating the mean to obtain a crack scale coefficient; obtaining the average image prominence of the crack area image formed by the crack of the crack feature information threshold; calculating the ratio of the image prominence and the average image prominence to obtain a significant scale coefficient; calculating the mean of the crack scale coefficient and the significant scale coefficient to obtain a potential deformation scale coefficient; obtaining a preset deformation detection range, and multiplying the potential deformation scale coefficient by the preset deformation detection range to obtain a bridge deformation detection range; performing deformation detection on the bridge deformation detection range adjacent to the target area to obtain a deformation detection result, which is combined with the crack feature information as a bridge quality inspection result.

[0071] Embodiment 3, the embodiment of the present invention provides a bridge construction quality inspection platform supported by machine vision, including a memory, a processor and a first computer program stored in the memory and executable on the processor, and the processor implements the following steps when executing the first computer program: through an image acquisition device, a surface image of a target bridge is acquired, and first-order crack recognition is performed through machine vision to obtain a target area; through the image acquisition device, a target area image of the target area is located and acquired, image significant area recognition is performed to obtain image significance and image significant area, and significant area images of the image significant area are divided; according to the image significance, the number of recognition nodes for crack recognition of the significant area image is configured, and second-order crack recognition is performed on the significant area image through machine vision to obtain crack feature information; according to the crack feature information and image significance, a bridge deformation detection range is configured, deformation detection is performed on the bridge deformation detection range adjacent to the target area, deformation detection results are obtained, and the crack feature information is combined as the bridge quality inspection result.

[0072] Example 4, please refer to Figure 3 , Figure 3 A schematic diagram of an embodiment of a computer-readable storage medium provided in an embodiment of the present invention. Figure 3As shown, this embodiment provides a computer-readable storage medium 600, on which a second computer program 611 is stored. When the second computer program 611 is executed by a processor, the following steps are implemented: using an image acquisition device, a surface image of a target bridge is acquired, and first-order crack recognition is performed through machine vision to obtain a target area; using the image acquisition device, a target area image of the target area is located and acquired, image significant area recognition is performed, image significance and image significant area are obtained, and significant area images of the image significant area are divided; according to the image significance, the number of recognition nodes for crack recognition of the significant area image is configured, and second-order crack recognition is performed on the significant area image through machine vision to obtain crack feature information; according to the crack feature information and image significance, a bridge deformation detection range is configured, deformation detection is performed on the bridge deformation detection range adjacent to the target area, and deformation detection results are obtained, which are combined with the crack feature information as bridge quality detection results.

[0073] It should be noted that in the above embodiments, the description of each embodiment has its own emphasis, and for parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0074] It will be appreciated by those skilled in the art that embodiments of the present invention may be provided as methods, systems, or computer program products. Therefore, the present invention may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Furthermore, the present invention may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0075] The present invention is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded computer, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0076] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0077] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0078] Although preferred embodiments of the present invention have been described, additional changes and modifications may occur to these embodiments once those skilled in the art understand the basic inventive concepts.

[0079] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention belong to the scope of the present invention and its equivalent technologies, the present invention is also intended to include these changes and variations.

Claims

1. A bridge construction quality inspection method supported by machine vision, characterized in that: Methods include: The surface image of the target bridge is collected by image acquisition equipment, and the first-order cracks are identified by machine vision to obtain the target area; The target area image of the target area is located and acquired by the image acquisition device, image salient area recognition is performed, image saliency and image salient area are obtained, and salient area images of the image salient area are divided, including: Performing pixel saliency analysis on all pixel points in the target area image to obtain all pixel saliencies; Filtering pixel prominences greater than a prominence threshold among all pixel prominences to obtain a plurality of sensitive pixel prominences, and marking pixel points corresponding to the plurality of sensitive pixel prominences as a plurality of sensitive pixel points; Calculating the sum of the multiple sensitive pixels to obtain image saliency; According to a preset anchor frame shape, the plurality of sensitive pixel points in the target area image are framed to obtain a salient area of ​​the image; Dividing the image of the salient area of ​​the image within the target area image to obtain a salient area image; According to the image saliency, the number of recognition nodes for crack recognition of the salient region image is configured, and second-order crack recognition is performed on the salient region image through machine vision to obtain crack feature information; According to the crack feature information and the image prominence, a bridge deformation detection range is configured, deformation detection is performed on the bridge deformation detection range adjacent to the target area, and a deformation detection result is obtained, which is combined with the crack feature information as a bridge quality detection result, including: Acquire a crack characteristic information threshold, wherein the crack characteristic information threshold includes a crack width threshold and a crack length threshold; Respectively calculating the ratios of the crack width and crack length in the crack characteristic information to the crack width threshold and crack length threshold, and calculating the average to obtain a crack scale coefficient; Obtaining the average image prominence of the crack region image formed by the cracks of the crack feature information threshold; Calculating the ratio of the image saliency to the average image saliency to obtain a saliency scale coefficient; Calculating the mean of the crack scale coefficient and the significant scale coefficient to obtain the potential deformation scale coefficient; Obtaining a preset deformation detection range, multiplying the potential deformation scale coefficient by the preset deformation detection range to obtain a bridge deformation detection range; A deformation detection is performed on the deformation detection range of the bridge adjacent to the target area to obtain a deformation detection result, which is combined with the crack feature information as a bridge quality detection result.

2. The machine vision-supported bridge construction quality inspection method according to claim 1 is characterized in that: The surface image of the target bridge is collected by image acquisition equipment, and first-order crack recognition is performed through machine learning to obtain the target area, including: A sample surface image set of similar bridges is collected, and a crack area in each sample surface image is marked to obtain a sample crack area set; Based on a convolutional neural network in machine vision, the sample surface image set and the sample crack region set are used as first-order machine vision training data to train a crack region identifier; Using an image acquisition device, a surface image of the target bridge is acquired; The surface image is input into the crack region identifier, and the crack region is obtained by identification output as the target region.

3. The machine vision-supported bridge construction quality inspection method according to claim 1 is characterized in that: Performing pixel saliency analysis on all pixels in the target area image to obtain all pixel saliencies includes: Performing grayscale processing on the target area image to obtain a target area grayscale image; Selecting a first gray value of a first pixel in the target area gray image, extracting a plurality of first neighboring gray values ​​of a plurality of first neighboring pixels of the first pixel, and randomly extracting a plurality of first random gray values ​​in the target area gray image; Calculating a deviation ratio between a mean of the plurality of first neighboring grayscale values ​​and a mean of the plurality of first random grayscale values ​​as a first pixel saliency of the first pixel; Continue to calculate and process to obtain all pixel saliencies of all pixel points in the target area image.

4. The machine vision-supported bridge construction quality inspection method according to claim 1, characterized in that: According to the image saliency, the number of recognition nodes for crack recognition of the salient region image is configured, and second-order crack recognition is performed on the salient region image through machine vision to obtain crack feature information, including: Training a second-order crack identification channel, wherein the second-order crack identification channel includes a total number of second-order crack identification nodes; According to the crack image recognition data in the historical time, the maximum image prominence is obtained; Calculating a ratio of the image saliency to the maximum image saliency, multiplying the ratio by the total number of nodes and rounding the ratio to obtain the number of identified nodes; Randomly select the second-order crack identification nodes of the identification node number from the second-order crack identification nodes of the total node number, perform crack identification on the significant area image respectively, and obtain node crack feature information of the identification node number, wherein each node crack feature information includes crack width and crack length; The mean of the node crack characteristic information of the identified node number is calculated to obtain the crack characteristic information.

5. The machine vision-supported bridge construction quality inspection method according to claim 4 is characterized in that: Training of the second-order crack identification channel, including: According to the significant area image processing records in the historical time, a sample significant area image set is collected, and the crack width and crack length in each sample significant area image are marked to obtain a sample crack feature information set; According to a preset division ratio, the first crack feature recognition training data is randomly divided in the sample salient area image set and the sample crack feature information set with replacement, and the first and second order crack recognition nodes are trained until convergence; Continue to randomly divide the crack feature recognition training data in the sample significant area image set and the sample crack feature information set according to the preset division ratio with replacement, and perform training of second-order crack recognition nodes until the training obtains the second-order crack recognition nodes with the total number of nodes that converge; The second-order crack identification nodes of the total number of nodes are integrated to obtain a second-order crack identification channel.

6. The machine vision-supported bridge construction quality inspection system is characterized by: The steps for implementing the machine vision-supported bridge construction quality inspection method according to any one of claims 1 to 5 include: A first-order crack recognition module is used to collect the surface image of the target bridge through an image acquisition device, perform first-order crack recognition through machine vision, and obtain the target area; The image salient region recognition module is used to locate and acquire the target region image of the target region through the image acquisition device, perform image salient region recognition, obtain image saliency and image salient region, and divide the salient region image of the image salient region, including: Performing pixel saliency analysis on all pixel points in the target area image to obtain all pixel saliencies; Filtering pixel prominences greater than a prominence threshold among all pixel prominences to obtain a plurality of sensitive pixel prominences, and marking pixel points corresponding to the plurality of sensitive pixel prominences as a plurality of sensitive pixel points; Calculating the sum of the multiple sensitive pixels to obtain image saliency; According to a preset anchor frame shape, the plurality of sensitive pixel points in the target area image are framed to obtain a salient area of ​​the image; Dividing the image of the salient area of ​​the image within the target area image to obtain a salient area image; A second-order crack recognition module is used to configure the number of recognition nodes for crack recognition on the image in the significant area according to the image saliency, perform second-order crack recognition on the image in the significant area through machine vision, and obtain crack feature information; The bridge deformation detection module is used to configure the bridge deformation detection range according to the crack feature information and the image prominence, perform deformation detection on the bridge deformation detection range adjacent to the target area, obtain deformation detection results, and combine the crack feature information as the bridge quality detection results, including: Acquire a crack characteristic information threshold, wherein the crack characteristic information threshold includes a crack width threshold and a crack length threshold; Respectively calculating the ratios of the crack width and crack length in the crack characteristic information to the crack width threshold and crack length threshold, and calculating the average to obtain a crack scale coefficient; Obtaining the average image prominence of the crack region image formed by the cracks of the crack feature information threshold; Calculating the ratio of the image saliency to the average image saliency to obtain a saliency scale coefficient; Calculating the mean of the crack scale coefficient and the significant scale coefficient to obtain the potential deformation scale coefficient; Obtaining a preset deformation detection range, multiplying the potential deformation scale coefficient by the preset deformation detection range to obtain a bridge deformation detection range; A deformation detection is performed on the deformation detection range of the bridge adjacent to the target area to obtain a deformation detection result, which is combined with the crack feature information as a bridge quality detection result.

7. The machine vision-supported bridge construction quality inspection platform is characterized by: include: Memory for storing computer software programs; A processor is used to read and execute the computer software program, thereby implementing the steps of the machine vision-supported bridge construction quality inspection method described in any one of claims 1 to 5.

8. A non-transitory computer-readable storage medium, characterized in that: The storage medium stores a computer software program, which, when executed by a processor, implements the steps of the machine vision-supported bridge construction quality inspection method as described in any one of claims 1 to 5.

Images