Underwater building concrete apparent defect detection method and system based on artificial intelligence

Abstract

The invention discloses an underwater building concrete apparent defect detection method and system based on artificial intelligence. The method comprises the following steps: firstly, carrying out aerial and underwater calibration on preset color and black-and-white cameras; a first image and a second image which are shot by the color camera after calibration and are provided with and not provided with laser stripes and a third image shot by the black-and-white camera are obtained; and matching the first image and the third image based on the laser stripes, constructing a point cloud image, and obtaining a target point cloud image by combining color enhancement of the second image. And finally, identifying the defect of the image by using an apparent defect detection model to obtain a defect detection result, thereby realizing efficient and accurate underwater building concrete apparent defect detection.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of artificial intelligence, in particular to a method and system for detecting concrete surface defects of underwater structures based on artificial intelligence. BACKGROUND

[0002] With the increasing number of underwater structures, their safety becomes increasingly important. The detection of concrete surface defects of underwater structures faces many challenges, and traditional detection methods have low precision, poor efficiency, and are greatly affected by underwater environments. SUMMARY

[0003] The purpose of the present application is to provide a method and system for detecting concrete surface defects of underwater structures based on artificial intelligence.

[0004] In a first aspect, the present application provides a method for detecting concrete surface defects of underwater structures based on artificial intelligence, comprising: air calibration and underwater calibration of a preset color camera and a preset black and white camera; obtaining a first image and a second image of the concrete surface of the underwater structure taken by the calibrated color camera, the first image having laser stripes, and the second image not having laser stripes, the laser stripes being generated based on underwater refraction; obtaining a third image of the concrete surface of the underwater structure taken by the calibrated black and white camera; matching the first image and the third image based on the laser stripes, and based on the matching, constructing a point cloud image of the concrete surface of the underwater structure based on the first image and the third image, and enhancing the color based on the second image to obtain a target point cloud image of the concrete surface of the underwater structure; defect recognition of the target point cloud image of the concrete surface of the underwater structure based on a surface defect detection model to obtain a defect detection result corresponding to the target point cloud image of the concrete surface of the underwater structure.

[0005] In a possible implementation, the defect recognition of the target point cloud image of the concrete surface of the underwater structure based on the surface defect detection model to obtain the defect detection result corresponding to the target point cloud image of the concrete surface of the underwater structure comprises: obtaining a first sample building concrete surface image set and a second sample building concrete surface image set, the first sample building concrete surface image set including a first sample building concrete surface image, and the second sample building concrete surface image set including a second sample building concrete surface image set configured with sample target values, the second sample building concrete surface image set being classified into a target defect category set; The first sample building concrete appearance image is trained by a multi-task learning method, and the second sample building concrete appearance image set is trained by a reinforcement learning method to obtain a master defect detection model; obtain a slave defect detection model, which is an original model without trained model parameters; The model parameters of the slave defect detection model are transferred by the first sample building concrete appearance image and the master defect detection model as a teacher model to obtain an appearance defect detection model. The target underwater building concrete appearance point cloud image is recognized by the appearance defect detection model to obtain a defect detection result of the target underwater building concrete appearance point cloud image in the target defect category set.

[0006] In a possible implementation, the slave defect detection model is obtained by transferring the model parameters of the slave defect detection model by the first sample building concrete appearance image and the master defect detection model as a teacher model, and includes: The first sample building concrete appearance image in the first sample building concrete appearance image set is recognized by the master defect detection model to obtain a pending target value corresponding to the first sample building concrete appearance image in the target defect category set. The first sample building concrete appearance image in the first sample building concrete appearance image set is recognized by the slave defect detection model to obtain a recognition result corresponding to the first sample building concrete appearance image. The model parameters of the slave defect detection model are optimized based on the deviation between the pending target value and the recognition result to obtain the appearance defect detection model.

[0007] In a possible implementation, the slave defect detection model is obtained by transferring the model parameters of the slave defect detection model by the first sample building concrete appearance image and the master defect detection model as a teacher model, and includes: Obtain a credibility index of the pending target value. Determine the deviation between the pending target value and the recognition result under the credibility index, and optimize the model parameters of the slave defect detection model based on the deviation to obtain the appearance defect detection model.

[0008] In a possible implementation, the defect identification on the first sample building concrete apparent image in the first sample building concrete apparent image set by the main defect detection model obtains a pending target value corresponding to a category in the target defect category set of the first sample building concrete apparent image, and the method comprises the following steps of: The defect identification on the first sample building concrete apparent image in the first sample building concrete apparent image set by the main defect detection model obtains a confidence degree of the first sample building concrete apparent image corresponding to a category in the target defect category set. The pending target value corresponding to the first sample building concrete apparent image is determined from the target defect category set based on the confidence degree.

[0009] In a possible implementation, the defect detection model comprises a first index vectorization unit and a first related element vectorization unit. The method further comprises the following steps of: The first sample building concrete apparent image is vectorized by the first index vectorization unit to obtain a first vector representation. A second vector representation of the first sample building concrete apparent image and data in a first preset database vectorized by the first related element vectorization unit is obtained. Knowledge migration is performed on the defect detection model based on the deviation between the first vector representation and the second vector representation, and the apparent defect detection model is obtained.

[0010] In a possible implementation, the main defect detection model is obtained by training the first sample building concrete apparent image in a multi-task learning method and the second sample building concrete apparent image set in a reinforcement learning method, and the method comprises the following steps of: The feature extraction component is subjected to multi-task learning based on the first sample building concrete apparent image in the first sample building concrete apparent image set. The defect detection model is obtained by combining the feature extraction component subjected to multi-task learning with a gated recurrent component, and the gated recurrent component is used for apparent defect detection in the target defect category set. The defect detection model is subjected to reinforcement learning by the second sample building concrete apparent image set and the sample target value, and the main defect detection model is obtained.

[0011] In a possible implementation, the feature extraction component comprises a second index vectorization unit and a second related element vectorization unit. The first sample building concrete appearance image in the first sample building concrete appearance image set is used for multi-task learning of the feature extraction component, including: The first sample building concrete appearance image is vectorized by the second index vectorization unit to obtain a third vector representation; The first sample building concrete appearance image and data in the second preset database are vectorized by the second correlation element vectorization unit to obtain a fourth vector representation; The feature extraction component is subjected to multi-task learning based on the deviation between the third vector representation and the fourth vector representation.

[0012] In a possible implementation, the gating cycle component is combined with the feature extraction component subjected to multi-task learning to obtain a defect detection model, including: The gating cycle component is cascaded with the second index vectorization unit subjected to multi-task learning to obtain the defect detection model.

[0013] In a second aspect, an embodiment of the present application provides a server system, including a server, and the server is configured to execute the method in the first aspect.

[0014] Compared with the prior art, the present application provides the following beneficial effects: by using the underwater building concrete appearance defect detection method and system based on artificial intelligence, the preset color and black-and-white cameras are calibrated in the air and underwater. Then, the first and second images with and without laser stripes captured by the calibrated color camera and the third image captured by the black-and-white camera are obtained. The first and third images are matched based on the laser stripes, a point cloud image is constructed, and the color of the second image is enhanced to obtain a target point cloud image. Finally, the appearance defect detection model is used to identify the defects of the image to obtain a defect detection result, and efficient and accurate underwater building concrete appearance defect detection is realized. BRIEF DESCRIPTION OF DRAWINGS

[0015] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiments. It should be understood that the following drawings only show some embodiments of the present application, and therefore should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can also be obtained without creative labor on the basis of these drawings.

[0016] Figure 1 The step flowchart of the underwater building concrete appearance defect detection method based on artificial intelligence provided by the embodiments of the present application is shown in the figure. Figure 2 The structural schematic block diagram of the computer device provided by the embodiments of the present application is shown in the figure. DETAILED DESCRIPTION

[0017] In order to make the objects, technical solutions and advantages of the embodiments of the present application clearer, the following will be combined with the accompanying drawings to make a clear and complete description of the technical solutions in the embodiments of the present application. Obviously, the described embodiments are some but not all of the embodiments of the present application. The components of the embodiments of the present application described and shown in the accompanying drawings can be arranged and designed in various different configurations.

[0018] The specific embodiments of the present application will be described in detail below with reference to the accompanying drawings.

[0019] In order to solve the technical problems in the foregoing background art, Figure 1 The flowchart of the method for detecting concrete surface defects of underwater structures based on artificial intelligence provided by the embodiments of the present application is shown in the following detailed description of the method for detecting concrete surface defects of underwater structures based on artificial intelligence.

[0020] Step S201, aerial calibration and underwater calibration are performed on a preset color camera and a preset black-and-white camera. Step S202, a first image and a second image of the concrete surface of the underwater structure are acquired by the calibrated color camera, the first image has laser stripes, and the second image does not have laser stripes, the laser stripes are generated based on water refraction; Step S203, a third image of the concrete surface of the underwater structure is acquired by the calibrated black-and-white camera. Step S204, the first image and the third image are matched based on the laser stripes, on the basis of successful matching, the concrete surface point cloud image of the underwater structure is constructed based on the first image and the third image, and color enhancement is performed based on the second image to obtain a target concrete surface point cloud image of the underwater structure. Step S205, the target concrete surface point cloud image of the underwater structure is subjected to defect recognition based on a surface defect detection model to obtain a defect detection result corresponding to the target concrete surface point cloud image of the underwater structure.

[0021] In the embodiments of the present application, for example, in a construction site of a large-scale water conservancy project, the server receives a task of defect detection on the concrete surface of the underwater structure. First, the preset color camera and the preset black-and-white camera need to be calibrated.

[0022] Aerial Calibration: The color camera and the black-and-white camera are mounted on a mechanical arm that can be precisely controlled to move. The mechanical arm is placed in a spacious room with distinct feature points, such as a room filled with regular patterns. The server controls the mechanical arm to move the cameras according to a pre-set trajectory, while the cameras capture images at different positions and angles. These images contain rich spatial feature information. The server analyzes the positions of the feature points in these images and the movement parameters of the cameras to calculate the internal parameters (such as focal length, principal point position, etc.) and external parameters (such as the position and orientation of the camera in space) of the cameras. This is the process of aerial calibration, which lays the foundation for accurate shooting and image analysis in the subsequent underwater environment.

[0023] Underwater Calibration: The color camera and the black-and-white camera that have completed aerial calibration are installed in a specially designed waterproof housing and placed in a large water tank filled with water. Inside the water tank, there are some standard objects with known sizes and positions, such as specially designed standard geometric modules. The server controls the cameras to shoot images of these standard objects underwater. Due to the refraction and absorption of water, the images will have certain distortions and distortions. The server analyzes these distorted and distorted images and combines the known size and position information of the standard objects to further correct and optimize the parameters of the cameras to adapt to the special optical conditions of the underwater environment. This process is underwater calibration, which ensures that the images shot by the cameras underwater can accurately reflect the true shape and position of the objects.

[0024] After completing the camera calibration, the color camera is installed on the underwater robot, and the underwater robot is placed under the dam of the water conservancy project. The server controls the underwater robot to approach the concrete surface of the underwater building.

[0025] When the color camera starts shooting, it first shoots the first image with laser stripes. The server controls the laser emitter installed on the underwater robot to emit laser beams of a specific pattern, which will form unique laser stripes when they contact the concrete surface of the underwater building due to the refraction of water. The color camera captures these images with laser stripes and transmits them back to the server.

[0026] Subsequently, the server controls the color camera to shoot the second image without laser stripes. This is to obtain the image information of the concrete surface under normal lighting conditions for subsequent color enhancement and comprehensive analysis.

[0027] At the same time as the color camera is shooting, the black-and-white camera mounted on the underwater robot is also working synchronously. The black-and-white camera captures high-contrast images of the concrete surface of the underwater structure, which has good capturing ability for the contours and details of the object. The server obtains the third images captured by the black-and-white camera, which complement the images captured by the color camera, providing more comprehensive information for subsequent point cloud image construction.

[0028] After receiving the first and third images, the server begins image matching work. Because the laser stripes have unique distribution and characteristics in the first image, the server first extracts the feature points of the laser stripes in the first image, such as the intersection points and endpoints of the stripes. Then, corresponding feature points are found in the third image. Through matching and calculating the feature points, the correspondence between the first and third images is determined.

[0029] After successful matching, the server constructs the underwater structure concrete surface point cloud image based on the first and third images. According to the camera calibration parameters and the feature point positions of the object in the image, the server calculates the three-dimensional coordinates of each point on the object surface. By connecting a large number of points, a point cloud image reflecting the shape of the underwater structure concrete surface is formed.

[0030] At the same time, the server obtains the second image and performs color enhancement processing. The server analyzes the color distribution and brightness values of the pixels in the second image, and adjusts parameters such as contrast, saturation, and brightness to enhance the brightness and clarity of the colors in the image, making the details of the concrete surface more obvious.

[0031] Finally, the second image after color enhancement processing is fused with the constructed point cloud image to obtain the target underwater structure concrete surface point cloud image. This fused image contains both the accurate three-dimensional shape information of the object surface and rich, clear color information, providing high-quality input data for subsequent defect detection.

[0032] The server has obtained the target underwater structure concrete surface point cloud image, and the next step is to use the pre-trained surface defect detection model for defect recognition.

[0033] Before training the surface defect detection model, the server first collects a large number of sample building concrete surface images, including normal and defective images. These images are divided into a first sample building concrete surface image set and a second sample building concrete surface image set. The first sample building concrete surface image set is used for multi-task learning to train the basic feature extraction ability of the model; the second sample building concrete surface image set has sample target values, which is used for reinforcement learning to optimize the model's defect detection ability.

[0034] The server trains the feature extraction component with a multi-task learning method using a first set of sample building concrete surface images. For example, for a set of concrete surface images under different lighting conditions, the server has the model learn how to extract common features such as texture, shape, etc., while also learning how to adapt to different lighting variations.

[0035] Then, the server combines the gating recurrent component with the feature extraction component trained with multi-task learning to form a preliminary defect detection model. The gating recurrent component can effectively perform surface defect detection in the target defect class set.

[0036] Next, the server uses a second set of sample building concrete surface images and corresponding sample target values to perform reinforcement learning on the defect detection model. For example, for a set of concrete surface images known to have defects such as cracks, spalling, etc., the server adjusts the parameters of the model so that it can accurately identify these defects and give corresponding classification and evaluation.

[0037] After repeated training and optimization, the server obtains a master defect detection model. Then, the server obtains a slave defect detection model, which is an original model whose model parameters have not been trained. The server performs defect recognition on the images in the first set of sample building concrete surface images using the master defect detection model to obtain tentative target values. For example, for a concrete surface image, the master defect detection model may give a confidence of 80% that it belongs to the crack defect.

[0038] At the same time, the server has the slave defect detection model perform defect recognition on the same image to obtain a recognition result. The server optimizes the model parameters of the slave defect detection model based on the deviation between the tentative target value and the recognition result. For example, if the recognition result given by the slave defect detection model is that it belongs to the spalling defect, which has a large deviation from the tentative target value (crack defect) given by the master defect detection model, the server will adjust the parameters of the slave defect detection model to make it closer to the judgment of the master defect detection model.

[0039] The server also obtains a credibility indicator of the tentative target value. For example, if the master defect detection model has a high confidence in its judgment of an image, it means that the credibility of the tentative target value is high; on the contrary, if the confidence is low, the credibility is low. The server optimizes the model parameters of the slave defect detection model more accurately based on the deviation under the credibility indicator, and finally obtains a surface defect detection model.

[0040] Finally, the server inputs the target underwater building concrete appearance point cloud image into the appearance defect detection model. The model analyzes and judges each region in the image, identifies whether there is a defect, and the type, location and severity of the defect, etc. The server obtains the defect detection result corresponding to the target underwater building concrete appearance point cloud image, such as detecting a crack of a certain length and width at a certain location, or a slight spalling at another location, etc. These results will be output and provided to relevant engineering personnel so that they can take appropriate maintenance and reinforcement measures to ensure the safety and stability of the underwater building.

[0041] In the embodiment of the present application, the appearance defect detection model is used to identify defects in the target underwater building concrete appearance point cloud image, and the defect detection result corresponding to the target underwater building concrete appearance point cloud image is obtained. The implementation can be performed by the following examples.

[0042] A first sample building concrete appearance image set and a second sample building concrete appearance image set are obtained, the first sample building concrete appearance image set includes a first sample building concrete appearance image, and the second sample building concrete appearance image set includes a second sample building concrete appearance image set configured with sample target values, and the second sample building concrete appearance image set is classified into a target defect category set; The first sample building concrete appearance image is trained by a multi-task learning method, and the second sample building concrete appearance image set is trained by a reinforcement learning method to obtain a main defect detection model; A slave defect detection model is obtained, which is an original model without trained model parameters; The model parameters of the slave defect detection model are transferred by the first sample building concrete appearance image and the main defect detection model as a teacher model to obtain an appearance defect detection model; The target underwater building concrete appearance point cloud image is identified by the appearance defect detection model to obtain a defect detection result of the target underwater building concrete appearance point cloud image belonging to the target defect category set.

[0043] In the embodiment of the present application, the server collects a large number of building concrete appearance images from the database of a plurality of large-scale construction projects. These images cover buildings of different types, different service lives and different environmental conditions.

[0044] The first sample building concrete appearance image set contains images of various normal states and images of concrete surfaces with minor defects. For example, there are images of concrete walls that have just been built and look good, and there are images of concrete floors that have been used for several years and have some minor cracks that do not affect the structural safety.

[0045] The second sample building concrete appearance image set is more targeted and severe defect images, and each image is configured with an accurate sample target value. These images are classified into a set of target defect categories. For example, there are images of concrete dams with serious cracks that cause water leakage, and the sample target value clearly labels the length, width, and depth of the cracks. There are also images of concrete bridge piers with large areas of spalling due to erosion, and the sample target value describes the area and depth of the spalling area in detail.

[0046] The server first uses the first sample building concrete appearance image for multi-task learning training. For these images, the server sets multiple tasks, such as simultaneously learning to identify different types of defect features such as cracks, holes, and corrosion, and adapting to different lighting and shooting angles.

[0047] During training, the server allows the model to continuously adjust parameters to extract general features from images that can be applied to multiple tasks at the same time. For example, for a concrete wall image with both minor cracks and slight corrosion, the model must learn to identify the direction and width of the cracks, as well as distinguish the color and texture changes in the corrosion area.

[0048] Next, the server uses the second sample building concrete appearance image set for reinforcement learning training. In this stage, the server provides the model with explicit reward or punishment signals based on the sample target value of the image.

[0049] For example, for a concrete bridge image labeled as having serious cracks, if the model can accurately detect the location, length, and width of the cracks, the server will give the model a large reward; otherwise, if the model's detection result deviates greatly from the sample target value, the server will give the model the corresponding punishment. By continuously receiving these feedback signals, the model gradually optimizes its parameters, improving the detection accuracy and accuracy of serious defects.

[0050] After repeated multi-task learning and reinforcement learning training, the server finally obtains a main defect detection model that can accurately detect various defects.

[0051] The server obtains a brand new, untrained defect detection model from the model library. This model has the same architecture and parameter settings as the main defect detection model, but all parameters are in their initial state and have not been adjusted or optimized by any data.

[0052] The server inputs the first sample building concrete apparent image into the main defect detection model and obtains output results of the main defect detection model on the images.

[0053] For each input image, the main defect detection model gives a prediction about the type and characteristics of possible defects. The server delivers these prediction results as "knowledge" to the slave defect detection model.

[0054] After receiving the "knowledge" from the main defect detection model, the slave defect detection model adjusts its model parameters according to the difference between the output results of the main defect detection model.

[0055] For example, if the main defect detection model predicts that there is a fine crack in a concrete floor image and gives a rough location and characteristic description of the crack, and the initial prediction result of the slave defect detection model is different, the server will adjust the relevant parameters of the slave defect detection model according to the difference, so that it gradually approaches the prediction result of the main defect detection model.

[0056] After multiple knowledge transfers and parameter adjustments, the slave defect detection model gradually learns the knowledge and experience of the main defect detection model, thereby forming an apparent defect detection model.

[0057] After the server receives the target underwater building concrete apparent point cloud image, it inputs the image into the apparent defect detection model.

[0058] The model comprehensively analyzes and processes the image, extracts various features in the image, and compares and judges the defect characteristics learned before.

[0059] Finally, the model gives a detection result, which clearly indicates the specific category of the image in the target defect category set. For example, the detection result may indicate that there is a crack with a length exceeding a certain threshold on the surface of the underwater building concrete, or there is a large area of peeling phenomenon, etc.

[0060] These detection results will be recorded and output by the server, providing an important basis for subsequent repair and maintenance work.

[0061] In the embodiment of the present application, the knowledge transfer on the model parameters of the slave defect detection model by the first sample building concrete apparent image and the main defect detection model as a teacher model to obtain an apparent defect detection model can be implemented through the following examples.

[0062] perform defect recognition on the first sample building concrete apparent image in the first sample building concrete apparent image set through the slave defect detection model, to obtain an identification result corresponding to the first sample building concrete apparent image; perform defect recognition on the first sample building concrete apparent image in the first sample building concrete apparent image set through the slave defect detection model, to obtain an identification result corresponding to the first sample building concrete apparent image; optimize the model parameters of the slave defect detection model based on the deviation between the pending target value and the identification result, to obtain the apparent defect detection model.

[0063] In an embodiment of the present application, the server obtains a large number of images in the first sample building concrete apparent image set. One of the images is the concrete surface of a water gate that has been used for a certain period of time, and there are some fine cracks and local erosion on the surface.

[0064] The server inputs this image into the master defect detection model. The master defect detection model performs complex calculations and analyses to identify the defects in this image. It comprehensively considers the shape, length, width, direction of the cracks, and the area, depth and texture of the erosion region.

[0065] After a series of operations and comparisons, the master defect detection model concludes that the defects in this image may belong to the "slight cracks and moderate erosion" category in the target defect category set, and gives a high confidence level for this judgment, such as 85%. This judgment result becomes the pending target value of this first sample building concrete apparent image.

[0066] The server continues to process other images in the first sample building concrete apparent image set in the same way to obtain the corresponding pending target value for each image.

[0067] Still taking the image of the concrete surface of the water gate as an example. The server inputs this image into the slave defect detection model.

[0068] The slave defect detection model may not be accurate enough in identifying the defects in the image due to insufficient training and optimization. It may conclude that the defects in this image belong to the "slight cracks and slight erosion" category, which is different from the judgment of the master defect detection model.

[0069] For other images in the first sample building concrete apparent image set, the slave defect detection model will also give corresponding identification results, but these results may have different degrees of deviation from the pending target values given by the master defect detection model.

[0070] The server compares the pending target value given by the main defect detection model with the recognition result given by the defect detection model. Taking the image of the surface of the concrete of the water gate as an example, there is a deviation between the defect category and severity judged by the defect detection model and the pending target value of the main defect detection model.

[0071] The server calculates the specific value of such deviation. For example, the deviation in crack category judgment is 1 level (assuming that the crack category is divided into three levels of slight, moderate and severe), and the deviation in erosion degree judgment is 1 level (assuming that the erosion degree is divided into three levels of slight, moderate and severe).

[0072] Based on these deviations, the server uses a specific optimization algorithm to adjust the model parameters of the defect detection model. For example, increase the weight related to crack feature extraction, or adjust the threshold related to erosion degree judgment.

[0073] The server performs such processing and optimization on each image in the first sample building concrete surface image set, so that the parameters of the defect detection model are continuously adjusted and improved.

[0074] After multiple iterations and optimizations, the recognition result of the defect detection model gradually approaches the pending target value given by the main defect detection model, and finally obtains the apparent defect detection model. This apparent defect detection model shows higher accuracy and reliability in defect recognition of the first sample building concrete surface image.

[0075] In the embodiment of the application, the model parameters of the defect detection model are optimized based on the deviation between the pending target value and the recognition result to obtain the apparent defect detection model, which can be implemented by the following examples.

[0076] Obtain a credibility index of the pending target value; Determine the deviation between the pending target value and the recognition result under the credibility index, and optimize the model parameters of the defect detection model based on the deviation to obtain the apparent defect detection model.

[0077] In the embodiment of the application, for example, when the server processes images from the first sample building concrete surface image set, taking the image of the concrete surface of an old bridge pier as an example. The main defect detection model performs defect recognition on this image and obtains the pending target value of "moderate crack and slight spalling".

[0078] The server obtains a confidence indicator for this pending target value. The confidence indicator can be calculated based on multiple factors. For example, the main defect detection model has a high historical accuracy in identifying this type of defect, or the image has good clarity and completeness, allowing the model to more accurately extract features for judgment, which can increase the confidence of the pending target value. In this example, the confidence indicator of the pending target value is determined to be 90% based on comprehensive evaluation.

[0079] The server continues to process other images, such as an image of a dam concrete surface, and the pending target value given by the main defect detection model is "severe erosion and fine cracks", and the confidence indicator may be determined to be 70% due to factors such as the complexity of the dam environment and the limitations of the image shooting angle.

[0080] Still taking the images of the old bridge pier and the dam as an example.

[0081] For the image of the old bridge pier, the recognition result from the defect detection model is "mild cracks and no spalling". There is a deviation compared to the pending target value "moderate cracks and mild spalling" given by the main defect detection model.

[0082] The server considers the high confidence indicator of the pending target value 90% and gives more attention to the deviation. The specific calculation of the deviation may involve quantitative comparison of the crack degree and spalling condition. For example, the crack degree is divided into 1 to 5 levels, and the moderate crack is level 3 and the mild crack is level 2, so the crack degree deviation is 1 level. The spalling condition has a difference between existence and non-existence, which is considered as a large deviation.

[0083] Based on these deviations, the server adjusts the relevant model parameters of the defect detection model from the main defect detection model. For example, increase the weight related to crack features, or adjust the threshold related to spalling judgment.

[0084] For the image of the dam, the recognition result from the defect detection model is "moderate erosion and no cracks". There is a deviation compared to the pending target value "severe erosion and fine cracks" given by the main defect detection model.

[0085] Since the confidence indicator is 70%, the server will be relatively cautious when evaluating the deviation. But still according to the quantitative calculation of the specific erosion degree and the difference between the existence and non-existence of cracks, the parameters of the defect detection model from the main defect detection model are adjusted appropriately.

[0086] The server processes a large number of images in the first sample building concrete surface image set in this way. The deviation of the pending target value and the recognition result of the defect detection model from the main defect detection model is used to continuously optimize the model parameters of the defect detection model from the main defect detection model.

[0087] After multiple iterations and optimizations, the defect detection model gradually learns and adjusts, making its recognition results closer and closer to the pending target values given by the main defect detection model. Finally, an apparent defect detection model that can accurately identify defects is obtained.

[0088] In the embodiments of the present application, the defect identification of the first sample building concrete apparent image in the first sample building concrete apparent image set by the main defect detection model obtains the pending target value of the category in the target defect category set corresponding to the first sample building concrete apparent image. The implementation can be performed by the following examples.

[0089] The defect identification of the first sample building concrete apparent image in the first sample building concrete apparent image set by the main defect detection model obtains the confidence of the classification corresponding to the first sample building concrete apparent image in the target defect category set; Based on the confidence, the pending target value corresponding to the first sample building concrete apparent image is determined from the target defect category set.

[0090] In the embodiments of the present application, for example, the server obtains an image in the first sample building concrete apparent image set, which is a concrete pillar surface image of a seaside wharf. In the image, some unobvious texture changes and slight crack signs can be seen.

[0091] The main defect detection model starts to analyze this image. It first extracts various features in the image, such as the direction of texture, the shape and length of cracks, the color change of the concrete surface, etc. Then, these features are compared and calculated with the feature patterns stored in the model about various defect types.

[0092] After a series of complex operations, the main defect detection model obtains the classification in the target defect category set that the image may belong to and the corresponding confidence. Assuming that for the image of the wharf pillar, the main defect detection model judges that it may belong to the "slight crack" category with a confidence of 75%, it may also belong to the "surface weathering" category with a confidence of 20%, and it has a small possibility of belonging to the "local peeling" category with a confidence of 5%.

[0093] The server continues to process other images in the first sample building concrete apparent image set. For example, a concrete surface image of a bridge abutment, the main defect detection model analyzes and gives a confidence of 80% for the "moderate corrosion" category, a confidence of 15% for the "fine crack" category, and a lower confidence for other categories.

[0094] For the above-mentioned pier support image, since the confidence of the "slight crack" category is the highest, reaching 75%, the server determines that the pending target value of this image is "slight crack" according to this confidence.

[0095] Similarly, for the image of the bridge abutment, since the confidence of the "moderate corrosion" category is 80%, which is significantly higher than that of other categories, the server determines "moderate corrosion" as the pending target value of this image.

[0096] Suppose the server processes another image of a dam slope concrete surface, the main defect detection model gives the confidence of "severe cracks" as 45%, the confidence of "large area spalling" as 40%, and the confidence of "severe erosion" as 15%. In this case, although the confidence of "severe cracks" is not absolutely dominant, it is relatively higher than the other two categories, and the server still determines "severe cracks" as the pending target value.

[0097] The server processes a large number of images in the first sample building concrete surface image set in this way. By comprehensively analyzing the confidence given by the main defect detection model, a most likely pending target value is determined for each image from the target defect category set, providing an accurate reference basis for subsequent optimization and training of the from defect detection model.

[0098] In the embodiment of the present application, the from defect detection model comprises a first index vectorization unit and a first related element vectorization unit; The embodiment of the present application also provides the following implementation.

[0099] The first index vectorization unit is used to vectorize the first sample building concrete surface image, obtaining a first vector representation; The first related element vectorization unit is used to vectorize the first sample building concrete surface image and the data in the first preset database, obtaining a second vector representation; The knowledge of the from defect detection model is transferred based on the deviation between the first vector representation and the second vector representation, obtaining the surface defect detection model.

[0100] In the embodiment of the present application, for example, the server receives an image from the first sample building concrete surface image set, which is a concrete surface image of a large underground parking lot support, and there are some fine cracks and local discoloration in the image.

[0101] The server starts the first index vectorization unit in the from defect detection model to process the image. The first index vectorization unit will convert various features in the image, such as the length, width, and direction of the cracks, the size and color value of the discoloration area, etc., into a digital vector.

[0102] Assuming that the processed image of the parking garage column is represented as a first vector with certain dimensions and numerical values, such as a vector containing 512 numerical values, each representing a quantized value of a feature in the image.

[0103] The server continues the same process for other images in the first sample set of building concrete surface images, obtaining a series of first vector representations.

[0104] For the same image of the underground parking garage column, the server simultaneously starts the first related element vectorization unit from the defect detection model.

[0105] The first related element vectorization unit not only considers the current image but also retrieves relevant data from a first pre-set database. This database may contain a large amount of past collected feature descriptions, statistical information, typical cases, etc. about building concrete defects.

[0106] The first related element vectorization unit comprehensively analyzes and vectorizes the features of the current image with the relevant data in the database, obtaining a second vector representation.

[0107] Assuming that the second vector is a 1024-dimensional vector that integrates the features of the current image and the quantized information of the relevant knowledge in the database.

[0108] The server also performs the same operation on other images in the first sample set of building concrete surface images, obtaining corresponding second vector representations.

[0109] Still taking the image of the underground parking garage column as an example, the server obtains its first vector representation and second vector representation.

[0110] The server begins to calculate the deviation between the two vectors. The deviation calculation may involve various measurement methods such as the difference between vector elements, Euclidean distance, cosine similarity, etc.

[0111] Assuming that the calculation result shows that there is a large deviation between the two vectors, it indicates that the output of the first index vectorization unit from the defect detection model in processing this image is significantly different from the output of the first related element vectorization unit combined with database knowledge.

[0112] The server adjusts the parameters of the defect detection model according to this deviation. For example, if the first index vectorization unit's quantization of crack features is not accurate enough, resulting in a large deviation from the second vector, the server will adjust the parameters related to crack feature extraction in that unit to make it more accurate in vectorizing the image.

[0113] The server performs such deviation calculation and parameter adjustment on each image in the first sample building concrete apparent image set.

[0114] Through multiple iterations and optimizations, the defect detection model gradually learns and improves, and the deviation between the first vector representation and the second vector representation is continuously reduced. Finally, the knowledge transfer is completed from the defect detection model to become an apparent defect detection model capable of accurately performing defect detection.

[0115] For example, for an image of a sewage treatment plant concrete pool wall, the optimized apparent defect detection model can give more accurate and reliable defect judgment results.

[0116] In the embodiments of the present application, the main defect detection model is trained by the first sample building concrete apparent image set through multi-task learning and the second sample building concrete apparent image set through reinforcement learning. The implementation can be performed through the following examples.

[0117] Multi-task learning is performed on the feature extraction component based on the first sample building concrete apparent image in the first sample building concrete apparent image set; The defect detection model is obtained by combining the gated recurrent component with the feature extraction component that has undergone multi-task learning, and the gated recurrent component is used for apparent defect detection in the target defect category set; The main defect detection model is obtained by reinforcement learning on the defect detection model through the second sample building concrete apparent image set and the sample target value in the second sample building concrete apparent image set.

[0118] In the embodiments of the present application, for example, the server obtains a series of images in the first sample building concrete apparent image set. One of them is an image of a high-rise residential basement concrete wall, and there are fine cracks and local water stains in the image.

[0119] The server starts multi-task learning on the feature extraction component using this image. The set tasks include identifying the type of cracks (such as longitudinal cracks, transverse cracks), evaluating the width of cracks, judging the area and depth of water stains, etc.

[0120] The feature extraction component starts to analyze this image and extract features related to each task. For example, for identifying the type of cracks, it focuses on the direction and shape features of the cracks; for evaluating the width of the cracks, it focuses on extracting pixel information of the crack edges; for judging the area and depth of water stains, it analyzes the color changes and continuity of the region.

[0121] The server continues to use other first sample building concrete appearance images, such as a concrete surface image of a bridge pier column, which has peeling traces and rust spots. The feature extraction component, when processing this image, simultaneously learns how to identify the degree of peeling, distinguish the types of rust spots, and so on.

[0122] Through such multi-task learning on a large number of different first sample building concrete appearance images, the feature extraction component gradually masters the ability to extract multiple useful features from complex images.

[0123] After completing the multi-task learning of the feature extraction component, the server combines the gating recurrent component with the trained feature extraction component.

[0124] The gating recurrent component is a component specially used for processing sequence data and making dynamic predictions. In this scenario, it is used to detect and classify the appearance defects in the target defect category set according to the features extracted by the feature extraction component.

[0125] For example, for the image of the high-rise residential basement concrete wall mentioned earlier, after the feature extraction component extracts the features of cracks and water stains, the gating recurrent component will determine which specific defect category the image belongs to, such as the "minor cracks and moderate water stains" category, according to the sequence and combination of these features.

[0126] Similarly, for the image of the bridge pier column, the gating recurrent component combines the peeling and rust features extracted by the feature extraction component to give the corresponding defect detection result.

[0127] In this way, by combining the two components, the server obtains a preliminary defect detection model.

[0128] The server obtains an image from the second sample building concrete appearance image set, which is a severely damaged image of a large sluice concrete structure, with obvious large cracks and large areas of concrete peeling, and the sample target value clearly labels the specific type and severity of the defect.

[0129] The server inputs this image into the previously obtained defect detection model. After the model gives the detection result, the server compares it with the sample target value. If the detection result of the model is inconsistent with the sample target value, the server will give the model a certain punishment according to the deviation; if the detection result is close to or consistent with the sample target value, the server will give the model a reward.

[0130] For example, the model may initially judge the defect in this sluice image as "moderate cracks and moderate peeling", while the sample target value is "severe cracks and severe peeling". According to this deviation, the server adjusts the parameters of the model so that it can give more accurate results when encountering similar images next time.

[0131] The server continues to use other images in the second sample building concrete apparent image set, such as an image of a severely corroded concrete foundation of a chemical plant storage tank, to repeatedly reinforce learning and parameter adjustment of the defect detection model.

[0132] After reinforcement learning on a large number of second sample building concrete apparent images with clear sample target values, the defect detection model is continuously optimized and improved, and finally the server obtains a main defect detection model that can accurately detect various complex defects.

[0133] In the embodiment of the application, the feature extraction component includes a second index vectorization unit and a second related element vectorization unit; The multi-task learning of the feature extraction component based on the first sample building concrete apparent image in the first sample building concrete apparent image set can be implemented by the following examples.

[0134] The first sample building concrete apparent image is vectorized by the second index vectorization unit to obtain a third vector representation; Obtain the fourth vector representation of the first sample building concrete apparent image and the data in the second preset database vectorized by the second related element vectorization unit; The multi-task learning of the feature extraction component based on the deviation between the third vector representation and the fourth vector representation.

[0135] In the embodiment of the application, for example, the server receives an image in the first sample building concrete apparent image set, which is an image of the concrete floor of an old warehouse, and there are multiple cracks and worn areas in the image.

[0136] The server starts the second index vectorization unit in the feature extraction component to process this image. This unit will analyze and quantify various features in the image. For example, it will focus on the length, width, distribution of cracks, the area, shape and roughness of worn areas, etc.

[0137] After complex calculation and conversion, the second index vectorization unit converts these features into a digital vector, i.e. a third vector representation. Assuming that this third vector is a vector with 256 elements, each element represents the quantified information of a certain aspect of the image.

[0138] The server continues to process other images in the first sample building concrete apparent image set, such as an image of the concrete at the expansion joint of a bridge, which has obvious damage and deformation. Similarly, through the second index vectorization unit, this image is converted into a corresponding third vector representation.

[0139] For the old warehouse's concrete floor image, the server simultaneously starts the second related element vectorization unit in the feature extraction component.

[0140] This unit not only analyzes the current image but also retrieves relevant data from a second pre-set database. This database may contain standard feature descriptions of various concrete defects, historical detection data, material properties, and other information.

[0141] The second related element vectorization unit synthesizes the features of the current image with the data in the database and converts them into a numerical vector, which is the fourth vector representation. Suppose this fourth vector is a 512-dimensional vector that integrates the features of the current image and the quantitative information of relevant knowledge in the database.

[0142] The server performs the same operation on other images in the first sample building's concrete surface image set and obtains the corresponding fourth vector representation.

[0143] Still taking the old warehouse's concrete floor image as an example, the server obtains its third vector representation and fourth vector representation.

[0144] The server begins to calculate the deviation between the two vectors. For example, by calculating the difference between the elements of the vectors, the Euclidean distance, or the cosine similarity, the degree of deviation is quantified.

[0145] Suppose the calculation result shows a large deviation, which means that the feature representation obtained by the second index vectorization unit alone processing the image is significantly different from the feature representation obtained by the second related element vectorization unit combining database information processing.

[0146] The server adjusts the parameters of the feature extraction component based on this deviation to optimize its performance in multi-task learning. For example, if the deviation indicates that the second index vectorization unit's quantification of crack length is not accurate enough, the server will adjust the relevant parameters to make it more accurate in extracting the features of crack length.

[0147] For the concrete image of the bridge expansion joint, the server also performs similar deviation calculation and parameter adjustment.

[0148] Through such deviation calculation and parameter optimization on a large number of images in the first sample building's concrete surface image set, the feature extraction component can continuously learn and improve, better adapting to the requirements of multi-task learning.

[0149] For example, for a concrete image of a water tower foundation, the optimized feature extraction component can more accurately extract multiple defect features such as cracks, corrosion, and deformation, providing a more reliable basis for subsequent defect detection and classification.

[0150] In the embodiments of the present application, the gating recurrent component is combined with the feature extraction component subjected to multi-task learning to obtain a defect detection model, which can be implemented through the following examples.

[0151] The gating recurrent component is cascaded with the second index vectorization unit subjected to multi-task learning to obtain the defect detection model.

[0152] In the embodiments of the present application, for example, in a large building quality monitoring center, the server is undertaking the important task of constructing a defect detection model.

[0153] The server has completed multi-task learning of the feature extraction component, and the second index vectorization unit therein has the ability to extract key features from the first sample building concrete surface image and vectorize.

[0154] Taking an image of a hydropower dam concrete surface as an example, the second index vectorization unit subjected to multi-task learning can convert defect features such as cracks, holes, and spalling on the dam surface into precise digital vector representations.

[0155] Next, the server cascades the gating recurrent component with the second index vectorization unit subjected to multi-task learning to construct a defect detection model.

[0156] The gating recurrent component is like an intelligent judgment center that can receive vectors output by the second index vectorization unit and make in-depth analysis and reasoning based on the information in these vectors.

[0157] The server first tightly connects the output end of the second index vectorization unit with the input end of the gating recurrent component. In this way, when a new concrete surface image is input, the vectors generated after the second index vectorization unit processes it can be seamlessly transmitted to the gating recurrent component.

[0158] For example, for an image of a port concrete pier, the second index vectorization unit converts features such as fine cracks and erosion marks on the pier surface into vectors, which are immediately delivered to the gating recurrent component.

[0159] After the gating recurrent component receives these vectors, it begins to play its unique role. It will memorize and update the input vector sequence, capturing long-term dependencies and dynamic changes therein.

[0160] Suppose there is a certain pattern in the development of crack features at different positions and times in this image of the port concrete pier, the gating recurrent component can discover this pattern by analyzing the input vector sequence and make more accurate judgments on the type, severity, and development trend of the defect.

[0161] The server continues to process other types of building concrete appearance images, such as a concrete image of a bridge pier of a viaduct. The second index vectorization unit trained by multi-task learning extracts a defect feature vector of the bridge pier surface, which is then passed to the gated recurrent component.

[0162] The gated recurrent component analyzes the continuous input vectors and determines whether the defects on the bridge pier surface are caused by crack expansion due to long-term vehicle vibration or concrete spalling due to environmental erosion.

[0163] In this cascading manner, the server successfully combines the second index vectorization unit and the gated recurrent component to form a model that can efficiently and accurately detect defects on the surface of building concrete.

[0164] For example, for a concrete column image of an underground parking lot of a large shopping mall, the defect detection model can quickly provide a detailed report on the surface defects of the column, including the specific type, location, size of the defects, and the possible impact on the safety of the building structure.

[0165] After continuous input of building concrete appearance images under various different scenarios and conditions for testing and optimization, the server finally obtains a defect detection model with excellent performance and high reliability, providing strong technical support for ensuring the safety and stability of buildings.

[0166] The embodiment of the present application provides a computer device 100, which comprises a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned artificial intelligence-based underwater building concrete surface defect detection method. As shown in Figure 2 Figure 2 The computer device 100 provided by the embodiment of the present application is a structural block diagram. The computer device 100 comprises a memory 111, a processor 112 and a communication unit 113.

[0167] In order to realize the transmission or interaction of data, the memory 111, the processor 112 and the communication unit 113 are directly or indirectly electrically connected with each other. For example, the electrical connection between these elements can be realized by one or more communication buses or signal lines.

[0168] ​The foregoing description, for purposes of explanation, is provided as to specific embodiments and implementations. However, the foregoing description is not intended to be exhaustive or to limit the disclosure to the precise form disclosed. Many modifications and variations are possible in light of the foregoing teaching. These embodiments and implementations were chosen and described in order to best explain the principles of the disclosure and its practical application, to thereby enable others skilled in the art to best utilize the disclosure, and to best enable others skilled in the art to best utilize the disclosure in various embodiments and with various modifications as are suited to the particular situation for each individual application.

Claims

1. A method for detecting apparent defects in concrete of underwater structures based on artificial intelligence, characterized in that, include: Perform aerial and underwater calibration on the preset color camera and the preset monochrome camera; Acquire a first image and a second image of the concrete appearance of an underwater structure taken by a calibrated color camera. The first image has laser stripes, and the second image does not have laser stripes. The laser stripes are generated based on underwater refraction. Acquire a third image of the concrete appearance of an underwater structure taken by a calibrated black-and-white camera; The first image and the third image are matched based on the laser stripes. If the match is successful, a viewpoint cloud image of the concrete surface of the underwater structure is constructed based on the first image and the third image. Color enhancement is performed based on the second image to obtain the viewpoint cloud image of the concrete surface of the target underwater structure. Based on the apparent defect detection model, defect identification is performed on the surface cloud image of the concrete of the target underwater structure to obtain the defect detection result corresponding to the surface cloud image of the concrete of the target underwater structure.

2. The method according to claim 1, characterized in that, The defect identification of the target underwater structure's concrete surface cloud image based on the apparent defect detection model, to obtain the defect detection result corresponding to the target underwater structure's concrete surface cloud image, includes: A first sample building concrete appearance image set and a second sample building concrete appearance image set are obtained. The first sample building concrete appearance image set includes first sample building concrete appearance images, and the second sample building concrete appearance image set includes a second sample building concrete appearance image set configured with sample target values. The second sample building concrete appearance image set is classified into a target defect category set. The main defect detection model is obtained by training the first sample building concrete appearance image set with a multi-task learning method and the second sample building concrete appearance image set with a reinforcement learning method. Obtain the defect detection model, wherein the defect detection model is the original model whose model parameters have not been trained; Using the first sample building concrete appearance image as the teacher model and the main defect detection model as the teacher model, knowledge transfer is performed on the model parameters of the secondary defect detection model to obtain the appearance defect detection model; The apparent defect detection model is used to identify defects in the surface cloud image of the concrete of the target underwater structure, and the defect detection result of the surface cloud image of the concrete of the target underwater structure in the target defect category set is obtained.

3. The method according to claim 2, characterized in that, The step of using the first sample building concrete appearance image as a teacher model and performing knowledge transfer on the model parameters of the defect detection model to obtain the appearance defect detection model includes: The main defect detection model is used to identify defects in the concrete appearance images of the first sample building in the first sample building concrete appearance image set, and to obtain the undetermined target value of the category in the target defect category set corresponding to the concrete appearance image of the first sample building. By performing defect identification on the concrete appearance images of the first sample building in the first sample building concrete appearance image set using the defect detection model, the identification result corresponding to the concrete appearance image of the first sample building is obtained. The model parameters of the defect detection model are optimized based on the deviation between the target value to be determined and the identification result to obtain the apparent defect detection model.

4. The method according to claim 3, characterized in that, The optimization of the model parameters of the defect detection model based on the deviation between the target value to be determined and the identification result to obtain the apparent defect detection model includes: Obtain the credibility index of the target value to be determined; The deviation between the target value to be determined and the identification result under the credibility index is determined, and the model parameters of the defect detection model are optimized based on the deviation to obtain the apparent defect detection model.

5. The method according to claim 3, characterized in that, The step of identifying defects in the concrete appearance images of the first sample building in the first sample building concrete appearance image set using the main defect detection model, and obtaining the undetermined target value of the category in the target defect category set corresponding to the concrete appearance image of the first sample building, includes: The main defect detection model is used to identify defects in the concrete appearance images of the first sample building in the first sample building concrete appearance image set, and the confidence level of the first sample building concrete appearance image in the target defect category set is obtained. Based on the confidence level, the target value corresponding to the concrete appearance image of the first sample building is determined from the target defect category set.

6. The method according to claim 3, characterized in that, The defect detection model includes a first index vectorization unit and a first related element vectorization unit; The method further includes: The first sample building concrete appearance image is vectorized by the first index vectorization unit to obtain a first vector representation; The first relevant element vectorization unit obtains a second vector representation of the first sample building concrete appearance image and the data in the first preset database; Based on the deviation between the first vector representation and the second vector representation, knowledge transfer is performed on the defect detection model to obtain the apparent defect detection model.

7. The method according to claim 2, characterized in that, The step of training the main defect detection model using the first set of sample building concrete appearance images with a multi-task learning method and the second set of sample building concrete appearance images with a reinforcement learning method includes: Multi-task learning is performed on the feature extraction component based on the concrete appearance images of the first sample building in the first sample building concrete appearance image set. By combining the gated recurrent component with the feature extraction component that has undergone multi-task learning, a defect detection model is obtained. The gated recurrent component is used to perform appearance defect detection in the target defect category set. The defect detection model is obtained by performing reinforcement learning on the defect detection model using the second sample building concrete appearance image set and the sample target value.

8. The method according to claim 7, characterized in that, The feature extraction component includes a second index vectorization unit and a second related element vectorization unit; The feature extraction component, based on the first sample building concrete appearance images in the first sample building concrete appearance image set, performs multi-task learning, including: The first sample building concrete appearance image is vectorized by the second index vectorization unit to obtain a third vector representation; The second related element vectorization unit obtains a fourth vector representation of the first sample building concrete appearance image and the data in the second preset database; The feature extraction component performs multi-task learning based on the deviation between the third vector representation and the fourth vector representation.

9. The method according to claim 8, characterized in that, The combination of the gated recurrent component and the feature extraction component learned through multi-task training to obtain the defect detection model includes: The gated loop component is concatenated with the second index vectorization unit, which has undergone multi-task learning, to obtain the defect detection model.

10. A server system, characterized in that, Includes a server, the server being used to perform the method according to any one of claims 1-9.

Images