Multi-objective optimization dam crack detection enhancement method and system based on expert system

Abstract

The present invention discloses a multi-objective optimization dam crack detection enhancement method and system based on an expert system, which relates to intelligent monitoring and detection technology, including inputting raw image data and preprocessing, constructing and training a hybrid expert model MoE, and performing initial coarse-grained detection through a gateway module of a gate control unit; performing link adaptation between the gateway module and the hybrid expert model MoE, training and adjusting the parameters of the gateway module and the hybrid expert model MoE according to the adaptation result through an optimization algorithm, and activating the hybrid expert model MoE for secondary fine-grained detection; fusing the output results of the fine-grained detection of the hybrid expert model MoE, and generating and providing a final crack detection report. The present invention improves the efficiency and accuracy of multi-objective crack detection, uses multi-objective optimization and Bayesian optimization to finely adjust model parameters, improves the performance of the model, and increases its adaptability and robustness in different detection scenarios, reflecting the advancement of the present invention in the field of crack detection technology.

Description

Technical Field

[0001] The present invention relates to intelligent monitoring and detection technology, and in particular to a multi-objective optimization dam crack detection enhancement method and system based on an expert system. Background Art

[0002] In the water conservancy industry, the integrity of dams and related facilities plays a vital role in ensuring regional water resources security, flood control, and providing necessary support for surrounding ecology and human activities. The crack detection task of these structures has therefore become a key link in ensuring the safe operation of water conservancy facilities. Although artificial intelligence and image processing technology have made significant progress in the field of crack detection in recent years, they still face a series of challenges in their specific application in the water conservancy industry.

[0003] 1. High repetitiveness of image features. The surfaces of dams and water conservancy facilities often have highly consistent texture and structural features, which leads to high repetitiveness of features extracted from these images. This high repetitiveness of features will cause a large amount of redundant information, making it difficult for the crack detection model to accurately distinguish between cracks and non-crack areas, affecting the accuracy and efficiency of detection.

[0004] 2. The existence of multiple types of cracks. In the detection tasks of dams and water conservancy facilities, there may be multiple types of cracks, such as small cracks, extended cracks, structural cracks, etc. These different types of cracks vary in shape, size and importance, which brings additional complexity to the detection task. The model needs to have a high degree of sensitivity and accuracy to distinguish and accurately detect various types of cracks.

[0005] 3. Lack of high-quality annotated samples. Accurate crack detection model training requires a large number of high-quality, accurately annotated training samples. However, in the water conservancy industry, especially for dams and large water conservancy facilities, it is challenging to obtain a large number of high-quality annotated images. This is mainly because obtaining these images on the spot requires professional equipment and personnel, and the annotation work itself also requires professional knowledge, which is a time-consuming and costly task.

[0006] Despite the above challenges, crack detection plays an irreplaceable role in the maintenance and safety monitoring of water conservancy facilities. Early detection and timely repair of cracks can greatly reduce potential safety hazards, avoid catastrophic consequences, and protect public safety and economic interests. Therefore, the development of efficient and accurate crack detection technology has great practical significance and application value for the water conservancy industry. In this context, exploring and implementing more advanced crack detection methods, especially those that can handle high feature repeatability, multi-type crack detection, and maintain high performance with limited labeled samples, has become an important research direction in this field. Summary of the invention

[0007] In view of the above-mentioned existing problems, the present invention improves the accuracy, efficiency and reliability of dam crack detection by adopting a hybrid expert model (MoE) and a multi-objective optimization algorithm, realizes automatic detection and classification of dam cracks, and detects in scenes with complex backgrounds and coexistence of multiple crack types. The crack detection system is optimized so that it can better adapt to different types of cracks and dam surface conditions, enhances the system's adaptability to image quality and its ability to integrate output results of different expert models, thereby better guiding dam maintenance and repair work.

[0008] In order to solve the above technical problems, a multi-objective optimization dam crack detection enhancement method based on expert system is proposed, including:

[0009] Input the original image data and perform preprocessing, build and train the hybrid expert model MoE, perform initial coarse-grained detection through the gateway module of the gating unit, detect multiple types of cracks on the input image to be detected through the multi-target detection model, output the crack detection results and corresponding probability values, and judge the further operation process according to the corresponding crack detection results, including no cracks detected, a single crack type, and multiple crack types. When it is judged that multiple crack types are detected, match the expert detection model in the hybrid expert model according to the crack type, and select the expert detection model with the highest matching degree for association according to the matching degree calculated by the expert detection model in the hybrid expert model. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model. The gating unit trains the crack types in the image through the SSD architecture, taking into account the unevenness of the crack types. The problem of balance is solved and the weight distribution is adjusted; the link adaptation of the gateway module and the hybrid expert model MoE is carried out, and the parameters of the gateway module and the hybrid expert model MoE are adjusted according to the adaptation results through the optimization algorithm, and the hybrid expert model MoE is activated for secondary fine-grained detection. The judgment error of the expert model and the gate control unit is judged according to the detection results of the secondary fine-grained detection, and the corresponding detection results are retained according to the error results; the output results of the fine-grained detection of the hybrid expert model MoE are fused, and the results are fused according to the crack position and crack type. The first round of judgment is made through the distance position of the crack to determine whether the distance between the crack positions detected by different models is the same crack. If it is the same crack, the position and size information are averaged and merged, and a second round of judgment is made to determine whether the crack types are consistent. If the results are consistent, they are directly fused to generate and provide a final crack detection report.

[0010] As a preferred solution of the expert system-based multi-objective optimization dam crack detection enhancement method described in the present invention, the original image data includes: using a high-resolution camera mounted on a drone to collect dam surface images to make the geocoding of the image data accurate, and using a mobile robot carrying a high-definition camera to perform close-range image collection of image data of suspected cracks at key locations and in images collected by the drone.

[0011] The preprocessing includes inputting the original image into the system, performing image denoising, enhancement, scaling and brightness enhancement to form an image to be detected and inputting the image into the gate control unit.

[0012] As a preferred solution of the expert system-based multi-objective optimization dam crack detection enhancement method described in the present invention, the initial coarse-grained detection includes detecting multiple types of cracks on the input image to be detected through a multi-objective detection model, and outputting the crack detection results and the corresponding probability value p.

[0013] When the crack detection result indicates that no cracks are detected in the image, the preprocessing module is re-entered to perform brightness enhancement.

[0014] When the crack detection result indicates that there is only a single crack type in the image and the corresponding probability p is greater than 90%, the result is output directly.

[0015] When the crack detection result indicates that multiple crack types are detected in the image , and the corresponding probability is , according to the crack type, match the expert detection model in the hybrid expert model:

[0016]

[0017] Among them, Q is the matching degree of the current expert detection model in the hybrid expert model, M is the number of crack types detected, is the detection probability of the j-th crack, is the characteristic type of the j-th crack, is the expert detection model corresponding to the j-th type of crack, is the deviation between the characteristic type of the j-th crack and the standard characteristic type, is the variance of the fracture characteristic type, The deviation between the expert detection model of the j-th crack and the standard model, is the variance of the crack expert detection model, is the average value of all crack detection probabilities, Calculate the matching degree between crack type and expert detection model:

[0018]

[0019] According to the matching degree of Q, the expert detection model with the highest matching degree is selected for association. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model:

[0020]

[0021] The gate control unit uses the SSD architecture to train the crack types in the image, considers the imbalance of crack types, and adjusts the weight distribution.

[0022] in, is the weight of the expert model selected for the jth crack type, is the detection probability of the j1-th type of crack, is the variance of the Gaussian distribution, is the matching degree between the j-th crack type and the expert model, is the maximum value of the matching degree between all crack types and the expert model, k is the steepness parameter of the sigmoid function, and are the standard deviations of the crack feature type and the crack expert detection model, respectively.

[0023] As a preferred solution of the multi-objective optimization dam crack detection enhancement method based on expert system described in the present invention, wherein: the adjustment weight allocation includes adjusting the loss function using a weight allocation mechanism based on sample relative frequency, specifically, for each crack type j, according to the relative frequency in the data set , update the weight of each category , so that crack types with a frequency of less than 20% have a high weight:

[0024]

[0025] in, It is to adjust the parameters and output the probability of each crack type according to the detection model , adjust the classification loss , optimize the multi-target detection model:

[0026]

[0027] Where N is the number of image samples, M is the number of crack types, i represents the sample index, and j represents the variable index of the crack type. is the true label, =0, 1, indicating whether sample i belongs to type j.

[0028] As a preferred solution of the multi-objective optimization dam crack detection enhancement method based on expert system described in the present invention, the optimization algorithm includes adjusting the parameter coordination between the gate control unit and all expert models through the multi-objective optimization algorithm, stipulating that the goal of the gateway module is to maximize the overall detection performance of the system, and Key crack detection priorities determined by the function:

[0029]

[0030] By adjusting the parameters of the gateway module, the key crack data are assigned to the expert model that is best at detecting key cracks, where T is the set of key crack types, is the detection accuracy corresponding to type j.

[0031] Gateway module through Overall detection performance of the function feedback system:

[0032]

[0033] in, , , , , They represent the average accuracy, average intersection-over-union ratio, average precision, average recall and average correctness respectively, and the coefficient , , , and It is a weight parameter that adjusts the influence of the performance index in the overall objective function.

[0034] pass Functions and Function definition objective function for multi-objective optimization:

[0035]

[0036] Among them, S is the objective function of multi-objective optimization, is the weight coefficient of the key crack detection priority, is the adjustment parameter for the key crack detection priority, For ideal critical crack detection priorities, is the normalization constant, is the weight coefficient of the overall detection performance of the system, It is the adjustment parameter for the overall detection performance of the system.

[0037] Fine-tuning via Bayesian optimization to select initial hyperparameter combinations and , calculate the objective function value ,in, and Represent the hyperparameters of the gateway module and the expert model respectively. Use the observed hyperparameter combination and the corresponding objective function to train the Gaussian process model and select the hyperparameters to maximize the acquisition function ,Will As the next hyperparameter combination that improves performance, the iterative update stops when the performance reaches a predetermined level, and the parameters of the gating unit and MoE are updated according to the optimization results.

[0038] As a preferred solution of the multi-objective optimization dam crack detection enhancement method based on expert system described in the present invention, the secondary fine-grained detection includes, after the image is matched by the gate control unit, activating the selected corresponding expert model to perform secondary fine-grained detection:

[0039]

[0040] when = 0, it means that the judgment error between the expert model and the gate control unit is the smallest, and the detection result of the initial coarse-grained detection is retained. =1, it means that the judgment error between the expert model and the gate control unit is the largest, and the detection result of the expert model is retained.

[0041] Among them, Δ is the total discrimination error of the secondary fine-grained detection model, is the number of image samples, is the i-th image sample, is the weight parameter of the gating unit, For the gate control unit to image The matching results of For the i-th expert model to image The detection output, For images The crack area in is the crack type discrimination function, is the characteristic type of the j-th type of crack, that is, the crack feature extraction result.

[0042] As a preferred solution of the multi-objective optimization dam crack detection enhancement method based on expert system described in the present invention, the output result fusion includes, after the crack detection result of the image to be detected is obtained through the hybrid expert model MoE, the result fusion is performed according to the crack position and crack type, and the first round of judgment is first performed based on the distance position of the crack, specifically, expert models A and B are specified, and the detection results are crack a and crack b.

[0043] Model A in position The information of crack a detected is ,in, are the coordinates of the center of crack a, and are the width and height of crack a, It is the type of crack a.

[0044] Model B is in position The information of crack b detected is ,in, are the coordinates of the center of crack b, and are the width and height of crack b, It is the type of crack b.

[0045] if and spacing If the crack is less than 5 cm, the detection results of model A and model B are considered to point to the same crack, and the average value is taken to combine the position and size:

[0046]

[0047]

[0048]

[0049]

[0050] when and When the spacing is less than 5 cm, a second round of judgment is conducted based on the crack type. and If the two cracks are consistent, the characteristics of the two cracks are highly overlapped and they are directly merged. and If they are inconsistent, the confidence of the crack type is calculated and the crack type with high confidence is selected as the fused crack type, t is the crack type, t∈T.

[0051] when and When the spacing is not less than 5 cm, it is judged as an independent crack and does not need to be fused. It is placed in the independent crack image library and the distribution pattern is compared with the images in the library. When a regular distribution is shown, the two types of cracks are considered to be related and removed from the image library for judgment by manual experts. When there is no regular distribution, it is retained in the independent crack image library.

[0052] Another object of the present invention is to provide a multi-objective optimization dam crack detection enhancement system based on an expert system, which improves the accuracy, efficiency and automation level of dam crack detection, ensuring the safety of dam structures and the effective use of water resources.

[0053] As a preferred solution of the expert system-based multi-objective optimization dam crack detection enhancement system described in the present invention, it is characterized by including a preprocessing module, a gating unit, and a detection module.

[0054] The preprocessing module performs preliminary processing on the input dam image, including image denoising, enhancement, and scaling.

[0055] The gate control unit includes a gateway module and a pre-classification and recognition module. The gateway module performs a preliminary analysis on the pre-processed image and dynamically allocates the image to a corresponding expert model for further detection according to the characteristics of the cracks in the image.

[0056] The pre-classification and recognition module performs preliminary classification of multiple types of cracks on the input samples through multi-target detection, outputs the detection results of each crack, and selectively activates the subsequent expert model to perform secondary fine-grained detection.

[0057] The detection module includes a hybrid expert model MoE module, an optimization module and a post-fusion module. The hybrid expert model MoE module uses a special expert model to perform in-depth analysis and detection on different types of cracks.

[0058] The optimization module uses multi-objective optimization and Bayesian optimization algorithms to adjust the coordination between the gating unit and each expert model, as well as the parameters within each expert model, to ensure the optimal performance of the overall system.

[0059] The post-fusion module collects the output results of all activated expert models, post-fuses the output results of multiple MoE experts based on crack type and crack location, and integrates the outputs of all experts to generate the final detection result.

[0060] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and is characterized in that when the processor executes the computer program, the steps of the method described in the method for enhancing dam crack detection based on multi-objective optimization of an expert system are implemented.

[0061] A computer-readable storage medium stores a computer program, wherein when the computer program is executed by a processor, the steps of the method described in the expert system-based multi-objective optimization dam crack detection enhancement method are implemented.

[0062] Beneficial effects of the invention: The invention includes the specific implementation of the hybrid expert model (MoE) architecture, in which each expert model is optimized for different types of cracks; the design and function of the front gateway module for preliminary analysis of images and decision on which expert models to activate for further detection; the application of multi-objective optimization and Bayesian optimization methods in model training and parameter adjustment; and the fusion strategy of crack detection results based on model output. These constitute the core part of the system and are the key to its efficient and accurate crack detection.

[0063] The present invention innovatively improves the efficiency and accuracy of multi-target crack detection by combining the MoE architecture with the front gateway module. In addition, the present invention uses multi-objective optimization and Bayesian optimization to fine-tune the model parameters, which not only improves the performance of the model, but also increases its adaptability and robustness in different detection scenarios. The adjustment strategy of the loss function, especially the use of different weights for different crack types, effectively solves the problem of class imbalance, which is a significant improvement in traditional crack detection methods. These innovations together constitute the core value of the present invention and demonstrate its advancement in the field of crack detection technology. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative work, among which:

[0065] Figure 1 An overall flow chart of a multi-objective optimization dam crack detection enhancement method based on an expert system provided in one embodiment of the present invention.

[0066] Figure 2 A technical architecture diagram of a multi-objective optimization dam crack detection enhancement method based on an expert system provided for an embodiment of the present invention.

[0067] Figure 3 A hybrid expert model architecture of a multi-objective optimization dam crack detection enhancement method based on an expert system is provided in one embodiment of the present invention.

[0068] Figure 4 A post-fusion strategy for the output results of an expert model of a multi-objective optimization dam crack detection enhancement method based on an expert system is provided in one embodiment of the present invention.

[0069] Figure 5 A system solution flow chart of a multi-objective optimization dam crack detection enhancement system based on an expert system provided in one embodiment of the present invention. DETAILED DESCRIPTION

[0070] In order to make the above-mentioned purposes, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the drawings of the specification. Obviously, the described embodiments are part of the embodiments of the present invention, but not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary persons in the art without creative work should fall within the scope of protection of the present invention.

[0071] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0072] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor is it an embodiment that is mutually exclusive with other embodiments, either individually or selectively.

[0073] The present invention is described in detail with reference to schematic diagrams. When describing the embodiments of the present invention, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.

[0074] At the same time, in the description of the present invention, it should be noted that the directions or positional relationships indicated by the terms "upper, lower, inner and outer" are based on the directions or positional relationships shown in the drawings, and are only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific direction, be constructed and operated in a specific direction, and therefore cannot be understood as limiting the present invention. In addition, the terms "first, second or third" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance.

[0075] In the present invention, unless otherwise clearly specified and limited, the terms "install, connect, connect" should be understood in a broad sense, for example: it can be a fixed connection, a detachable connection or an integral connection; it can also be a mechanical connection, an electrical connection or a direct connection, or it can be indirectly connected through an intermediate medium, or it can be the internal communication of two components. For ordinary technicians in this field, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0076] Example 1, reference Figure 1-Figure 4, which is the first embodiment of the present invention, provides a multi-objective optimization dam crack detection enhancement method based on an expert system, comprising:

[0077] S1: Input the original image data and perform preprocessing, build and train the hybrid expert model MoE, perform initial coarse-grained detection through the gateway module of the gating unit, detect multiple types of cracks on the input image to be detected through the multi-target detection model, output the crack detection results and corresponding probability values, and judge the further operation process according to the corresponding crack detection results, including no cracks detected, a single crack type, and multiple crack types. When it is judged that multiple crack types are detected, match the expert detection model in the hybrid expert model according to the crack type, and select the expert detection model with the highest matching degree for association according to the matching degree calculated by the expert detection model in the hybrid expert model. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model. The gating unit trains the crack types in the image through the SSD architecture, considers the imbalance problem of crack types, and adjusts the weight distribution.

[0078] Furthermore, high-resolution cameras mounted on UAVs were used to collect images of the dam surface to enable accurate geocoding of image data, and mobile robots equipped with high-definition cameras were used to collect close-range image data of key locations and suspected cracks in images collected by UAVs.

[0079] The original image is input into the system, and after image denoising, enhancement, scaling and brightness enhancement, the image to be detected is formed and input into the gating unit.

[0080] It should be noted that the initial coarse-grained detection includes detecting multiple types of cracks on the input image to be detected through a multi-target detection model, and outputting the crack detection results and the corresponding probability values ​​p.

[0081] When the crack detection result indicates that no cracks are detected in the image, the preprocessing module is re-entered to perform brightness enhancement.

[0082] When the crack detection result indicates that there is only a single crack type in the image and the corresponding probability p is greater than 90%, the result is output directly.

[0083] When the crack detection result indicates that multiple crack types are detected in the image , and the corresponding probability is ,For example, Figure 3 The hybrid expert model architecture designed by the present invention is demonstrated, wherein for different types of cracks, expert models with high detection accuracy are applied for association. For example, the YOLO architecture is used for cracks of the tortoise-type, and the SSD model architecture is applied for horizontal and vertical cracks.

[0084] Different expert models are applied to different types of cracks, and the corresponding detection accuracy results are shown in Table 1.

[0085] Table 1 Different expert models are good at detecting different types of cracks

[0086] Expert Model Adaptive crack detection Detection accuracy First expert detection model Crack Type A 99.2% Second expert detection model Crack Type B 99.7% The third expert detection model Crack Type C 99.4%

[0087] According to the crack type, match the expert detection model in the hybrid expert model:

[0088]

[0089] Among them, Q is the matching degree of the current expert detection model in the hybrid expert model, M is the number of crack types detected, is the detection probability of the j-th type of crack, is the characteristic type of the j-th crack, is the expert detection model corresponding to the j-th type of crack, is the deviation between the characteristic type of the j-th crack and the standard characteristic type, is the variance of the fracture characteristic type, The deviation between the expert detection model of the j-th crack and the standard model, is the variance of the crack expert detection model, is the average value of all crack detection probabilities, Calculate the matching degree between crack type and expert detection model:

[0090]

[0091] According to the matching degree of Q, the expert detection model with the highest matching degree is selected for association. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model:

[0092]

[0093] in, is the weight of the expert model selected for the jth crack type, is the detection probability of the j1-th type of crack, is the variance of the Gaussian distribution, is the matching degree between the j-th crack type and the expert model, is the maximum value of the matching degree between all crack types and the expert model, k is the steepness parameter of the sigmoid function, and are the standard deviations of the crack feature type and the crack expert detection model, respectively.

[0094] The gate control unit uses the SSD architecture to train the crack types in the image, considers the imbalance of crack types, and adjusts the weight distribution.

[0095] The loss function is adjusted using a weight distribution mechanism based on the relative frequency of samples. Specifically, for each crack type j, the relative frequency of the crack type in the dataset is , update the weight of each category , so that crack types with a frequency of less than 20% have a high weight:

[0096]

[0097] in, It is to adjust the parameters and output the probability of each crack type according to the detection model , adjust the classification loss , optimize the multi-target detection model:

[0098]

[0099] Where N is the number of image samples, M is the number of crack types, i represents the sample index, and j represents the variable index of the crack type. is the true label, =0, 1, indicating whether sample i belongs to type j.

[0100] Preferably, Top-K gating is selected to selectively activate the subsequent expert model for secondary fine-grained detection

[0101] S2: Perform link adaptation between the gateway module and the hybrid expert model MoE. Use the optimization algorithm to train and adjust the parameters of the gateway module and the hybrid expert model MoE according to the adaptation results and activate the hybrid expert model MoE for secondary fine-grained detection. According to the detection results of the secondary fine-grained detection, the judgment error size of the expert model and the gating unit is judged, and the corresponding detection result is retained according to the error result.

[0102] Furthermore, the link adaptation between the gating unit and the hybrid expert model enhances the combination and collaboration capabilities of the two by optimizing the algorithm fine-tuning.

[0103] The multi-objective optimization algorithm is used to adjust the parameters between the gate control unit and all expert models, and the goal of the gateway module is to maximize the overall detection performance of the system. Key crack detection priorities determined by the function:

[0104]

[0105] By adjusting the parameters of the gateway module, the key crack data are assigned to the expert model that is best at detecting key cracks, where T is the set of key crack types, is the detection accuracy corresponding to type j.

[0106] Gateway module through Overall detection performance of the function feedback system:

[0107]

[0108] in, , , , , They represent the average accuracy, average intersection-over-union ratio, average precision, average recall and average correctness respectively, and the coefficient , , , and It is a weight parameter that adjusts the influence of the performance index in the overall objective function.

[0109] pass Functions and Function definition objective function for multi-objective optimization:

[0110]

[0111] Among them, S is the objective function of multi-objective optimization, is the weight coefficient of the key crack detection priority, is the adjustment parameter for the key crack detection priority, For ideal critical crack detection priorities, is the normalization constant, is the weight coefficient of the overall detection performance of the system, It is the adjustment parameter for the overall detection performance of the system.

[0112] Through this design, and They can jointly guide the multi-objective optimization process, not only focusing on the detection of critical cracks, but also comprehensively considering the overall performance, so as to achieve the optimal crack detection performance under limited resources. During the fine-tuning process, Bayesian optimization is used to adjust the hyperparameters of the routing model and the expert model to find the best configuration.

[0113] Fine-tuning via Bayesian optimization to select initial hyperparameter combinations and , calculate the objective function value ,in, and Represent the hyperparameters of the gateway module and the expert model respectively. Use the observed hyperparameter combination and the corresponding objective function to train the Gaussian process model and select the hyperparameters to maximize the acquisition function ,Will As the next hyperparameter combination that improves performance, the iterative update stops when the performance reaches a predetermined level, and the parameters of the gating unit and MoE are updated according to the optimization results.

[0114] In this way, Bayesian optimization can help us find effective configurations in high-dimensional and possibly non-convex hyperparameter spaces, thereby optimizing the performance of the entire system. This process helps to achieve effective collaboration between the routing model and the MoE expert system, ultimately improving the accuracy and efficiency of crack detection.

[0115] It should be noted that the secondary fine-grained detection is to activate the selected corresponding expert model for secondary fine-grained detection after the image is matched by the gated unit:

[0116]

[0117] when = 0, it means that the judgment error between the expert model and the gate control unit is the smallest, and the detection result of the initial coarse-grained detection is retained. =1, it means that the judgment error between the expert model and the gate control unit is the largest, and the detection result of the expert model is retained.

[0118] Among them, Δ is the total discrimination error of the secondary fine-grained detection model, is the number of image samples, is the i-th image sample, is the weight parameter of the gating unit, For the gate control unit to image The matching results of For the i-th expert model to image The detection output, For images The crack area in is the crack type discrimination function, is the characteristic type of the j-th type of crack, that is, the crack feature extraction result.

[0119] Compared with the target detection expert models in the hybrid expert model, the detection model in the gated unit is only responsible for coarse-grained detection, and is not as accurate as the expert model in identifying specific types of cracks. In practical applications, the gated unit determines which expert models to activate. The input of the gated unit is the image to be detected, and the output is the detected crack type and its corresponding probability value. The subsequent expert model is activated based on Top-K gating, and the expert model performs more fine-grained crack detection on the image.

[0120] S3: The output results of the fine-grained detection of the hybrid expert model MoE are integrated according to the crack location and crack type. The first round of judgment is made based on the distance position of the crack to determine whether the distance between the crack locations detected by different models is the same crack. If it is the same crack, the position and size information are averaged and merged. A second round of judgment is performed to determine whether the crack types are consistent. If the results are consistent, they are directly integrated to generate and provide a final crack detection report.

[0121] Furthermore, after the crack detection results of the image to be detected are obtained through the hybrid expert model MoE, the results are fused according to the crack position and crack type. The first round of judgment is made through the distance position of the crack. Specifically, expert models A and B are specified, and the detection results are crack a and crack b.

[0122] Model A in position The information of detecting crack a is ,in, are the coordinates of the center of crack a, and are the width and height of crack a, It is the type of crack a.

[0123] Model B is in position The information of crack b detected is ,in, are the coordinates of the center of crack b, and are the width and height of crack b, It is the type of crack b.

[0124] if and spacing If the crack is less than 5 cm, the detection results of model A and model B are considered to point to the same crack, and the average value is taken to combine the position and size:

[0125]

[0126]

[0127]

[0128]

[0129] when and When the spacing is less than 5 cm, a second round of judgment is conducted based on the crack type. and If the two cracks are consistent, the characteristics of the two cracks are highly overlapped and they are directly merged. and If they are inconsistent, the confidence of the crack type is calculated and the crack type with high confidence is selected as the fused crack type, t is the crack type, t∈T.

[0130] when and When the spacing is not less than 5 cm, it is judged as an independent crack and does not need to be fused. It is placed in the independent crack image library and the distribution pattern is compared with the images in the library. When a regular distribution is shown, the two types of cracks are considered to be related and removed from the image library for judgment by manual experts. When there is no regular distribution, it is retained in the independent crack image library.

[0131] In this way, we can combine the detection results of different expert models to improve the overall detection accuracy and reliability. At the same time, this approach allows the system to effectively manage and allocate limited computing resources while maintaining high accuracy.

[0132] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

[0133] Embodiment 2, the second embodiment of the present invention, is different from the first two embodiments in that:

[0134] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server, or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk, etc., which can store program codes.

[0135] The logic and / or steps represented in the flowchart or otherwise described herein, for example, can be considered as an ordered list of executable instructions for implementing logical functions, and can be embodied in any computer-readable medium for use by an instruction execution system, device or apparatus (such as a computer-based system, a system including a processor, or other system that can fetch instructions from an instruction execution system, device or apparatus and execute instructions), or in conjunction with such instruction execution systems, devices or apparatuses. For the purposes of this specification, "computer-readable medium" can be any device that can contain, store, communicate, propagate or transmit a program for use by an instruction execution system, device or apparatus, or in conjunction with such instruction execution systems, devices or apparatuses.

[0136] More specific examples of computer-readable media (a non-exhaustive list) include the following: an electrical connection with one or more wires (electronic device), a portable computer disk case (magnetic device), a random access memory (RAM), a read-only memory (ROM), an erasable and programmable read-only memory (EPROM or flash memory), an optical fiber device, and a portable compact disk read-only memory (CDROM). In addition, the computer-readable medium may even be a paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium, followed by editing, deciphering or, if necessary, processing in another suitable manner, and then stored in a computer memory.

[0137] It should be understood that the various parts of the present invention can be implemented by hardware, software, firmware or a combination thereof. In the above-mentioned embodiments, multiple steps or methods can be implemented by software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented by hardware, as in another embodiment, it can be implemented by any one of the following technologies known in the art or their combination: a discrete logic circuit having a logic gate circuit for implementing a logic function for a data signal, a dedicated integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0138] Example 3, reference Figure 5 , which is the third embodiment of the present invention, provides a multi-objective optimization dam crack detection enhancement system based on an expert system, including a preprocessing module, a gating unit, and a detection module.

[0139] The preprocessing module performs preliminary processing on the input dam image, including image denoising, enhancement, and scaling.

[0140] The gate control unit includes a gateway module and a pre-classification and recognition module. The gateway module performs a preliminary analysis on the pre-processed image and dynamically allocates the image to a corresponding expert model for further detection according to the characteristics of the cracks in the image.

[0141] The pre-classification and recognition module performs preliminary classification of multiple types of cracks on the input samples through multi-target detection, outputs the detection results of each crack, and selectively activates the subsequent expert model to perform secondary fine-grained detection.

[0142] Training and deployment process of the pre-classification recognition module

[0143] Training process: classify and annotate the input images, indicate which images should be processed by which expert models, and select or design a lightweight classification model as the gateway module, such as Figure 4 The SSD model shown uses labeled data to train the gateway model, enabling it to decide which expert models to activate based on the features of the input image.

[0144] The deployment process deploys the trained gateway model at the front end of the system to ensure that the output of the gateway model can correctly guide the expert model in the MoE architecture to be activated.

[0145] The linkage and adaptation between the front-end classification and recognition module and MoE, the output of the gateway module needs to be adapted to the input of the MoE architecture to ensure that the correct expert model is activated; the gateway model is adjusted through the feedback mechanism so that its output more accurately matches the requirements of the MoE architecture.

[0146] The detection module includes a hybrid expert model MoE module, an optimization module and a post-fusion module. The hybrid expert model MoE module uses a special expert model to perform in-depth analysis and detection on different types of cracks.

[0147] The optimization module uses multi-objective optimization and Bayesian optimization algorithms to adjust the coordination between the gating unit and each expert model, as well as the parameters within each expert model, to ensure the optimal performance of the overall system.

[0148] The post-fusion module collects the output results of all activated expert models, post-fuses the output results of multiple MoE experts based on crack type and crack location, and integrates the outputs of all experts to generate the final detection result.

[0149] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.

Claims

1. A multi-objective optimization dam crack detection enhancement method based on an expert system, characterized by: include, Input the original image data and perform preprocessing, build and train the hybrid expert model MoE, perform initial coarse-grained detection through the gateway module of the gating unit, detect multiple types of cracks on the input image to be detected through the multi-target detection model, output the crack detection results and corresponding probability values, and judge the further operation process according to the corresponding crack detection results, including no cracks detected, a single crack type, and multiple crack types. When it is judged that multiple crack types are detected, match the expert detection model in the hybrid expert model according to the crack type, and select the expert detection model with the highest matching degree for association according to the matching degree calculated by the expert detection model in the hybrid expert model. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model. The gating unit trains the crack types in the image through the SSD architecture, considers the imbalance problem of crack types, and adjusts the weight distribution; Perform link adaptation between the gateway module and the hybrid expert model MoE, adjust the parameters of the gateway module and the hybrid expert model MoE according to the adaptation results through the optimization algorithm, activate the hybrid expert model MoE for secondary fine-grained detection, judge the judgment error size of the expert model and the gating unit according to the detection results of the secondary fine-grained detection, and retain the corresponding detection results according to the error results; The output results of the fine-grained detection of the hybrid expert model MoE are fused, and the results are fused according to the crack location and crack type. The first round of judgment is made by the distance position of the crack to determine whether the distance between the crack locations detected by different models is the same crack. If it is the same crack, the position and size information are averaged and merged. A second round of judgment is performed to determine whether the crack types are consistent. If the results are consistent, they are directly merged to generate and provide a final crack detection report. The initial coarse-grained detection includes detecting multiple types of cracks on the input image to be detected through a multi-target detection model, and outputting crack detection results and corresponding probability values ​​p; When the crack detection result indicates that no cracks are detected in the image, the preprocessing module is re-entered to perform brightness enhancement; When the crack detection result indicates that there is only a single crack type in the image and the corresponding probability p is greater than 90%, the result is directly output; When the crack detection result indicates that multiple crack types M = {crack 1, crack 2, ..., crack M} are detected in the image, and the corresponding probabilities are {p1, p2, ..., p M }, according to the crack type, match the expert detection model in the hybrid expert model: Among them, Q is the matching degree of the current expert detection model in the hybrid expert model, M is the number of crack types detected, and p j is the detection probability of the j-th crack, T j is the characteristic type of the jth crack, E j is the expert detection model corresponding to the j-th type of crack, ΔT j is the deviation between the characteristic type of the j-th crack and the standard characteristic type, σ T is the variance of the crack characteristic type, ΔE j is the deviation between the expert detection model of the j-th type of crack and the standard model, σ E is the variance of the crack expert detection model, is the average value of all crack detection probabilities, g(T j ,E j Calculate the matching degree between crack type and expert detection model: According to the matching degree of Q, the expert detection model with the highest matching degree is selected for association. The gateway module of the gating unit assigns the weight of each expert model according to the crack probability value through the multi-target detection model: The gate control unit uses the SSD architecture to train the crack types in the image, considers the imbalance of crack types, and adjusts the weight distribution; Among them, ω j is the weight of the expert model selected for the jth crack type, p j1 is the detection probability of the j1-th type of crack, σ p is the variance of the Gaussian distribution, Q j is the matching degree between the jth crack type and the expert model, Q max is the maximum value of the matching degree between all crack types and the expert model, k is the steepness parameter of the sigmoid function, μ T and μ E are the standard deviations of the crack feature types and crack expert detection models, respectively; The optimization algorithm includes adjusting the parameter coordination between the gate control unit and all expert models through a multi-objective optimization algorithm, stipulating that the goal of the gateway module is to maximize the overall detection performance of the system, and determining the key crack detection priority through the f1 function: By adjusting the parameters of the gateway module, the key crack data are assigned to the expert model that is best at detecting key cracks, where T is the set of key crack types, AC p is the detection accuracy corresponding to type j; The gateway module uses the f2 function to feedback the overall detection performance of the system: in, They represent the average accuracy, average intersection-over-union ratio, average precision, average recall and average correctness, respectively, and the coefficients β, γ, δ, and ε are weight parameters that adjust the influence of the performance index in the overall objective function; The objective function of multi-objective optimization is defined by the f1 function and the f2 function: Among them, S is the objective function of multi-objective optimization, s1 is the weight coefficient of key crack detection priority, ad1 is the adjustment parameter of key crack detection priority, is the ideal key crack detection priority, κ is the normalization constant, s2 is the weight coefficient of the overall detection performance of the system, and ad2 is the adjustment parameter of the overall detection performance of the system; Fine-tuning via Bayesian optimization to select the initial hyperparameter combination θ route and θ experts , calculate the objective function value S(θ route ,θ experts ), where θ route and θ experts Represent the hyperparameters of the gateway module and the expert model respectively. The Gaussian process model is trained using the observed hyperparameter combination and the corresponding objective function. The hyperparameters are selected to maximize the acquisition function θ next = argmax θ EI(θ), next As the next hyperparameter combination to improve performance, the iterative update stops when the performance reaches a predetermined level, and the parameters of the gating unit and MoE are updated according to the optimization results; The secondary fine-grained detection includes, after the image is matched by the gated unit, activating the selected corresponding expert model to perform secondary fine-grained detection: When Δ=0, it means that the judgment error between the expert model and the gate control unit is the smallest, and the detection result of the initial coarse-grained detection is retained. When Δ=1, it means that the judgment error between the expert model and the gate control unit is the largest, and the detection result of the expert model is retained; Among them, Δ is the total discrimination error of the secondary fine-grained detection model, N is the number of image samples, and I i is the i-th image sample, ω j is the weight parameter of the gating unit, The gate control unit is the image I i The matching results, E i (I i ; W e ) is the i-th expert model for image I i The detection output of image I i The crack area in the, G(x is the crack type discrimination function, T j is the characteristic type of the j-th type of crack, that is, the crack feature extraction result; The output result fusion includes, after the crack detection result of the image to be detected is obtained through the hybrid expert model MoE, the result fusion is performed according to the crack position and crack type, and the first round of judgment is first performed according to the distance position of the crack, specifically, the expert models A and B are specified, and the detection results are crack a and crack b; Model A is at position l A The information of detecting crack a is (x A ,y A ,w A ,h A ,t A ), where (x A ,y A ) is the coordinate of the center of crack a, w A and h A is the width and height of the crack a, t A It is the type of crack a; Model B is at position l B The information of detecting crack b is (x B ,y B ,w B ,h B ,t B ), where (x B ,y B ) is the coordinate of the center of crack b, w B and h B is the width and height of crack b, t B It is the type of crack b; If l A and l B spacing If the crack is less than 5 cm, the detection results of model A and model B are considered to point to the same crack, and the average value is taken to combine the position and size: When l A and l B When the spacing is less than 5 cm, a second round of judgment is performed based on the crack type. If t A and t B If the two cracks are consistent, the characteristics of the two cracks are highly overlapped and they are directly merged. A and t B If they are inconsistent, the confidence of the crack type is calculated and the crack type with high confidence is selected as the fused crack type, t is the crack type, t∈T; When l A and l B When the spacing is not less than 5 cm, it is judged as an independent crack and does not need to be fused. It is placed in the independent crack image library and the distribution pattern is compared with the images in the library. When a regular distribution is shown, the two types of cracks are considered to be related and removed from the image library for judgment by manual experts. When there is no regular distribution, it is retained in the independent crack image library.

2. The multi-objective optimization dam crack detection enhancement method based on expert system as claimed in claim 1 is characterized by: The raw image data includes the use of a high-resolution camera mounted on a drone to collect images of the dam surface so that the geocoding of the image data is accurate, and the use of a mobile robot carrying a high-definition camera to collect close-range image data of suspected cracks at key locations and in images collected by the drone; The preprocessing includes inputting the original image into the system, performing image denoising, enhancement, scaling and brightness enhancement to form an image to be detected and inputting the image into the gate control unit.

3. The multi-objective optimization dam crack detection enhancement method based on expert system as claimed in claim 2 is characterized by: The weight allocation adjustment includes adjusting the loss function using a weight allocation mechanism based on the relative frequency of samples, specifically, for each crack type j, according to the relative frequency f in the data set j , update the weight ω' of each category j , so that crack types with a frequency of less than 20% have a high weight: Among them, α is the adjustment parameter, according to the probability p of each crack type output by the detection model ij , adjust the classification loss L cls , optimize the multi-target detection model: Where N is the number of image samples, M is the number of crack types, i represents the sample index, j represents the variable index of the crack type, and y ij is the true label, y ij =0, 1, indicating whether sample i belongs to type j.

4. A system using the multi-objective optimization dam crack detection enhancement method based on an expert system as claimed in any one of claims 1 to 3, characterized in that: It includes a pre-processing module, a gating unit, and a detection module; The preprocessing module performs preliminary processing on the input dam image, including image denoising, enhancement, and scaling; The gate control unit includes a gateway module and a pre-classification and recognition module. The gateway module performs a preliminary analysis on the pre-processed image and dynamically allocates the image to a corresponding expert model for further detection according to the characteristics of the cracks in the image. The pre-classification and recognition module performs preliminary classification of multiple types of cracks on the input samples through multi-target detection, outputs the detection results of each crack, and selectively activates the subsequent expert model to perform secondary fine-grained detection; The detection module includes a hybrid expert model MoE module, an optimization module and a post-fusion module. The hybrid expert model MoE module uses a special expert model to perform in-depth analysis and detection on different types of cracks; The optimization module uses multi-objective optimization and Bayesian optimization algorithms to adjust the coordination between the gating unit and each expert model, as well as the parameters within each expert model, to ensure the optimal performance of the overall system; The post-fusion module collects the output results of all activated expert models, post-fuses the output results of multiple MoE experts based on crack type and crack location, and integrates the outputs of all experts to generate the final detection result.

5. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the multi-objective optimization dam crack detection enhancement method based on an expert system according to any one of claims 1 to 3 are implemented.

6. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the multi-objective optimization dam crack detection enhancement method based on an expert system according to any one of claims 1 to 3 are implemented.

Images