ECC multi-seam cracking real-time detection system and method based on DETR model

Abstract

The invention discloses an ECC multi-crack real-time detection system and method based on a DETR model, and relates to the technical field of crack detection, and the method comprises the steps: collecting first image data according to the collection frequency, and carrying out the preprocessing of the image data, and obtaining second image data; second image data are detected based on a DETR model, and the DETR model comprises a self-attention mechanism and a global context modeling mechanism; determining a plurality of relation matrixes among the cracks through a self-attention mechanism; based on the relation matrixes, positioning and distinguishing of the multiple cracks are determined at the same time through a global context modeling mechanism; and if the detection result exceeds the preset safety threshold, triggering alarm equipment to give an alarm. The crack image is detected through the self-attention mechanism and the global context modeling mechanism based on the DERT model, the crack detection problem in a complex background can be effectively solved, tiny cracks can be more accurately recognized, multiple cracks can be efficiently recognized, the bounding box of each crack can be accurately positioned, and therefore the multi-crack detection accuracy is improved.

Description

Technical Field

[0001] The present application relates to the field of crack detection technology, and in particular to a real-time detection system and method for ECC multi-crack cracking based on a DETR model. Background Art

[0002] Crack monitoring of ECC materials mainly relies on image processing technology and target detection algorithm. Traditional crack detection methods are usually based on image processing technology, such as edge detection, threshold segmentation, template matching, etc. These methods identify cracks by capturing the crack edges in the image, and are suitable for simple and clear crack detection tasks. The working principle of this type of method is to extract image features by preprocessing the image, and then determine the crack area through matching algorithms; technologies based on target detection algorithms are also widely used in crack detection tasks. These models can detect and locate crack areas in images with good accuracy.

[0003] At present, traditional crack detection methods based on image processing technology have poor detection effects on complex backgrounds, tiny cracks and multiple cracks, and are greatly affected by factors such as lighting and background clutter. Technologies based on target detection algorithms still face great challenges when dealing with complex scenes, overlapping cracks and small cracks. Although technologies based on target detection algorithms have faster processing speeds, they have low accuracy and recall rates when detecting multiple targets (multiple cracks), especially for the recognition of small cracks.

[0004] Therefore, how to improve the accuracy of multiple crack detection is a problem that needs to be solved urgently.

[0005] The above contents are only used to assist in understanding the technical solution of the present application and do not constitute an admission that the above contents are prior art. Summary of the invention

[0006] The main purpose of this application is to provide a real-time detection system and method for ECC multiple cracks based on the DETR model, aiming to solve the technical problem of how to improve the accuracy of multiple crack detection.

[0007] To achieve the above-mentioned purpose, the present application proposes a real-time detection system for ECC multi-crack cracking based on a DETR model, wherein the real-time detection system for ECC multi-crack cracking based on a DETR model comprises an image acquisition device, a computing device, a display device, a control system, and an alarm device, wherein the image acquisition device is connected to the computing device, the computing device is connected to the control system, and the control system is connected to the alarm device:

[0008] The image acquisition device is used to acquire ECC surface image data and transmit the ECC surface image data to a computing device;

[0009] The computing device is used to receive the image data, perform crack detection based on the image data using a DETR model, and transmit the crack detection result to a display device and a control system;

[0010] The display device and control system are used to receive the detection results and transmit control instructions to the alarm device;

[0011] The alarm device is used to receive the control instruction and trigger an alarm prompt.

[0012] Optionally, the computing device is used to perform denoising and image enhancement operations on the image data, and use the processed image data as input of the DETR model;

[0013] The computing device is further used to use the trained DETR model to infer the input and identify the category and bounding box of the crack;

[0014] The computing device is also used to simultaneously detect multiple cracks in the same image.

[0015] Optionally, the display device and control system include a display device and a control system:

[0016] The control system is used to receive the detection result and alarm information;

[0017] The control system is further used to control the alarm device based on the alarm information;

[0018] The display device is used to display the image data, the detection results and the alarm information, and provide a graphical interface for the user so that the user can monitor and respond to the output of the system in real time.

[0019] In addition, to achieve the above purpose, the present application also proposes a real-time detection method for ECC multi-crack cracking based on the DETR model, which is applied to the real-time detection system for ECC multi-crack cracking based on the DETR model. The method includes:

[0020] Collecting first image data according to an acquisition frequency, and preprocessing the image data to obtain second image data;

[0021] Detecting the second image data based on a DETR model, wherein the DETR model includes a self-attention mechanism and a global context modeling mechanism;

[0022] Determine multiple relationship matrices between cracks through the self-attention mechanism;

[0023] Based on each of the relationship matrices, simultaneously determining the location and differentiation of multiple cracks through the global context modeling mechanism;

[0024] If the detection result exceeds the preset safety threshold, the alarm device will be triggered to sound an alarm.

[0025] Optionally, acquiring the first image data according to the acquisition frequency, and preprocessing the image data to obtain the second image data includes:

[0026] Acquire first image data by an image acquisition device;

[0027] The image data is subjected to denoising and image enhancement processing to obtain second image data.

[0028] Optionally, determining a plurality of relationship matrices between cracks by the self-attention mechanism includes:

[0029] Extracting features of the second image data through a convolutional neural network to obtain a two-dimensional feature map;

[0030] Flatten the two-dimensional feature map into a one-dimensional sequence;

[0031] Adding a position code to each of the sequence elements to retain the spatial position information of the feature point, wherein each of the sequence elements is a feature vector corresponding to each of the feature points;

[0032] The correlations between the sequence elements are calculated in sequence to obtain a plurality of relationship matrices.

[0033] Optionally, the two-dimensional feature map includes multiple feature maps and feature map sizes, the feature map size includes the number of channels, the length and width of the feature map, and each of the feature maps corresponds one-to-one to the feature information of each position of the second image data.

[0034] Optionally, the sequentially calculating the correlations between the sequence elements to obtain a plurality of relationship matrices includes:

[0035] Selecting a sequence element from the one-dimensional sequence as a query vector, and the remaining sequence elements as key vectors;

[0036] Calculating the dot product of the query vector and each of the key vectors in sequence to obtain a relationship matrix corresponding to the query vector;

[0037] Obtaining a scaled relationship matrix according to the dimension of the key vector and the relationship matrix;

[0038] The one-dimensional sequence is traversed, and the above steps are performed on each element of the sequence to obtain multiple relationship matrices.

[0039] Optionally, the simultaneously determining the locations and distinctions of multiple cracks based on the relationship matrices through the global context modeling mechanism includes:

[0040] Based on each of the relationship matrices, the feature vectors are classified and bounding box predicted to obtain a plurality of cracks and corresponding crack prediction bounding box positions;

[0041] Acquiring prior information of crack characteristics, wherein the prior information of crack characteristics includes common geometric shapes and size distribution of cracks;

[0042] During the training process of the DETR model, adjustments are made in combination with the prior information of the crack characteristics;

[0043] Get the improved bounding box regression loss function;

[0044] Based on the relationship matrices, the improved bounding box regression loss function and the adjusted DETR model, the feature vectors are classified and bounding box predicted to obtain multiple cracks and corresponding crack prediction bounding box positions.

[0045] Optionally, if the detection result exceeds a preset safety threshold, after triggering an alarm device to sound an alarm, the method further includes:

[0046] The crack detection results are displayed on the user interface so that the user can view and monitor the distribution and detailed information of the cracks through the interactive module.

[0047] One or more technical solutions proposed in this application have at least the following technical effects:

[0048] The first image data is collected according to the collection frequency, and the image data is preprocessed to obtain the second image data; the second image data is detected based on the DETR model, wherein the DETR model includes a self-attention mechanism and a global context modeling mechanism; multiple relationship matrices between cracks are determined through the self-attention mechanism; based on each of the relationship matrices, the location and distinction of multiple cracks are simultaneously determined through the global context modeling mechanism; if the detection result exceeds the preset safety threshold, the alarm device is triggered to alarm. By detecting crack images based on the self-attention mechanism and the global context modeling mechanism of the DERT model, it is possible to effectively handle the problem of crack detection in complex backgrounds, more accurately identify tiny cracks, efficiently identify multiple cracks, and accurately locate the bounding box of each crack, thereby improving the accuracy of multi-crack detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.

[0050] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0051] Figure 1 This is a structural block diagram of the first embodiment of the ECC multi-crack real-time detection system based on the DETR model of the present application;

[0052] Figure 2 A flow chart of the second embodiment of the real-time detection method for ECC multi-crack cracking based on the DETR model of the present application;

[0053] Figure 3 A schematic diagram of a simplified process of a real-time detection method for ECC multi-crack cracking based on a DETR model provided in Example 2 of the present application;

[0054] Description of Figure Numbers:

[0055] Image acquisition device 10, computing device 20, display device and control system 30, alarm device 40, display device 301, control system 302.

[0056] The purpose, features and advantages of this application will be further described in conjunction with the embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0057] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of the present application and are not used to limit the present application.

[0058] Reference Figure 1 , Figure 1 This is a structural block diagram of the first embodiment of the ECC multi-crack real-time detection system based on the DETR model of the present application.

[0059] The ECC multi-crack real-time detection system based on the DETR model includes: an image acquisition device 10, a computing device 20, a display device 301, a control system 302 and an alarm device 40, wherein the image acquisition device 10 is connected to the computing device 20, the computing device 20 is connected to the control system 302, and the control system 302 is connected to the alarm device 40.

[0060] In addition, the display device 301 and the control system 302 include a display device 301 and a control system 302 .

[0061] In this embodiment, the image acquisition device 10 is used to acquire ECC surface image data and transmit the ECC surface image data to the computing device 20 .

[0062] It should be noted that the image acquisition device 10 can be a hardware device capable of capturing image information, such as a high-definition digital camera or an industrial camera, for real-time acquisition of ECC surface images. The image acquisition device 10 needs to have a high resolution (such as 4K or higher) to ensure that the minute details of the cracks can be clearly captured. The camera and the computing device 20 transmit the acquired image data via a network or direct connection.

[0063] The computing device 20 is used to receive the image data, perform crack detection based on the image data using the DETR model, and transmit the crack detection results to the display device 301 and the control system 302 .

[0064] It should be noted that the computing device 20 can be an electronic device capable of performing computing tasks. In the present application, the computing device 20 can select a high-performance GPU (such as NVIDIA A100, V100, etc.) or an edge computing device 20 with strong adaptability. The computing device 20 is connected to the image acquisition device 10 through a high-speed transmission interface to ensure smooth and real-time data transmission.

[0065] In addition, the computing device 20 is also used to perform denoising and image enhancement operations on the image data, and use the processed image data as input of the DETR model, use the trained DETR model to infer the input, identify the category and bounding box of the cracks, and then detect the precise locations of multiple cracks in the same image simultaneously based on the category and bounding box information of the cracks.

[0066] Specifically, after the computing device 20 receives the crack image data collected by the image acquisition device 10, the image data is subjected to denoising and image enhancement. The denoising process may be to use Gaussian filtering, mean filtering and other methods to remove noise in the image to ensure that the details of the cracks are clearly displayed. Image enhancement may use data enhancement technology (such as rotation, cropping, brightness adjustment, etc.) to increase sample diversity and improve the adaptability of the model to crack detection under different environmental conditions. The processed image data is used as the input of the DETR model, and the trained DETR model is used to infer the processed image data, identify the category and bounding box of the crack, and then detect the precise positions of multiple cracks in the same image at the same time based on the category and bounding box information of the crack.

[0067] The display device 301 and the control system 302 are used to receive the detection result and transmit control instructions to the alarm device 40.

[0068] It should be noted that the display device 301 and the control system 302 are important components of crack detection result display and interaction.

[0069] pass Figure 1 As can be seen from the structural block diagram, the display device 301 and the control system 302 include a display device 301 and a control system 302. The display device 301 and the control system 302 include a display device 301 and a control system 302: the control system 302 is used to receive the detection result and the alarm information; the control system 302 is also used to control the alarm device 40 based on the alarm information; the display device 301 is used to display the image data, the detection result and the alarm information, and provide a user with a graphic interface so that the user can monitor and respond to the output of the system in real time.

[0070] In addition, the display device 301 can be a device that can convert electronic signals into visual images or tactile information, such as a touch screen or a PC terminal, and the control system 302 can be a collection of devices and algorithms for managing and regulating mechanical equipment, processes or systems, which ensure that the system operates in a predetermined manner and makes adjustments when necessary to maintain performance standards or respond to external changes. Specifically, the control system 302 receives crack detection results in real time by connecting to the computing device 20. The display device 301 displays the detection results and alarm information, which may include information such as the position, size, and boundary box of the crack in the image. The user can view the real-time image and crack position through the display device 301, while the control system 302 operates.

[0071] The alarm device 40 is used to receive the control instruction and trigger an alarm prompt.

[0072] It should be noted that the alarm device 40 may be a device that issues an alarm in an emergency or under certain conditions. In this embodiment, when the system detects that the crack detection result reaches a set threshold, for example, when the size of the crack exceeds 200 microns, the alarm device 40 is automatically triggered (such as sound alarm, flashing light, etc.) to alert the staff, so as to ensure that the staff can respond to the abnormal situation in time.

[0073] In this embodiment, the image acquisition device 10 acquires ECC surface image data, and transmits the ECC surface image data to the computing device 20; when the computing device 20 receives the image data, crack detection is performed based on the image data through the DETR model, and the crack detection result is transmitted to the display device 301 and the control system 302; the display device 301 and the control system 302 receive the detection result and transmit a control instruction to the alarm device 40; the alarm device 40 receives the control instruction and triggers an alarm prompt. By adopting a deep learning method based on the DETR model, combined with image acquisition, processing, analysis and alarm equipment, automatic crack detection on the surface of ECC materials is achieved.

[0074] The present application embodiment provides a real-time detection method for ECC multi-crack cracking based on the DETR model, referring to Figure 2 , Figure 2 This is a flow chart of a first embodiment of a real-time detection method for ECC multi-cracks based on a DETR model according to the present application.

[0075] In this embodiment, the dual vehicle controller in-loop testing method includes:

[0076] Step S10: collecting first image data according to the collection frequency, and preprocessing the image data to obtain second image data.

[0077] It should be noted that the acquisition frequency may be the inverse of the time interval at which the image acquisition device 10 acquires image data, such as once every 2 seconds. In this case, the acquisition frequency is 0.5, and the first image data may be an ECC surface image acquired in real time by the image acquisition device 10. Preprocessing may be a key step in image processing and computer vision tasks, the purpose of which is to improve image quality and enhance image features so that subsequent processing and analysis are more effective.

[0078] It can be understood that preprocessing the collected images is helpful to highlight the key features in the images, thereby facilitating subsequent feature extraction and recognition.

[0079] In a feasible implementation, step S10 may include acquiring first image data through the image acquisition device 10; and performing denoising and image enhancement processing on the image data to obtain second image data.

[0080] It should be noted that the first image data may be an ECC surface image acquired by the image acquisition device 10. De-noising may be performed by using Gaussian filtering, mean filtering, and other methods to remove noise from the image, and image enhancement may be achieved by data enhancement techniques, such as rotation, cropping, brightness adjustment, and the second image data is processed.

[0081] The ECC surface image is collected in real time by the imaging device, and the surface image is processed by denoising and image enhancement to make the image clearer, the details more prominent, and the image quality improved.

[0082] Step S20: Detect the second image data based on the DETR model, wherein the DETR model includes a self-attention mechanism and a global context modeling mechanism.

[0083] It should be noted that the self-attention mechanism can be used to capture the global context and enhance the feature representation by looking at the position and relationship of other objects in the image. The global context modeling mechanism can be achieved by establishing a connection between each position of the input feature and other positions in the entire image through the Transformer architecture of the DETR model, and updating each element by aggregating information about the connection, thereby achieving global context modeling.

[0084] It can be understood that by using the self-attention mechanism and global context modeling mechanism of the DETR model, the shortcomings of existing target detection technologies in the simultaneous detection of multiple cracks, complex background processing and the accuracy of tiny cracks are avoided, thereby providing higher accuracy in cases of tiny cracks and complex backgrounds.

[0085] Step S30: Determine multiple relationship matrices between cracks through the self-attention mechanism.

[0086] It should be noted that the relationship matrix may be a mathematical structure used to represent the relationship between entities. In this embodiment, it is used to represent the relationship between cracks in an image.

[0087] It can be understood that by establishing the relationship matrix between cracks, the relative information between cracks is fully understood, which provides data support for subsequent correct positioning and differentiation when multiple cracks exist at the same time.

[0088] In a feasible implementation, step S30 may include steps A10 to A70:

[0089] Step A10, extracting features of the second image data through a convolutional neural network to obtain a two-dimensional feature map;

[0090] It should be noted that the convolutional neural network can be a deep learning model, such as Google's network (GoogLeNet: Google Inception Net), and the feature can be information extracted from the image that helps to describe the image content and attributes, such as the image size, color distribution, etc. The two-dimensional feature map can include multiple feature maps and feature map sizes, and the feature map size includes the number of channels, the length of the feature map, and the width. Each feature map corresponds to the feature information of each position of the second image data one by one. The number of channels can be the depth in the feature map, that is, the length of the feature vector at each position (pixel), and each value of the feature vector corresponds to an image feature. The length of the feature map refers to the size of the feature map in the vertical direction, that is, the number of rows, and the width of the feature map refers to the size of the feature map in the horizontal direction.

[0091] Step A20, flattening the two-dimensional feature map into a one-dimensional sequence;

[0092] What needs to be explained is that according to the length and width of the feature map, a coordinate system is established to unfold the two-dimensional feature map, which is similar to imagining the two-dimensional feature map as a rectangle composed of many small squares. Each small square stores the feature vector corresponding to the image position. The feature vector records the information of the corresponding position of the image. The feature vector is unfolded as a whole, that is, these small squares are taken out one by one from left to right and from top to bottom and arranged in a row. This row of small squares is the one-dimensional sequence obtained after flattening the two-dimensional feature map.

[0093] Step A30, adding a position code to each of the sequence elements to retain the spatial position information of the feature point, wherein each of the sequence elements is a feature vector corresponding to each of the feature points;

[0094] It should be noted that position encoding can be used to record the relative position of each element in the one-dimensional sequence in the two-dimensional feature map.

[0095] There are different ways to encode positions. A simple way is to add two extra numbers to each sequence element, representing its row and column position in the original two-dimensional space. Another way is to use more complex functions, such as trigonometric functions, to encode position information, which can capture the relative relationship between positions.

[0096] Step A40, selecting a sequence element from the one-dimensional sequence as a query vector, and the remaining sequence elements as key vectors;

[0097] It should be noted that the query vector may be the currently selected sequence element, and the key vector may be the remaining sequence elements except the currently selected sequence element.

[0098] Step A50, calculating the dot product of the query vector and each of the key vectors in turn to obtain a relationship matrix corresponding to the query vector;

[0099] Step A60, obtaining a scaled relationship matrix according to the dimension of the key vector and the relationship matrix;

[0100] It should be noted that the dot product, also known as the scalar product or the dot product, is an algebraic operation between two vectors in mathematics.

[0101] It can be understood that by calculating the dot product of the query vector and the key vector, and then obtaining the scaled relationship matrix according to the number of channels of the feature map and the Softmax function, this process allows the model to dynamically focus on the most relevant part of the input sequence when processing sequence data.

[0102] Among them, Q represents the query vector, which represents the feature vector (sequence element) that needs to be paid attention to at present, K represents the key vector, which represents all the feature vectors that may be associated with the query vector, C represents the dimension of the key vector, that is, the number of features of each key vector, and Softmax represents a normalization function that converts the result of the dot product into a probability distribution so that the sum of all attention weights is 1. The calculation formula is as follows:

[0103]

[0104] Step A70, traversing the one-dimensional sequence, executing the above steps for each element of the sequence, and obtaining a plurality of relationship matrices;

[0105] It can be understood that by calculating the planning matrix of each sequence element, the relationship matrix between feature vectors can be comprehensively established, thereby providing a data basis for global context modeling.

[0106] Step S40: Based on each of the relationship matrices, the locations and distinctions of multiple cracks are simultaneously determined through the global context modeling mechanism.

[0107] It is understandable that in the case of tiny cracks, small crack spacing or complex background, the global context modeling mechanism is used to represent the relative relationship between cracks using the relationship matrix between cracks as the data basis, thereby achieving the distinction between cracks and improving the accuracy of crack detection.

[0108] In a feasible implementation, step S40 may include classifying and predicting bounding boxes on the feature vectors based on each of the relationship matrices to obtain multiple cracks and corresponding crack prediction bounding box positions; obtaining prior information on crack features; making adjustments during the training of the DETR model based on the prior information on crack features; obtaining an improved bounding box regression loss function; and classifying and predicting bounding boxes on the feature vectors based on each of the relationship matrices, the improved bounding box regression loss function, and the adjusted DETR model to obtain multiple cracks and corresponding crack prediction bounding box positions.

[0109] It should be noted that the classification of cracks can include the levels of crack length and width, and the prior information of crack characteristics includes the common geometric shapes and size distribution of cracks; the bounding box regression loss function can be used in target detection tasks to optimize the position and size of the predicted bounding box.

[0110] Specifically, during the training process of the DETR model, images of common geometric shapes and size distributions of cracks are used as input to train the DETR model. Then, based on the trained DETR model, the cracks corresponding to the feature vectors are classified and bounding boxes are predicted. At the same time, in the bounding box prediction process, an improved bounding box regression loss function is introduced to predict the bounding box of the crack. For example, bounding box regression loss functions such as generalized intersection over union (GIoU) ​​or distance intersection over union (DIoU) can be adopted to predict the bounding box of the crack.

[0111] It can be understood that by dynamically adjusting the target embedding during the DERT model training process, the decoder part of the DERT model can be more focused on the crack area and reduce false detections in irrelevant areas. In the process of bounding box prediction, the introduction of an improved bounding box regression loss function is conducive to further improving the accuracy of crack bounding box prediction.

[0112] Step S50: If the detection result exceeds the preset safety threshold, the alarm device 40 is triggered to sound an alarm.

[0113] It should be noted that the preset safety threshold may refer to the maximum allowable value of the crack width without affecting the structural performance and durability of the material. Taking engineered cementitious composites (ECC) as an example, the crack width should usually be controlled within 100 microns to ensure the durability of the material and the safety of the structure. Therefore, the preset safety threshold is 100 microns.

[0114] It is understandable that by combining the crack detection results, setting a preset safety threshold, automatically triggering the alarm system and feeding back key information such as the crack location, type, size, etc., it is possible to intelligently judge the severity of the cracks, and help engineers respond quickly through real-time alarms and feedback, reducing manual intervention, effectively improving the response speed and accuracy of the monitoring system, and avoiding the lag of manual intervention.

[0115] This embodiment collects first image data according to the collection frequency, preprocesses the image data, and obtains second image data; detects the second image data based on the DETR model, wherein the DETR model includes a self-attention mechanism and a global context modeling mechanism; determines multiple relationship matrices between cracks through the self-attention mechanism; based on each of the relationship matrices, simultaneously determines the location and distinction of multiple cracks through the global context modeling mechanism; if the detection result exceeds the preset safety threshold, the alarm device 40 is triggered to alarm. By detecting crack images based on the self-attention mechanism and the global context modeling mechanism of the DERT model, it is possible to effectively handle the problem of crack detection in complex backgrounds, more accurately identify tiny cracks, efficiently identify multiple cracks, and accurately locate the boundary box of each crack, thereby improving the accuracy of multi-crack detection.

[0116] For example, to help understand the implementation process of this embodiment combined with the above-mentioned embodiment 1, please refer to Figure 3 , Figure 3 A brief flow chart of a real-time detection method for ECC multi-crack cracking based on the DETR model is provided. Specifically:

[0117] First, real-time image data (first image data) is collected by an image acquisition device 10 (image acquisition module), and the real-time image data is transmitted to a data preprocessing module, that is, the image data is transmitted to a computing device 20, and the image data is preprocessed (denoising and data enhancement processing) to enhance the image quality to obtain second image data, and then the second image data is used as the input of a crack detection model (DETR model), and the DERT model self-attention mechanism is used to determine multiple relationship matrices between cracks, and the positioning and differentiation of multiple cracks are simultaneously determined through the global context modeling mechanism, that is, the DETR model is used to complete the positioning and differentiation of multiple cracks to obtain crack detection results, which include the location (bounding box) and type of the crack, and then it is determined whether the detection result is less than a preset safety threshold of the crack. If it exceeds the preset safety threshold, the alarm device 40 is triggered, and the detection result and alarm information are displayed at the same time, so that the user can view the real-time detection result through the interactive interface, and the alarm information includes information such as the width and length of the crack.

[0118] It should be understood that the above is only an example and does not constitute any limitation on the technical solution of the present application. In specific applications, technicians in this field can make settings as needed, and the present application does not impose any limitation on this.

[0119] It should be noted that the workflow described above is merely illustrative and does not limit the scope of protection of the present application. In practical applications, technicians in this field can select part or all of it according to actual needs to achieve the purpose of the present embodiment, and no limitation is made here.

[0120] In addition, it should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "comprises a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element.

[0121] The serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0122] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A real-time detection system for ECC multi-cracks based on DETR model, characterized in that: The ECC multi-crack real-time detection system based on the DETR model includes an image acquisition device, a computing device, a display device, a control system, and an alarm device, wherein the image acquisition device is connected to the computing device, the computing device is connected to the control system, and the control system is connected to the alarm device: The image acquisition device is used to acquire ECC surface image data and transmit the ECC surface image data to a computing device; The computing device is used to receive the image data, perform crack detection based on the image data using a DETR model, and transmit the crack detection result to a display device and a control system; The display device and control system are used to receive the detection results and transmit control instructions to the alarm device; The alarm device is used to receive the control instruction and trigger an alarm prompt.

2. The system according to claim 1, characterized in that The computing device further comprises: The computing device is used to perform denoising and image enhancement operations on the image data, and use the processed image data as input of the DETR model; The computing device is further used to use the trained DETR model to infer the input and identify the category and bounding box of the crack; The computing device is also used to simultaneously detect multiple cracks in the same image.

3. The system according to claim 1, characterized in that The display device and control system include a display device and a control system: The control system is used to receive the detection result and alarm information; The control system is further used to control the alarm device based on the alarm information; The display device is used to display the image data, the detection results and the alarm information, and provide a graphical interface for the user so that the user can monitor and respond to the output of the system in real time.

4. A real-time detection method for ECC multi-cracks based on DETR model, characterized in that: The method is applied to a computing device in a real-time ECC multi-crack detection system based on a DETR model as claimed in any one of claims 1 to 3, and the method comprises: Collecting first image data according to an acquisition frequency, and preprocessing the image data to obtain second image data; Detecting the second image data based on a DETR model, wherein the DETR model includes a self-attention mechanism and a global context modeling mechanism; Determine multiple relationship matrices between cracks through the self-attention mechanism; Based on each of the relationship matrices, simultaneously determining the location and differentiation of multiple cracks through the global context modeling mechanism; If the detection result exceeds the preset safety threshold, the alarm device will be triggered to sound an alarm.

5. The method according to claim 4, characterized in that The step of acquiring the first image data according to the acquisition frequency and preprocessing the image data to obtain the second image data includes: Acquire first image data by an image acquisition device; The image data is subjected to denoising and image enhancement processing to obtain second image data.

6. The method according to claim 4, characterized in that Determining multiple relationship matrices between cracks through the self-attention mechanism includes: Extracting features of the second image data through a convolutional neural network to obtain a two-dimensional feature map; Flatten the two-dimensional feature map into a one-dimensional sequence; Adding a position code to each of the sequence elements to retain the spatial position information of the feature point, wherein each of the sequence elements is a feature vector corresponding to each of the feature points; The correlations between the sequence elements are calculated in sequence to obtain a plurality of relationship matrices.

7. The method according to claim 6, characterized in that The two-dimensional feature map includes multiple feature maps and feature map sizes, and the feature map size includes the number of channels, the length and width of the feature map. Each of the feature maps corresponds one-to-one to the feature information of each position of the second image data.

8. The method according to claim 6, characterized in that The correlation between the sequence elements is calculated in sequence to obtain a plurality of relationship matrices, including: Selecting a sequence element from the one-dimensional sequence as a query vector, and the remaining sequence elements as key vectors; Calculating the dot product of the query vector and each of the key vectors in sequence to obtain a relationship matrix corresponding to the query vector; Obtaining a scaled relationship matrix according to the dimension of the key vector and the relationship matrix; The one-dimensional sequence is traversed, and the above steps are performed on each element of the sequence to obtain multiple relationship matrices.

9. The method according to claim 4, characterized in that The method of simultaneously determining the locations and distinctions of multiple cracks based on the relationship matrices through the global context modeling mechanism includes: Based on each of the relationship matrices, the feature vectors are classified and bounding box predicted to obtain a plurality of cracks and corresponding crack prediction bounding box positions; Acquiring prior information of crack characteristics, wherein the prior information of crack characteristics includes common geometric shapes and size distribution of cracks; During the training process of the DETR model, adjustments are made in combination with the prior information of the crack characteristics; Get the improved bounding box regression loss function; Based on the relationship matrices, the improved bounding box regression loss function and the adjusted DETR model, the feature vectors are classified and bounding box predicted to obtain multiple cracks and corresponding crack prediction bounding box positions.

10. The method according to claim 4, characterized in that If the detection result exceeds the preset safety threshold, after the alarm device is triggered to alarm, the method further includes: The crack detection results are displayed on the user interface so that the user can view and monitor the distribution and detailed information of the cracks through the interactive module.

Images