Structural high-analogue simulation damage crack image generation method and system

Abstract

The invention belongs to the technical field of structural crack damage detection, and particularly relates to a structural high-simulation damage crack image generation method and system, and when a deep convolutional neural network model is trained, a small amount of real crack image data can be utilized to generate high-simulation simulation cracks continuously and repeatedly. A large number of clear simulation crack images are automatically generated in a short time, and then the images are used for training a deep convolutional neural network model, so that a deep learning model with high accuracy and high reliability for crack recognition can be obtained.

Description

Technical Field

[0001] The present application belongs to the technical field of structural crack damage detection, and specifically relates to a method and system for generating a highly simulated damage crack image on a structure. Background Art

[0002] Cracks are a common form of damage to aviation structures. For example, crack damage on frame structures and beam structures can be avoided by timely detection and discovery of crack damage on aviation structures and appropriate maintenance and management.

[0003] With the in-depth development of robotics and artificial intelligence technologies, machine vision provides a new solution for the automated detection of crack damage on aviation structures. Specifically, high-definition images of the inspection parts on aviation structures are first obtained through high-precision motion systems (such as crawling robots, robotic arms) and industrial cameras, and then target detection algorithms are used to automatically identify cracks and perform damage warnings, which can greatly reduce the cost of manual inspections, have good real-time performance, and can effectively avoid the occurrence of dangerous accidents.

[0004] Deep convolutional neural network is a commonly used model for crack damage target detection algorithm. A large number of high-quality images are required to train a deep learning model with high accuracy and reliability for crack identification.

[0005] Currently, the structural crack images used for training deep convolutional neural network models are mostly collected through structural fatigue tests. The fatigue test cycle is long and the test environment is complex. It is difficult to detect structural cracks in time for collection. Therefore, it is difficult to obtain a large number of clear structural crack images in a short period of time to train the deep convolutional neural network model. The final deep convolutional neural network model often has insufficient crack recognition accuracy and reliability.

[0006] This application is proposed in view of the above-mentioned technical defects. Summary of the invention

[0007] The purpose of this application is to provide a method and system for generating highly realistic simulated damage crack images on structures, so as to generate a large number of clear highly realistic simulated damage crack images on aviation structures in a short period of time, train a deep convolutional neural network model, and then obtain a deep learning model with high accuracy and high reliability for crack identification, which is used to detect crack damage on aviation structures and avoid dangerous accidents.

[0008] The technical solution of this application is:

[0009] On the one hand, a method for generating a highly simulated damage crack image on a structure is provided, comprising:

[0010] Real structural crack image acquisition steps: acquiring a real structural crack image;

[0011] Real crack identification step: identifying real cracks in real structural crack images;

[0012] Steps for obtaining real crack information: obtaining real crack information from real cracks;

[0013] Step of acquiring the image of the structure to be tested: acquiring the image of the structure to be tested;

[0014] The step of identifying the position of the target structure to be measured: identifying the position of the target structure to be measured in the image of the structure to be measured;

[0015] Target position detection frame coordinate acquisition step: from the target position to be measured, obtain the detection frame outline coordinates;

[0016] Simulated crack generation steps: using real crack information and its detection frame contour coordinates to generate simulated cracks;

[0017] The steps of generating the simulated crack image are as follows: on the image of the structure to be measured, the pixel value at the coordinate of the simulated crack pixel point is taken as the pixel value of the corresponding pixel point of the simulated crack to generate the simulated crack image.

[0018] According to at least one embodiment of the present application, in the above-mentioned method for generating a highly realistic simulated damage crack image on a structure, in the step of acquiring real crack information, real crack information is acquired from the real crack, including coordinates and pixel values ​​of real crack pixel points.

[0019] According to at least one embodiment of the present application, in the above-mentioned method for generating a highly simulated damage crack image on a structure, the simulated crack generation step is specifically:

[0020] Select any point in the detection frame contour coordinates as the starting point of the simulated crack;

[0021] Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected;

[0022] Calculate the angle that the simulated crack pixel needs to rotate:

[0023] Calculate the coordinates of the simulated crack pixel points;

[0024] The pixel value of the simulated crack pixel point is taken as the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate the simulated crack.

[0025] According to at least one embodiment of the present application, in the above-mentioned method for generating a highly simulated damage crack image on a structure, in the simulated crack generating step, the angle by which the simulated crack pixel point needs to be rotated is calculated, specifically:

[0026]

[0027] in,

[0028] ang is the angle that the simulated crack pixel needs to rotate;

[0029] k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame;

[0030] k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack;

[0031] Calculate the coordinates of the simulated crack pixel points, specifically:

[0032]

[0033]

[0034] in,

[0035] is the coordinate of the simulated crack pixel;

[0036] is the corresponding coordinate of the real crack point on the image of the structure to be detected;

[0037] x s ,y s are the starting coordinates of the simulated crack.

[0038] According to at least one embodiment of the present application, in the above-mentioned method for generating a highly realistic simulated damage crack image on a structure, in the simulated crack image generating step, an opening and closing operation is performed on the simulated crack image, and color correction and edge fusion are performed on the simulated crack.

[0039] On the other hand, a system for generating a structural high-simulation damage crack image is provided, comprising:

[0040] A real structural crack image acquisition module, used to acquire a real structural crack image;

[0041] A real crack recognition module is used to identify real cracks in real structural crack images;

[0042] A real crack information acquisition module, used to obtain real crack information from real cracks;

[0043] The image acquisition module of the structure to be tested is used to acquire the image of the structure to be tested;

[0044] A target position recognition module for a structure to be measured is used to recognize the position of the target to be measured in the image of the structure to be measured;

[0045] A target position detection frame coordinate acquisition module is used to obtain the detection frame contour coordinates from the target position to be measured;

[0046] A simulated crack generation module is used to generate simulated cracks using real crack information and its detection frame contour coordinates;

[0047] The simulated crack image generation module is used to obtain the pixel value of the pixel point corresponding to the simulated crack from the pixel value at the coordinate of the simulated crack pixel point on the structure image to be measured, so as to generate a simulated crack image.

[0048] According to at least one embodiment of the present application, in the above-mentioned structural high-simulation simulated damage crack image generation system, the real crack information acquisition module obtains real crack information from the real crack, including the coordinates and pixel values ​​of the real crack pixel points.

[0049] According to at least one embodiment of the present application, in the above-mentioned structural high-simulation simulated damage crack image generation system, the simulated crack generation module is used to generate simulated cracks using real crack information and its detection frame contour coordinates, specifically:

[0050] Select any point in the detection frame contour coordinates as the starting point of the simulated crack;

[0051] Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected;

[0052] Calculate the angle that the simulated crack pixel needs to rotate to

[0053] Calculate the coordinates of the simulated crack pixel points;

[0054] The pixel value of the simulated crack pixel point is taken as the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate the simulated crack.

[0055] According to at least one embodiment of the present application, in the above-mentioned structural high-simulation simulated damage crack image generation system, in the simulated crack generation module, the angle that the simulated crack pixel point needs to be rotated is calculated, specifically,

[0056]

[0057] in,

[0058] ang is the angle that the simulated crack pixel needs to rotate;

[0059] k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame;

[0060] k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack;

[0061] Calculate the coordinates of the simulated crack pixel points, specifically,

[0062]

[0063]

[0064] in,

[0065] is the coordinate of the simulated crack pixel;

[0066] is the corresponding coordinate of the real crack point on the image of the structure to be detected;

[0067] x s ,y s are the starting coordinates of the simulated crack.

[0068] According to at least one embodiment of the present application, in the above-mentioned structural high-simulation simulated damage crack image generation system, in the simulated crack image generation module, opening and closing operations are performed on the simulated crack image, and color correction and edge fusion are performed on the simulated crack.

[0069] This application has at least the following beneficial technical effects:

[0070] A method and system for generating highly realistic simulated damage crack images on a structure is provided. When training a deep convolutional neural network model, a small number of real crack images can be used to repeatedly generate highly realistic simulated cracks, so that a large number of clear simulated crack images can be automatically generated in a short time. The images are then used to train the deep convolutional neural network model, thereby obtaining a deep learning model with high accuracy and reliability for crack identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0071] Figure 1 It is a schematic diagram of a method for generating a highly simulated damage crack image on a structure provided in an embodiment of the present application;

[0072] Figure 2 It is a schematic diagram of a structurally highly simulated damage crack image generation system provided in an embodiment of the present application.

[0073] In order to better illustrate the present embodiment, some contents of the drawings may be omitted, which is only used for illustrative purposes and should not be construed as limiting the present application. DETAILED DESCRIPTION

[0074] In order to make the technical solution and advantages of the present application clearer, the technical solution of the present application will be described in further detail in detail and in detail with reference to the accompanying drawings. It can be understood that the specific embodiments described here are only partial embodiments of the present application, which are only used to explain the present application, not to limit the present application. It should be noted that, for the convenience of description, only the parts related to the present application are shown in the accompanying drawings, and other related parts can refer to the general design.

[0075] In addition, unless otherwise defined, the technical terms or scientific terms used in the description of this application shall have the usual meanings understood by those of ordinary skill in the art to which this application belongs. The words used in the description of this application to indicate orientation are only used to indicate relative directions or positional relationships. When the absolute position of the described object changes, its relative positional relationship may also change accordingly. The word "include" used in the description of this application indicates that the elements or objects appearing before the word include the elements or objects listed after the word and their equivalents, but does not exclude other elements or objects.

[0076] In addition, it should be noted that, unless otherwise clearly specified and limited, the words "installation", "connection" and similar terms used in the description of this application should be understood in a broad sense. For example, the connection can be a fixed connection or a detachable connection; it can be a mechanical connection or an electrical connection; it can be a direct connection or an indirect connection through an intermediate medium. Technical personnel in the field can understand its specific meaning in this application according to the specific circumstances.

[0077] A method for generating a highly simulated damage crack image on a structure, such as Figure 1 shown.

[0078] Steps for acquiring a real structural crack image: acquiring a real structural crack image.

[0079] You can take pictures of cracks produced during structural fatigue tests and save them, or select them from online galleries.

[0080] Real crack identification step: identifying real cracks in real structural crack images.

[0081] Steps for obtaining real crack information: obtaining real crack information from real cracks.

[0082] The real crack information includes the coordinates and pixel values ​​of the real crack pixel points, wherein the coordinates of the real crack pixel points can represent the shape of the real crack, and the pixel values ​​of the real crack pixel points can represent the color of the real crack.

[0083] Step of acquiring the image of the structure to be tested: acquiring the image of the structure to be tested.

[0084] The image of the structure to be tested can be obtained by shooting.

[0085] The step of identifying the position of the target structure to be measured is to identify the position of the target structure to be measured in the image of the structure to be measured.

[0086] Steps for obtaining the coordinates of the target position detection frame: From the target position to be measured, obtain the coordinates of the detection frame outline.

[0087] Simulated crack generation steps: Generate simulated cracks using real crack information and its detection frame contour coordinates.

[0088] Select any point in the detection frame contour coordinates as the starting point of the simulated crack;

[0089] Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected;

[0090] Calculate the angle that the simulated crack pixel needs to rotate:

[0091]

[0092] in,

[0093] ang is the angle that the simulated crack pixel needs to rotate;

[0094] k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame;

[0095] k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack;

[0096] Calculate the coordinates of the simulated crack pixel points:

[0097]

[0098]

[0099] in,

[0100] is the coordinate of the simulated crack pixel;

[0101] is the corresponding coordinate of the real crack point on the image of the structure to be detected;

[0102] x s ,y s is the starting point coordinate of the simulated crack;

[0103] The pixel value of the simulated crack pixel point is the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate a simulated crack. The fixed pixel value coefficient here generally ranges from 0.5 to 1.5, and can also be determined by calculating the ratio of the brightness of the image of the structure to be detected to the brightness of the real crack image.

[0104] The coordinates of the simulated crack pixel points are obtained by the translation and rotation operations of the real crack pixel point coordinates, which can ensure that the shape and extension direction of the generated simulated crack are consistent with the actual situation of the actual engineering test.

[0105] The steps of generating the simulated crack image are as follows: on the image of the structure to be measured, the pixel value at the coordinate of the simulated crack pixel point is taken as the pixel value of the corresponding pixel point of the simulated crack to generate the simulated crack image.

[0106] In order to make the color of the simulated crack pixel point better integrated with the color of the overall pixel point of the measured structure image, the simulated crack image is opened and closed, and the simulated crack is color corrected and edge fused.

[0107]

[0108]

[0109] in,

[0110] I morpho is the pixel value set of all pixels after the opening and closing operations are performed on the simulated crack image;

[0111] I original is the original pixel value set of all pixels of the simulated crack image;

[0112] α is the fixed pixel value coefficient;

[0113] kernel is a rectangular neighborhood matrix for opening and closing operations. is an opening operation and Θ is a closing operation.

[0114] When training the deep convolutional neural network model, a large number of clear simulated crack images can be generated in a short time by repeating the above process, and then used to train the deep convolutional neural network model. A deep learning model with high accuracy and high reliability in crack identification can be obtained. In this way, crack damage on the structure can be detected to avoid dangerous accidents.

[0115] On the other hand, a system for generating a structurally high-simulation damage crack image is provided, such as Figure 2 As shown, including:

[0116] A real structural crack image acquisition module, used to acquire a real structural crack image;

[0117] A real crack recognition module is used to identify real cracks in real structural crack images;

[0118] A real crack information acquisition module, used to obtain real crack information from real cracks;

[0119] The image acquisition module of the structure to be tested is used to acquire the image of the structure to be tested;

[0120] A target position recognition module for a structure to be measured is used to recognize the position of the target to be measured in the image of the structure to be measured;

[0121] A target position detection frame coordinate acquisition module is used to obtain the detection frame contour coordinates from the target position to be measured;

[0122] A simulated crack generation module is used to generate simulated cracks using real crack information and its detection frame contour coordinates;

[0123] The simulated crack image generation module is used to obtain the pixel value of the pixel point corresponding to the simulated crack from the pixel value at the coordinate of the simulated crack pixel point on the structure image to be measured, so as to generate a simulated crack image.

[0124] Optionally, in the above-mentioned structural high-simulation simulated damage crack image generation system, in the real crack information acquisition module, real crack information is acquired from the real crack, including the coordinates and pixel values ​​of the real crack pixel points.

[0125] Optionally, in the above-mentioned structural high-simulation simulated damage crack image generation system, the simulated crack generation module is used to generate simulated cracks using real crack information and its detection frame coordinates, specifically:

[0126] Select any point in the detection frame contour coordinates as the starting point of the simulated crack;

[0127] Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected;

[0128] Calculate the angle that the simulated crack pixel needs to rotate to

[0129] Calculate the coordinates of the simulated crack pixel points;

[0130] The pixel value of the simulated crack pixel point is taken as the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate the simulated crack.

[0131] Optionally, in the above-mentioned structural high-simulation simulated damage crack image generation system, in the simulated crack generation module, the angle that the simulated crack pixel point needs to be rotated is calculated, specifically,

[0132]

[0133] in,

[0134] ang is the angle that the simulated crack pixel needs to rotate;

[0135] k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame;

[0136] k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack;

[0137] Calculate the coordinates of the simulated crack pixel points, specifically,

[0138]

[0139]

[0140] in,

[0141] is the coordinate of the simulated crack pixel;

[0142] is the corresponding coordinate of the real crack point on the image of the structure to be detected;

[0143] x s ,y s are the starting coordinates of the simulated crack.

[0144] Optionally, in the above-mentioned structural high-simulation simulated damage crack image generation system, in the simulated crack image generation module, opening and closing operations are performed on the simulated crack image, and color correction and edge fusion are performed on the simulated crack.

[0145] Regarding the structural high-simulation simulated damage crack image generation system disclosed in the above-mentioned embodiment, since it corresponds to the structural high-simulation simulated damage crack image generation method disclosed in the above-mentioned embodiment, the description is relatively simple. For specific related matters, please refer to the relevant instructions in the part of the structural high-simulation simulated damage crack image generation method. Its technical effects can also refer to the technical effects of the relevant parts of the structural high-simulation simulated damage crack image generation method, which will not be repeated here.

[0146] In addition, technicians in the field should also be able to realize that the various modules and units of the device disclosed in the embodiments of the present application can be implemented by electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, this application generally describes them according to their functions. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Technicians in the field can choose to adopt different methods to implement the described functions for each specific application and its actual constraints, but such implementation should not be considered to be beyond the scope of this application.

[0147] So far, the technical solution of the present application has been described in conjunction with the preferred embodiments shown in the accompanying drawings. Those skilled in the art should understand that the scope of protection of the present application is obviously not limited to these specific embodiments. Without departing from the principles of the present application, those skilled in the art can make equivalent changes or substitutions to the relevant technical features, and the technical solutions after these changes or substitutions will fall within the scope of protection of the present application.

Claims

1. A method for generating a highly realistic simulated damage crack image on a structure, characterized in that: include: Real structural crack image acquisition steps: acquiring a real structural crack image; Real crack identification step: identifying real cracks in real structural crack images; Steps for obtaining real crack information: obtaining real crack information from real cracks; Step of acquiring the image of the structure to be tested: acquiring the image of the structure to be tested; The step of identifying the position of the target structure to be measured: identifying the position of the target structure to be measured in the image of the structure to be measured; Target position detection frame coordinate acquisition step: from the target position to be measured, obtain the detection frame outline coordinates; Simulated crack generation steps: using real crack information and its detection frame contour coordinates to generate simulated cracks; The steps of generating the simulated crack image are as follows: on the image of the structure to be measured, the pixel value at the coordinate of the simulated crack pixel point is taken as the pixel value of the corresponding pixel point of the simulated crack to generate the simulated crack image.

2. The method for generating a highly realistic simulated damage crack image on a structure according to claim 1, characterized in that: In the step of obtaining the real crack information, the real crack information is obtained from the real crack, including the coordinates and pixel values ​​of the real crack pixel points.

3. The method for generating a highly realistic simulated damage crack image on a structure according to claim 2, characterized in that: The steps of simulating crack generation are as follows: Select any point in the detection frame contour coordinates as the starting point of the simulated crack; Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected; Calculate the angle that the simulated crack pixel needs to rotate: Calculate the coordinates of the simulated crack pixel points; The pixel value of the simulated crack pixel point is taken as the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate the simulated crack.

4. The method for generating a highly realistic simulated damage crack image on a structure according to claim 3, characterized in that: In the step of simulating crack generation, the angle that the simulated crack pixel needs to rotate is calculated as follows: in, ang is the angle that the simulated crack pixel needs to rotate; k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame; k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack; Calculate the coordinates of the simulated crack pixel points, specifically: in, is the coordinate of the simulated crack pixel; is the corresponding coordinate of the real crack point on the image of the structure to be detected; x s ,y s are the starting coordinates of the simulated crack.

5. The method for generating a highly realistic simulated damage crack image on a structure according to claim 4, characterized in that: In the step of generating the simulated crack image, an opening and closing operation is performed on the simulated crack image, and color correction and edge fusion are performed on the simulated crack.

6. A structural high-simulation damage crack image generation system, characterized in that: include: A real structural crack image acquisition module, used to acquire a real structural crack image; A real crack recognition module is used to identify real cracks in real structural crack images; A real crack information acquisition module, used to obtain real crack information from real cracks; The image acquisition module of the structure to be tested is used to acquire the image of the structure to be tested; A target position recognition module for a structure to be measured is used to recognize the position of the target to be measured in the image of the structure to be measured; A target position detection frame coordinate acquisition module is used to obtain the detection frame contour coordinates from the target position to be measured; A simulated crack generation module is used to generate simulated cracks using real crack information and its detection frame contour coordinates; The simulated crack image generation module is used to obtain the pixel value of the pixel point corresponding to the simulated crack from the pixel value at the coordinate of the simulated crack pixel point on the structure image to be measured, so as to generate a simulated crack image.

7. The structural high-simulation damage crack image generation system according to claim 6, characterized in that: In the real crack information acquisition module, real crack information is acquired from the real crack, including coordinates and pixel values ​​of real crack pixel points.

8. The structural high-simulation damage crack image generation system according to claim 7, characterized in that: In the simulated crack generation module, the real crack information and its detection frame contour coordinates are used to generate simulated cracks, specifically: Select any point in the detection frame contour coordinates as the starting point of the simulated crack; Calculate the relative distance between the coordinates of the real crack pixel point and the coordinates of the starting point of the simulated crack to obtain the corresponding coordinates of the real crack pixel point on the image of the structure to be detected; Calculate the angle that the simulated crack pixel needs to rotate to Calculate the coordinates of the simulated crack pixel points; The pixel value of the simulated crack pixel point is taken as the product of the pixel value of the corresponding pixel point on the real crack and the fixed pixel value coefficient to generate the simulated crack.

9. The structural high-simulation damage crack image generation system according to claim 8, characterized in that: In the simulated crack generation module, the angle that the simulated crack pixel needs to rotate is calculated, specifically, in, ang is the angle that the simulated crack pixel needs to rotate; k1 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of the center point of the detection frame; k2 is the vector slope from the coordinates of the starting point of the simulated crack to the coordinates of any pixel point of the real crack; Calculate the coordinates of the simulated crack pixel points, specifically, in, is the coordinate of the simulated crack pixel; is the corresponding coordinate of the real crack point on the image of the structure to be detected; x s ,y s are the starting coordinates of the simulated crack.

10. The structural high-simulation damage crack image generation system according to claim 9, characterized in that: In the simulated crack image generation module, the simulated crack image is opened and closed, and the simulated crack is color corrected and edge fused.

Images