A plate processing method, device, terminal equipment and storage medium

Abstract

The present invention discloses a plate processing method, a device terminal device and a storage medium, the method comprising: obtaining a first image and a second image of a current detection surface of a plate to be processed; the first image is obtained by photographing a camera device; the second image is detected by an ultrasonic device; initial crack recognition is performed according to the first image, and the identified crack is used as a suspected crack; when it is determined that the suspected crack has spread to the edge of the current detection surface, the corresponding edge is used as a target edge, and the side corresponding to the target edge is used as a selected side; obtaining a third image and a fourth image of the selected side; the third image is obtained by photographing a camera device, and the fourth image is detected by an ultrasonic device; each image is input into a preset crack identification model to identify whether the suspected crack is a true crack; when it is determined that the suspected crack is a true crack, crack repair processing is performed on the plate to be processed during the processing process. By implementing the present invention, the probability of errors in subsequent processing steps can be reduced.

Description

Technical Field

[0001] The present invention relates to the field of plate processing, and in particular to a plate processing method, device, terminal equipment and storage medium. Background Art

[0002] In the process of plate processing, it is usually necessary to first identify the defects of the plate, and then determine the processing steps required in the plate processing process according to the identified defects to repair the defects of the plate;

[0003] Cracks are common defects in plates. With the rapid development of artificial intelligence, existing technologies usually use cameras to acquire images of plates, then extract features from the images and identify cracks on the surface of the plates based on neural networks.

[0004] However, in the actual processing environment, due to the influence of light or artificially drawn lines, line-like shadows will be left on the plate. Since the plate image obtained by the camera device can only extract information on the surface of the plate, these non-crack shadows can easily be mistakenly identified as cracks in the plate during subsequent identification, resulting in misjudgment and errors in subsequent processing steps. Summary of the invention

[0005] The embodiments of the present invention provide a plate processing method, apparatus, terminal device and computer-readable storage medium, which can improve the accuracy of plate surface crack identification and thereby reduce the probability of errors in subsequent processing steps.

[0006] An embodiment of the present invention provides a plate processing method, comprising: obtaining a first image and a second image of a current detection surface of a plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves to the current detection surface;

[0007] Performing initial crack recognition according to the first image, when a crack is recognized, treating the recognized crack as a suspected crack, and identifying an area where the suspected crack is located;

[0008] When it is determined that the suspected crack has spread to the edge of the current detection surface according to the area where the suspected crack is located, the edge to which the suspected crack has spread is used as the target edge, and the side surface that intersects with the current detection surface and forms the target edge is used as the selected side surface;

[0009] Acquire a third image and a fourth image of the selected side surface; wherein the third image is obtained by photographing the selected side surface by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface;

[0010] Inputting the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fused feature, and fuses the image features of the second image and the third image to obtain a second fused feature; identifying whether the suspected crack is a real crack according to the first fused feature and the second fused feature;

[0011] When it is determined that the suspected crack is a real crack, the plate to be processed is subjected to crack repair processing during the processing.

[0012] Further, when it is determined that the suspected crack has not spread to the edge of the current detection surface, determining the main direction of the suspected crack in the first image;

[0013] According to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side surface intersecting with the current detection surface and forming the target edge is used as the selected side surface.

[0014] Furthermore, determining the main direction of the suspected crack in the first image includes:

[0015] Extracting an image of a region where a suspected crack is located to obtain an image of a suspected crack region;

[0016] Performing threshold segmentation on the suspected crack region image to generate a binary image of the suspected crack;

[0017] Perform skeleton extraction on the binary image of suspected cracks to obtain the skeleton image of the cracks;

[0018] For each pixel in the skeleton image, the gradient direction of the pixel is calculated, and then the local area corresponding to the pixel is set, and the average gradient direction of all pixels in the local area is calculated, and the average gradient direction is used as the local direction of the pixel;

[0019] According to the local orientation of all pixels in the skeleton image, the distribution of the number of pixels in different orientations is counted, and then the main orientation of the suspected crack is determined according to the distribution of the number of pixels in different orientations.

[0020] Further, the crack identification model includes a first identification sub-model and a second identification sub-model; and the identifying whether the suspected crack is a real crack according to the first fusion feature and the second fusion feature includes:

[0021] Identify the first fusion feature through the first identification sub-model to generate a first identification result;

[0022] Identify the second fusion feature by using the second identification sub-model to generate a second identification result;

[0023] When it is determined that both the first identification result and the second identification result are true cracks, the suspected crack is determined to be a true crack; otherwise, the suspected crack is determined to be a false crack.

[0024] Furthermore, the crack identification and recognition model is generated after adversarial training based on the positive sample set and the negative sample set;

[0025] Each of the positive sample sets includes: a first image sample of a first detection surface where a real crack exists, a second image sample of the first detection surface, a third image sample of a first selected side surface corresponding to the first detection surface, and a fourth image sample of the first selected side surface; wherein the first image sample and the third image sample are obtained by taking a camera, and the second image sample and the fourth image sample are obtained by detecting an ultrasonic device;

[0026] Each of the negative sample sets includes: a fifth image sample of a second detection surface where a virtual crack exists, a sixth image sample of the second detection surface, a seventh image sample of a second selected side surface corresponding to the second detection surface, and an eighth image sample of the second selected side surface; wherein the fifth image sample and the seventh image sample are obtained by taking a camera device; the sixth image sample and the eighth image sample are obtained by detecting an ultrasonic device; and a virtual crack is set in the second detection surface.

[0027] Further, the identifying the area where the suspected crack is located includes:

[0028] Extracting the grayscale value of each pixel in the first image;

[0029] Pixels with grayscale values ​​greater than a preset threshold are regarded as abnormal pixels;

[0030] Select any abnormal pixel point as the seed point, and then start from the seed point and traverse the adjacent pixels one by one. During the traversal process, calculate the Euclidean distance between the currently traversed pixel point and the reference point of the current area according to the grayscale value of the pixel point. If the Euclidean distance is less than the threshold, add the currently traversed pixel point to the crack area, and set the currently traversed pixel point as the new reference point. After all the pixel points are traversed, the area where the suspected crack is located is obtained; wherein, the initial reference point is the seed point.

[0031] Based on the above-mentioned embodiments of the invention, corresponding device embodiments are provided;

[0032] An embodiment of the present invention provides a plate processing device, comprising: a first image acquisition module, a suspected crack recognition module, a first selected side recognition module, a second image acquisition module, a crack identification module, and a repair module;

[0033] The first image acquisition module is used to acquire a first image and a second image of the current detection surface of the plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves;

[0034] The suspected crack identification module is used to perform initial crack identification according to the first image, and when a crack is identified, the identified crack is regarded as a suspected crack, and an area where the suspected crack is located is identified;

[0035] The first selected side recognition module is used to, when it is determined that the suspected crack has spread to the edge of the current detection surface according to the area where the suspected crack is located, use the edge to which the suspected crack has spread as the target edge, and use the side that intersects with the current detection surface and forms the target edge as the selected side;

[0036] The second image acquisition module is used to acquire a third image and a fourth image of the selected side surface; wherein the third image is obtained by photographing the selected side surface by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface;

[0037] The crack identification module is used to input the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fused feature, and fuses the image features of the second image and the third image to obtain a second fused feature; and identify whether the suspected crack is a real crack according to the first fused feature and the second fused feature;

[0038] The repair module is used to perform crack repair processing on the plate to be processed during the processing process when the suspected crack is determined to be a real crack.

[0039] Furthermore, the plate processing device further includes: a second selected side recognition module;

[0040] The second selected side recognition module is used to determine the main direction of the suspected crack in the first image when it is determined that the suspected crack has not spread to the edge of the current detection surface;

[0041] According to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side surface intersecting with the current detection surface and forming the target edge is used as the selected side surface.

[0042] Based on the above method embodiment, a terminal device is provided, which includes a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor. When the processor executes the computer program, it implements any one of the sheet metal processing methods described in the present invention.

[0043] Based on the above method item embodiments, the present invention provides a corresponding storage medium item embodiment. Another embodiment of the present invention provides a storage medium, wherein the storage medium includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the plate processing method described in any method item embodiment of the present invention.

[0044] The following beneficial effects are achieved by implementing the embodiments of the present invention:

[0045] The embodiment of the present invention provides a plate processing method, device, terminal equipment and storage medium. The method obtains a first image of the current detection surface of the plate to be processed by a camera device, extracts a second image of the current detection surface by an ultrasonic device, and then performs initial crack recognition based on the first image. The crack is recognized as a suspected crack. At this time, the crack is recognized based on the first image taken by the camera device, which may be misjudged due to the influence of factors such as light. Therefore, it is necessary to further identify the suspected crack. Specifically, if the suspected crack has spread to the edge of the current detection surface, if the suspected crack is a real crack, the corresponding side surface must also have a crack on the surface at this time. If it is a false crack caused by the shadow of light on the detection surface, it generally does not affect the corresponding side surface. Therefore, it can be based on the side. The crack situation is compared with the situation of the detection surface to improve the accuracy of crack identification. In addition, ultrasonic detection can detect the internal structure of the object. The comparison of the image obtained by ultrasonic detection can further improve the accuracy. For this reason, the present invention synchronously obtains the third image of the selected side taken by the camera device and the fourth image of the selected side detected by the ultrasonic device; then the first image, the second image, the third image and the fourth image obtained are input into the crack identification model, the crack identification model fuses the first image and the fourth image to obtain the first fusion feature, and fuses the second image and the third image to obtain the second fusion feature. Finally, the suspected crack is judged according to the first fusion feature and the second fusion feature, and finally it is determined whether the suspected crack is a real crack. When it is determined to be a real crack, the crack is repaired during the processing. In the actual scene, when a detection surface has a crack, if a camera device is used for detection, the image of the current detection surface should be directly collected, so that more surface image information of the detection surface can be obtained. When ultrasonic waves are used for detection, it is better to emit ultrasonic waves from the side for detection. This is because when ultrasonic waves propagate in the plate, cracks will reflect, scatter and refract the ultrasonic waves. Emitting ultrasonic waves from the side can irradiate the crack more directly and produce a more obvious reflection signal, which is conducive to the accurate detection and positioning of the crack. Based on the above reasons, the present invention creatively fuses the first image of the detection surface and the fourth image of the selected side, and fuses the second image of the detection surface and the third image of the selected side, thereby realizing the feature fusion of the front camera image and the side ultrasonic image, further improving the accuracy of crack identification, and thus ensuring the accuracy of subsequent version processing. In summary, this invention first improves the accuracy of crack identification based on the combination of ultrasonic images and camera images, and secondly, according to the location information of the crack, collects the camera image and ultrasonic image of the corresponding side and combines them with the image of the detection surface to further improve the accuracy of crack identification, and again, in the process of combining the image of the detection surface with the image of the side, the cross-feature fusion of the front camera image and the side ultrasonic image is adopted, thereby further improving the accuracy of plate crack identification. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] Figure 1 It is a schematic flow chart of a plate processing method provided by one embodiment of the present invention.

[0047] Figure 2 It is a schematic diagram of a real crack on a plate provided by an embodiment of the present invention.

[0048] Figure 3 It is a schematic diagram of a false crack on a plate provided by an embodiment of the present invention.

[0049] Figure 4 It is a schematic diagram of a suspected crack on a plate provided by an embodiment of the present invention.

[0050] Figure 5 It is a structural schematic diagram of a plate processing device provided by one embodiment of the present invention. DETAILED DESCRIPTION

[0051] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0052] like Figure 1 As shown, an embodiment of the present invention provides a plate processing method, comprising:

[0053] S1: Acquire a first image and a second image of the current detection surface of the plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves to the current detection surface.

[0054] Specifically, a pre-set camera device is used to face the current inspection surface of the plate to be processed and shoot it to obtain the above-mentioned first image. The texture features of the current inspection surface of the plate to be processed can be subsequently identified through the first image; then an ultrasonic device is used to emit ultrasonic waves to the current inspection surface for detection to generate a second image. The internal structural features of the version to be processed can be learned through the second image.

[0055] S2: Performing initial crack recognition based on the first image, when a crack is recognized, treating the recognized crack as a suspected crack, and identifying an area where the suspected crack is located.

[0056] Specifically, image recognition is performed based on the first image, cracks in the first image are detected, and the detected cracks are used as suspected cracks. When the initial crack recognition is performed based on the first image, the existing crack recognition model can be used for recognition, or the cracks in the first image can be recognized through morphological operations, which are based on the shape and structural features of the image. First, the first image is preprocessed, including operations such as image grayscale and binarization, so as to facilitate the application of morphological operations. Then, the preset structural element is convolved with the first image through an erosion operation, and the cracks can be refined into thin lines through multiple iterative erosion operations. Finally, the structural element is convolved with the image through an expansion operation, and the cracks can be connected and the voids inside the cracks can be filled through multiple iterative expansion operations, so as to identify the cracks.

[0057] In a preferred embodiment, the cracks may be identified and the area where the suspected cracks are located may be determined by: extracting the gray value of each pixel in the first image;

[0058] Pixels with grayscale values ​​greater than a preset threshold are regarded as abnormal pixels;

[0059] Select any abnormal pixel point as the seed point, and then start from the seed point and traverse the adjacent pixels one by one. During the traversal process, calculate the Euclidean distance between the currently traversed pixel point and the reference point of the current area according to the grayscale value of the pixel point. If the Euclidean distance is less than the threshold, add the currently traversed pixel point to the crack area, and set the currently traversed pixel point as the new reference point. After all the pixel points are traversed, the area where the suspected crack is located is obtained; wherein, the initial reference point is the seed point.

[0060] By using the above region growing method, the suspected crack in the first image can be identified, and the region where the crack is located in the first image can be determined.

[0061] S3: When it is determined that the suspected crack has spread to the edge of the current detection surface based on the area where the suspected crack is located, the edge to which the suspected crack has spread is taken as the target edge, and the side that intersects with the current detection surface and forms the target edge is taken as the selected side.

[0062] Specific as Figure 2 or Figure 3 As shown, when the suspected crack 1 propagates to the edge 2 of the current detection surface, the edge 2 is used as the target edge, and the side 3 that intersects with the current detection surface to form the target edge is used as the selected side. In the present invention, the intersection of the suspected crack area and the edge of the detection surface is determined. If the suspected crack area and the edge of the detection surface intersect, it is determined that the suspected crack has propagated to the edge of the current detection surface. At this time, according to the above diagram, the corresponding selected side is selected.

[0063] The reason why the selected side is determined in the above manner is that the side that intersects with the crack direction can be selected for ultrasonic detection so that the ultrasonic wave can be irradiated onto the crack more directly, thereby enhancing the strength and clarity of the crack signal. Assuming that the direction of the crack is from left to right, then the ultrasonic wave should be selected for ultrasonic detection from the right side, so that when the ultrasonic wave is irradiated from the right side, it can more easily pass through the crack and generate a clear reflection signal, which is conducive to detecting and locating the crack. Therefore, the present invention selects the side that intersects with the crack direction as the selected side, which can further improve the accuracy of crack identification in the future.

[0064] In another embodiment of the present invention, when it is determined that the suspected crack has not spread to the edge of the current detection surface, a main direction of the suspected crack in the first image is determined;

[0065] According to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side surface intersecting with the current detection surface and forming the target edge is used as the selected side surface;

[0066] like Figure 4 As shown, schematically, if the suspected crack has not spread to the edge of the current detection surface, a side surface must still be selected for subsequent operations. At this time, the main direction of the suspected crack can be identified. According to the main direction of the suspected crack, it is predicted that if the suspected crack 1 is extended along its direction, the edge 2 that will eventually intersect with the current detection surface will be used as the target edge, and then the selected side surface 3 is obtained.

[0067] In a preferred embodiment, the main direction of the suspected crack in the first image may be determined by: extracting an image of the area where the suspected crack is located to obtain an image of the suspected crack area;

[0068] Performing threshold segmentation on the suspected crack region image to generate a binary image of the suspected crack;

[0069] Perform skeleton extraction on the binary image of suspected cracks to obtain the skeleton image of the cracks;

[0070] For each pixel in the skeleton image, the gradient direction of the pixel is calculated, and then the local area corresponding to the pixel is set, and the average gradient direction of all pixels in the local area is calculated, and the average gradient direction is used as the local direction of the pixel;

[0071] According to the local orientation of all pixels in the skeleton image, the distribution of the number of pixels in different orientations is counted, and then the main orientation of the suspected crack is determined according to the distribution of the number of pixels in different orientations.

[0072] S4: Acquire the third image and the fourth image of the selected side; wherein the third image is obtained by photographing the selected side by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface.

[0073] Specifically, after the selected side is determined, the selected side is photographed by a camera device to obtain a third image; assuming that the suspected crack has spread to the edge of the current detection surface, if it is a real crack, then Figure 2 As shown in FIG. 1 , cracks will also appear on the selected side. Then, the third image collected at this time will have crack features. If it is a shadow caused by ambient light, then Figure 3 As shown in the figure, at this time, there will be no cracks on the selected side, so the third image collected at this time will have crack features. Therefore, the features of the third image can be compared with the features of the second image of the current detection surface to finally determine whether the crack is real or fake. If, assuming that the suspected crack has not spread to the edge of the current detection surface, then Figure 4 As shown, at this time, no matter whether the crack is real or fake, there will be no crack features in the third image of the selected side, and further identification can be performed based on the ultrasonic detection image later.

[0074] In addition, the fourth image mentioned above can be obtained by detecting the selected side through an ultrasonic detection device. The reason for performing ultrasonic detection on the selected side is to compare it with the first image of the current detection surface later. In addition, the selected side is the side that intersects with the suspected crack, or the side that intersects with the extension of its direction. Performing ultrasonic detection from this side can make the ultrasonic wave irradiate the crack more directly, thereby enhancing the strength and clarity of the crack signal.

[0075] S5: Input the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fusion feature, and fuses the image features of the second image and the third image to obtain a second fusion feature; identify whether the suspected crack is a real crack based on the first fusion feature and the second fusion feature.

[0076] The crack identification model provided by the present invention is generated after adversarial training based on a positive sample set and a negative sample set, each of the positive sample sets includes: a first image sample of a first detection surface with a real crack, a second image sample of the first detection surface, a third image sample of a first selected side surface corresponding to the first detection surface, and a fourth image sample of the first selected side surface; wherein the first image sample and the third image sample are obtained by taking a camera device, and the second image sample and the fourth image sample are obtained by detecting an ultrasonic device;

[0077] Each of the negative sample sets includes: a fifth image sample of a second detection surface where a virtual crack exists, a sixth image sample of the second detection surface, a seventh image sample of a second selected side surface corresponding to the second detection surface, and an eighth image sample of the second selected side surface; wherein the fifth image sample and the seventh image sample are obtained by taking a camera device; the sixth image sample and the eighth image sample are obtained by detecting an ultrasonic device; and a virtual crack is set in the second detection surface.

[0078] The virtual cracks can be drawn on the detection surface manually or generated by setting a strip shadow on the detection surface by ambient light. The model learns the feature differences between real and fake samples through adversarial training to acquire the ability to distinguish real and fake cracks.

[0079] There are two sub-models in the crack identification model, including: a first identification sub-model and a second identification sub-model; in the crack identification process, the first identification sub-model is used to identify the first fusion feature to generate a first identification result; the second identification sub-model is used to identify the second fusion feature to generate a second identification result; when it is determined that the first identification result and the second identification result are both true cracks, the suspected crack is determined to be a true crack, otherwise the suspected crack is determined to be a false crack. The identification results of the two sub-models are compared to improve accuracy, and the identification results of each sub-model are obtained after identification based on the fusion features of a front camera image and a corresponding side ultrasonic detection image. The fusion of the front camera image and the corresponding side ultrasonic detection image further improves the accuracy of crack identification.

[0080] S6: When it is determined that the suspected crack is a real crack, the plate to be processed is subjected to crack repair treatment during the processing.

[0081] In a possible implementation, when it is determined that the pseudo-crack is a real crack, corresponding equipment is controlled, such as a robotic arm to clamp a filler (such as wood filler, expansion glue, etc.), and the crack is filled according to the area where the crack is located to achieve crack repair processing.

[0082] Based on the above method embodiment, a corresponding device embodiment is provided;

[0083] like Figure 5 An embodiment of the present invention provides a plate processing device, comprising: a first image acquisition module, a suspected crack recognition module, a first selected side recognition module, a second image acquisition module, a crack identification module and a repair module;

[0084] The first image acquisition module is used to acquire a first image and a second image of the current detection surface of the plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves;

[0085] The suspected crack identification module is used to perform initial crack identification according to the first image, and when a crack is identified, the identified crack is regarded as a suspected crack, and an area where the suspected crack is located is identified;

[0086] The first selected side recognition module is used to, when it is determined that the suspected crack has spread to the edge of the current detection surface according to the area where the suspected crack is located, use the edge to which the suspected crack has spread as the target edge, and use the side that intersects with the current detection surface and forms the target edge as the selected side;

[0087] The second image acquisition module is used to acquire a third image and a fourth image of the selected side surface; wherein the third image is obtained by photographing the selected side surface by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface;

[0088] The crack identification module is used to input the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fused feature, and fuses the image features of the second image and the third image to obtain a second fused feature; and identify whether the suspected crack is a real crack according to the first fused feature and the second fused feature;

[0089] The repair module is used to perform crack repair processing on the plate to be processed during the processing process when the suspected crack is determined to be a real crack.

[0090] In a preferred embodiment, it further comprises: a second selected side identification module;

[0091] The second selected side recognition module is used to determine the main direction of the suspected crack in the first image when it is determined that the suspected crack has not spread to the edge of the current detection surface;

[0092] According to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side surface intersecting with the current detection surface and forming the target edge is used as the selected side surface.

[0093] It should be noted that the above-mentioned device item embodiment corresponds to the method item embodiment of the present invention, and it can implement the plate processing method described in any one of the above-mentioned method item embodiments of the present invention. The device embodiments described above are merely schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, in the drawings of the device embodiments provided by the present invention, the connection relationship between the modules indicates that there is a communication connection between them, which can be specifically implemented as one or more communication buses or signal lines. A person of ordinary skill in the art can understand and implement it without creative work.

[0094] Based on the above method embodiment, a corresponding terminal device embodiment is provided.

[0095] An embodiment of the present invention provides a terminal device, including a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, and when the processor executes the computer program, the plate processing method described in any embodiment of the present invention is implemented.

[0096] It should be noted that the terminal device mentioned here can be a computing device such as a desktop computer, a notebook, a PDA, and a cloud server. The terminal device may include, but is not limited to, a processor and a memory. Those skilled in the art will understand that, for example, it may also include input and output devices, network access devices, buses, etc.

[0097] The processor may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. The general-purpose processor may be a microprocessor or any conventional processor, etc. The processor is the control center of the terminal device, and uses various interfaces and lines to connect various parts of the entire terminal device.

[0098] The memory can be used to store the computer program and / or module, and the processor realizes various functions of the terminal device by running or executing the computer program and / or module stored in the memory, and calling the data stored in the memory. The memory can mainly include a program storage area and a data storage area, wherein the program storage area can store an operating system, an application required for at least one function (such as a sound playback function, an image playback function, etc.), etc.; the data storage area can store data created according to the use of the mobile phone (such as audio data, a phone book, etc.), etc. In addition, the memory can include a high-speed random access memory, and can also include a non-volatile memory, such as a hard disk, a memory, a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), at least one disk storage device, a flash memory device, or other volatile solid-state storage devices.

[0099] Based on the above method item embodiment, a corresponding storage medium item embodiment is provided.

[0100] An embodiment of the present invention provides a storage medium, which includes a stored computer program, wherein when the computer program is running, the device where the storage medium is located is controlled to execute the plate processing method described in any method embodiment of the present invention.

[0101] The above storage medium is a computer-readable storage medium. The present invention implements all or part of the processes in the above embodiment method, and can also be completed by instructing related hardware through a computer program. The computer program can be stored in a computer-readable storage medium. When the computer program is executed by the processor, the steps of each of the above method embodiments can be implemented. Among them, the computer program includes computer program code, and the computer program code can be in source code form, object code form, executable file or some intermediate form. The computer-readable medium may include: any entity or device that can carry the computer program code, recording medium, U disk, mobile hard disk, disk, optical disk, computer memory, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), electrical carrier signal, telecommunication signal and software distribution medium, etc.

[0102] The above is a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications are also considered to be within the scope of protection of the present invention.

Claims

1. A plate processing method, characterized in that: include: Acquire a first image and a second image of the current detection surface of the plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves to the current detection surface; Performing initial crack recognition according to the first image, when a crack is recognized, treating the recognized crack as a suspected crack, and identifying an area where the suspected crack is located; When it is determined that the suspected crack has spread to the edge of the current detection surface according to the area where the suspected crack is located, the edge to which the suspected crack has spread is used as the target edge, and the side surface that intersects with the current detection surface and forms the target edge is used as the selected side surface; When it is determined that the suspected crack has not spread to the edge of the current detection surface, determine the main direction of the suspected crack in the first image; according to the main direction of the suspected crack, take the edge of the current detection surface intersected by the suspected crack after extending to the main direction as the target edge, and take the side surface that intersects with the current detection surface and forms the target edge as the selected side surface; Acquire a third image and a fourth image of the selected side surface; wherein the third image is obtained by photographing the selected side surface by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface; Inputting the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fused feature, and fuses the image features of the second image and the third image to obtain a second fused feature; identifying whether the suspected crack is a real crack according to the first fused feature and the second fused feature; When it is determined that the suspected crack is a real crack, performing crack repair processing on the plate to be processed during the processing; Wherein, determining the main direction of the suspected crack in the first image includes: Extracting an image of a region where a suspected crack is located to obtain an image of a suspected crack region; Performing threshold segmentation on the suspected crack region image to generate a binary image of the suspected crack; Perform skeleton extraction on the binary image of suspected cracks to obtain the skeleton image of the cracks; For each pixel in the skeleton image, the gradient direction of the pixel is calculated, and then the local area corresponding to the pixel is set, and the average gradient direction of all pixels in the local area is calculated, and the average gradient direction is used as the local direction of the pixel; According to the local orientation of all pixels in the skeleton image, the distribution of the number of pixels in different orientations is counted, and then the main orientation of the suspected crack is determined according to the distribution of the number of pixels in different orientations.

2. The sheet material processing method according to claim 1, characterized in that: The crack identification model includes a first identification sub-model and a second identification sub-model; and the identifying whether the suspected crack is a real crack according to the first fusion feature and the second fusion feature includes: Identify the first fusion feature through the first identification sub-model to generate a first identification result; Identify the second fusion feature by using the second identification sub-model to generate a second identification result; When it is determined that both the first identification result and the second identification result are true cracks, the suspected crack is determined to be a true crack; otherwise, the suspected crack is determined to be a false crack.

3. The sheet material processing method according to claim 2, characterized in that: The crack identification model is generated after adversarial training based on a positive sample set and a negative sample set; Each of the positive sample sets includes: a first image sample of a first detection surface where a real crack exists, a second image sample of the first detection surface, a third image sample of a first selected side surface corresponding to the first detection surface, and a fourth image sample of the first selected side surface; wherein the first image sample and the third image sample are obtained by taking a camera, and the second image sample and the fourth image sample are obtained by detecting an ultrasonic device; Each of the negative sample sets includes: a fifth image sample of a second detection surface where a virtual crack exists, a sixth image sample of the second detection surface, a seventh image sample of a second selected side surface corresponding to the second detection surface, and an eighth image sample of the second selected side surface; wherein the fifth image sample and the seventh image sample are obtained by taking a camera device; the sixth image sample and the eighth image sample are obtained by detecting an ultrasonic device; and a virtual crack is set in the second detection surface.

4. The plate processing method according to claim 3, characterized in that: The step of identifying the area where the suspected crack is located includes: Extracting the grayscale value of each pixel in the first image; Pixels with grayscale values ​​greater than a preset threshold are regarded as abnormal pixels; Select any abnormal pixel point as the seed point, and then start from the seed point and traverse the adjacent pixels one by one. During the traversal process, calculate the Euclidean distance between the currently traversed pixel point and the reference point of the current area according to the grayscale value of the pixel point. If the Euclidean distance is less than the threshold, add the currently traversed pixel point to the crack area, and set the currently traversed pixel point as the new reference point. After all the pixel points are traversed, the area where the suspected crack is located is obtained; wherein, the initial reference point is the seed point.

5. A plate processing device, characterized in that: include: A first image acquisition module, a suspected crack recognition module, a first selected side recognition module, a second selected side recognition module, a second image acquisition module, a crack identification module, and a repair module; The first image acquisition module is used to acquire a first image and a second image of the current detection surface of the plate to be processed; wherein the first image is obtained by photographing the current detection surface by a camera device; and the second image is obtained by detecting the current detection surface after an ultrasonic device emits ultrasonic waves; The suspected crack identification module is used to perform initial crack identification according to the first image, and when a crack is identified, the identified crack is regarded as a suspected crack, and an area where the suspected crack is located is identified; The first selected side recognition module is used to, when it is determined that the suspected crack has spread to the edge of the current detection surface according to the area where the suspected crack is located, use the edge to which the suspected crack has spread as the target edge, and use the side that intersects with the current detection surface and forms the target edge as the selected side; The second selected side recognition module is used to determine the main direction of the suspected crack in the first image when it is determined that the suspected crack has not spread to the edge of the current detection surface; according to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side that intersects with the current detection surface and forms the target edge is used as the selected side; The second image acquisition module is used to acquire a third image and a fourth image of the selected side surface; wherein the third image is obtained by photographing the selected side surface by a camera device, and the fourth image is obtained by detecting after an ultrasonic device emits ultrasonic waves to the selected detection surface; The crack identification module is used to input the first image, the second image, the third image and the fourth image into a preset crack identification model, so that the crack identification model fuses the image features of the first image and the fourth image respectively to obtain a first fused feature, and fuses the image features of the second image and the third image to obtain a second fused feature; and identify whether the suspected crack is a real crack according to the first fused feature and the second fused feature; The repair module is used to perform crack repair processing on the plate to be processed during the processing process when the suspected crack is determined to be a real crack; Wherein, determining the main direction of the suspected crack in the first image includes: Extracting an image of a region where a suspected crack is located to obtain an image of a suspected crack region; Performing threshold segmentation on the suspected crack region image to generate a binary image of the suspected crack; Perform skeleton extraction on the binary image of suspected cracks to obtain the skeleton image of the cracks; For each pixel in the skeleton image, the gradient direction of the pixel is calculated, and then the local area corresponding to the pixel is set, and the average gradient direction of all pixels in the local area is calculated, and the average gradient direction is used as the local direction of the pixel; According to the local orientation of all pixels in the skeleton image, the distribution of the number of pixels in different orientations is counted, and then the main orientation of the suspected crack is determined according to the distribution of the number of pixels in different orientations.

6. The plate processing device according to claim 5, characterized in that: Also includes: a second selected side identification module; The second selected side recognition module is used to determine the main direction of the suspected crack in the first image when it is determined that the suspected crack has not spread to the edge of the current detection surface; According to the main direction of the suspected crack, the edge of the current detection surface intersected by the suspected crack after extending to the main direction is used as the target edge, and the side surface intersecting with the current detection surface and forming the target edge is used as the selected side surface.

7. A terminal device, characterized in that: The method comprises a processor, a memory, and a computer program stored in the memory and configured to be executed by the processor, wherein the processor implements the plate processing method according to any one of claims 1 to 4 when executing the computer program.

8. A storage medium, characterized in that: The storage medium includes a stored computer program, wherein when the computer program is executed, the device where the storage medium is located is controlled to execute the plate processing method according to any one of claims 1 to 4.

Images