Multi-module parallel metal surface defect detection method based on data buffering
By employing a multi-module parallel detection method with data buffering in steel plate surface defect detection, and utilizing multiple sub-processes and multiple models for detection, the problems of large computational load and poor complementarity of multi-model inference are solved, thus achieving fast and accurate defect detection.
Patent Information
- Application Number
- CN202410991722.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-23
- Publication Date
- 2024-11-22
AI Technical Summary
In existing technologies, multi-model inference in steel plate surface defect detection involves large computational loads and poor complementarity, resulting in slow detection speed and poor timeliness, making it difficult to meet the requirements of real-time response and accurate classification.
A multi-module parallel detection method based on data buffering is adopted, which divides the image processing process into a main process and multiple sub-processes. Defect detection is performed through the collaborative work of the processes, and multiple sub-processes are used for parallel processing. Multiple models are combined to detect different types of defects, and data transfer and filtering are realized through data buffer.
It achieves parallel processing of multi-model detection, improves detection speed and accuracy, can quickly detect large defects and deeply explore small defects, prevents missed detections, and meets the needs of real-time detection.
Smart Images

Figure CN119006891A_ABST
Abstract
Description
Technical Field
[0001] The invention relates to the technical field of machine vision detection, and in particular to a multi-module parallel metal surface defect detection method based on data buffering. Background Art
[0002] In the production process of medium and thick plates, various defects are often prone to appear on their surfaces, which will seriously affect product quality and further damage the company's reputation and economic benefits. Therefore, in order to prevent low-quality steel plates from flowing into the hands of downstream users, it is necessary to accurately detect defects on the surface of steel plates and issue alarms in a timely manner, so as to achieve effective tracking and early warning of surface defects and ensure that the quality of steel plates meets the strict requirements of the market and customers.
[0003] However, steel plate surface defect detection faces multiple challenges. Interference factors such as water stains and iron oxide make detection much more difficult. Traditional detection methods, such as using a single model for detection, often have poor results, and have problems such as insufficient detection capabilities, poor timeliness, and inaccurate classification. Since there are many types of steel plate defects, each defect has different shapes and characteristics, and the focus of data is also different, it is difficult to meet comprehensive and detailed detection needs by relying solely on a single model.
[0004] Although multi-model reasoning has emerged in the prior art, this method still has many shortcomings in practical applications. For example, multi-model reasoning will lead to a sharp increase in the amount of calculation, the complementarity between the models is not strong, the detection speed is affected, and it is difficult to respond in time. These problems limit the practical application of multi-model reasoning in steel plate surface defect detection. Summary of the invention
[0005] The present invention provides a multi-module parallel metal surface defect detection method based on data buffering to solve the technical problem that multi-model reasoning is limited in the practical application of steel plate surface defect detection.
[0006] In order to solve the above technical problems, the present invention provides the following technical solutions:
[0007] On the one hand, the present invention provides a multi-module parallel metal surface defect detection method based on data buffering. The method uses the data buffer as a link, allocates the image processing process into a main process and multiple sub-processes, and completes the metal surface defect detection through the collaborative work between the processes; wherein the sub-processes include a defect detection sub-process and a contour detection sub-process; the multi-module parallel metal surface defect detection method includes:
[0008] Acquire an image of a metal plate to be detected as an image to be detected;
[0009] The image to be detected is stored in the buffer of the defect detection subprocess and the buffer of the contour detection subprocess respectively; the defect detection subprocess and the contour detection subprocess are parallel; wherein the defect detection subprocess performs the defect detection task, detects the defect data on the surface of the metal plate in the image, and stores the detected defect data in the buffer of the main process; the contour detection subprocess performs the contour detection task, detects the contour of the metal plate in the image;
[0010] The main process determines the defect category and filters the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, and stores the filtered defect data in the system database.
[0011] Further, the acquiring of the image of the metal plate to be detected as the image to be detected includes:
[0012] Constructing a metal surface defect detection system; wherein the metal surface defect detection system comprises a plurality of linear array cameras, and the acquisition positions of the plurality of linear array cameras are parallel and collinear after calibration;
[0013] The pulse signal output by the laser velocimeter is used to synchronously trigger each camera to collect the surface image of the metal plate. After the overlapping area is removed, the real-time frame images collected by each camera are horizontally spliced and used as the image to be detected.
[0014] Furthermore, the number of the defect detection subprocesses is multiple; each defect detection subprocess is parallel;
[0015] The process of storing the image to be inspected into the buffer of the defect detection subprocess includes:
[0016] Set the index value of the image to be detected and number each defect detection subprocess; wherein the index value of the image to be detected is set sequentially starting from 0, and the number of the defect detection subprocess is numbered sequentially starting from 0;
[0017] According to the index values of the images to be detected, the images to be detected are evenly distributed to multiple defect detection sub-processes.
[0018] Furthermore, according to the index value of the image to be detected, the image to be detected is evenly distributed to multiple defect detection sub-processes as follows: the index value corresponding to the image to be distributed is modulo the number of defect detection sub-processes, and assuming that the remainder obtained is i, the image to be distributed is distributed to the i-th defect detection sub-process.
[0019] Furthermore, each defect detection sub-process includes a first-order defect detection process and a second-order defect detection process; in each defect detection sub-process, the image to be detected is first stored in a buffer corresponding to the first-order defect detection process; the first-order defect detection process first processes the image in its buffer, detects the defect data of the metal surface in the image, and stores the detected defect data in the buffer of the main process, and at the same time stores the image in its buffer and the detected defect data in the buffer corresponding to the second-order defect detection process, and then the second-order defect detection process processes the image in its buffer, detects the defect data of the metal surface in the image, and compares the detected defect data with the defect data detected by the first-order defect detection process, screens out the defect data that is repeated with the defect data detected by the first-order defect detection process, and then stores the remaining defect data in the buffer of the main process.
[0020] Furthermore, the first-order defect detection process processes the image in its buffer and detects the defect data of the metal surface in the image in a single-model multi-scale detection method. During the detection, the image to be detected is firstly cropped multiple times along its width direction with different widths to obtain multiple sub-images; then the obtained sub-images are uniformly normalized to the size required by the model and input into the preset model to obtain the defect detection results;
[0021] The second-order defect detection process processes the images in its buffer and detects the defect data of the metal surface in the image using a multi-model single-scale detection method. During the detection, the image to be detected is first cropped with a fixed width along its width direction to obtain multiple sub-images of the same size to form a sub-image set; then the obtained sub-image sets are input into different preset models for defect detection to obtain defect detection results, and the defect detection results output by each model are fused through non-maximum suppression.
[0022] Furthermore, the second-order defect detection process compares the defect data detected by it with the defect data detected by the first-order defect detection process, and screens out the defect data that are repeated with the defect data detected by the first-order defect detection process in the following manner: the second-order defect detection process calculates the IOU intersection-over-union ratio of each defect data detected by it and each defect data detected by the first-order defect detection process; if the calculated IOU intersection-over-union ratio is greater than a preset threshold, the corresponding defect data detected by the second-order defect detection process is screened out.
[0023] Furthermore, the main process determines the category of the defect and screens the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, including:
[0024] The main process first classifies the defect data in its buffer according to the preset classification standard, and then judges the classified defect data. If it contains defect data of the preset type that needs to be further classified, it is further classified to determine the category of the defect;
[0025] After determining the defect category, the boundary of the defect is refined through the preset semantic segmentation model;
[0026] Based on the location and size of the defects after boundary refinement, the defect data is screened in combination with the contour output by the contour detection subprocess.
[0027] Furthermore, the preset semantic segmentation model is a PIDNet segmentation model, and the samples during its training are annotated as polygonal areas composed of a number of points along the exact boundary of the defect; after the preset semantic segmentation model completes boundary refinement, the position coordinates of the defect are replaced with the refined boundary position.
[0028] Furthermore, the defect data is screened based on the position and size of the defect after boundary refinement, in combination with the contour output by the contour detection subprocess, including:
[0029] Screen out defects whose size is smaller than a preset size threshold;
[0030] Compare the relative relationship between the defect position and the metal plate contour and screen out the defects outside the area to be inspected.
[0031] On the other hand, the present invention further provides an electronic device, comprising a processor and a memory; wherein the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the above method.
[0032] In yet another aspect, the present invention further provides a computer-readable storage medium, wherein at least one instruction is stored in the storage medium, and the instruction is loaded and executed by a processor to implement the above method.
[0033] The beneficial effects brought about by the technical solution provided by the present invention include at least:
[0034] The solution of the present invention adopts a processing method of multiple sub-processes, so that multiple models can be processed in parallel, and a single model can be processed in a shunt manner, thereby greatly ensuring real-time performance. Different models in the multi-model structure are used to process different types of defects, thereby achieving complementary processing and taking into account both large defects and subtle defects. In the classification process, the categories are determined as large categories and fine categories, so as to determine more accurate categories. On the one hand, this method can quickly detect defects that are easy to detect in a timely manner, and on the other hand, it can run more models with in-depth mining in the background to detect small defects and prevent missed detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0035] In order to more clearly illustrate the technical solutions in the embodiments of the present invention, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.
[0036] Figure 1 It is a schematic diagram of the execution flow of a multi-module parallel metal surface defect detection method based on data buffering provided by an embodiment of the present invention;
[0037] Figure 2 It is a multi-camera stitched image of the same surface provided by an embodiment of the present invention;
[0038] Figure 3 It is a schematic diagram of the principle of the boundary refinement model provided by an embodiment of the present invention;
[0039] Figure 4 It is a system block diagram of an electronic device provided by an embodiment of the present invention. DETAILED DESCRIPTION
[0040] In order to make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described in detail below with reference to the accompanying drawings.
[0041] First of all, it should be noted that in the embodiments of the present invention, words such as "exemplarily" and "for example" are used to indicate examples, illustrations or explanations. Any embodiment or design described as "exemplary" in the present invention should not be interpreted as being more preferred or more advantageous than other embodiments or designs. Specifically, the use of the word "exemplarily" is intended to present the concept in a concrete way. In addition, in the embodiments of the present invention, the meaning expressed by "and / or" can be both, or it can be either of the two.
[0042] First embodiment
[0043] This embodiment provides a multi-module parallel metal surface defect detection method based on data buffering. With the data buffer as the link, the image processing process is divided into a main process and multiple sub-processes, and the metal surface defect detection is completed through the collaborative work between the processes; wherein the sub-processes include a defect detection sub-process and a contour detection sub-process; the method can be implemented by an electronic device, which can be a terminal or a server. The execution flow of the multi-module parallel metal surface defect detection method is as follows: Figure 1 As shown, the following steps are included:
[0044] S1, obtaining an image of a metal plate to be detected as an image to be detected;
[0045] Taking the surface defect detection system of the wide and thick plate production line as an example, the four linear array cameras in this surface defect detection system that shoot the same steel plate surface are calibrated to have parallel and collinear acquisition positions, and the resolution of each camera is 4096pix 1024pix, the pulse signal output by the laser velocimeter is used to synchronously trigger the camera acquisition. After the overlapping areas are removed, the real-time frame images collected by each camera are horizontally spliced and used as the basic image for subsequent data processing, such as Figure 2 As shown, in this embodiment, the size of the basic image is 15600 pix 1024pix.
[0046] S2, storing the image to be detected in the buffer of the defect detection subprocess and the buffer of the contour detection subprocess respectively; the defect detection subprocess and the contour detection subprocess are parallel; wherein the defect detection subprocess performs the defect detection task, detects the defect data on the surface of the metal plate in the image, and stores the detected defect data in the buffer of the main process; the contour detection subprocess performs the contour detection task, detects the contour of the metal plate in the image;
[0047] Furthermore, the number of the defect detection subprocesses is multiple; each defect detection subprocess is parallel; on the one hand, the image data to be detected is evenly distributed to multiple defect detection subprocesses and stored in the buffer of the corresponding process, and on the other hand, all data are sequentially input into the buffer corresponding to the contour detection subprocess for extracting the contour coordinates of the steel plate. Specifically, the process of storing the image to be detected in the buffer of the defect detection subprocess includes: setting the index value of the image to be detected and numbering each defect detection subprocess; wherein the index value of the image to be detected is set sequentially starting from 0, and the number of the defect detection subprocess is sequentially numbered starting from 0; according to the index value of the image to be detected, the image to be detected is evenly distributed to multiple defect detection subprocesses. According to the index value of the image to be detected, the method of evenly distributing the image to be detected to multiple defect detection subprocesses is as follows: the index value corresponding to the image to be distributed is modulo the number of defect detection subprocesses, and assuming the remainder is i, the image to be distributed is distributed to the i-th defect detection subprocess.
[0048] Furthermore, each defect detection sub-process includes a first-order defect detection process and a second-order defect detection process; in each defect detection sub-process, the image to be detected is first stored in a buffer corresponding to the first-order defect detection process; the first-order defect detection process first processes the image in its buffer, detects the defect data of the metal surface in the image, and stores the detected defect data in the buffer of the main process, and at the same time stores the image in its buffer and the detected defect data in the buffer corresponding to the second-order defect detection process, and then the second-order defect detection process processes the image in its buffer, detects the defect data of the metal surface in the image, and compares the detected defect data with the defect data detected by the first-order defect detection process, screens out the defect data that is repeated with the defect data detected by the first-order defect detection process, and then stores the remaining defect data in the buffer of the main process.
[0049] Furthermore, the first-order defect detection process detects defect data in the image in a single-model multi-scale detection manner, which mainly detects defects with complete contours, obvious features, and intuitive morphology, focuses on large defects, and uses the entire sample set to train a target detection model. During reasoning, the image to be detected is first cropped multiple times with different widths along its width direction to obtain multiple sub-images; then the obtained sub-images are uniformly normalized to the size required by the model and input into the preset model to obtain the defect detection results.
[0050] Specifically, in this embodiment, the basic image is first traversed and cropped with a height size of 1 (i.e., 1024), and there is a 128-pixel overlap between adjacent crops. If it is less than 1024, it is cropped from the right to the left, wherein the first image is intercepted from 0 to 1024, the second image is intercepted from 896 to 1920, the third image is intercepted from 1792 to 2816, ..., and the 18th image is intercepted from 14576 to 15600. The basic image is then traversed and cropped with a height size of 2 (i.e., 2048), and there is a 128-pixel overlap between adjacent crops. If it is less than 2048, it is cropped from the right to the left, wherein the first image is intercepted from 0 to 2048, the second image is intercepted from 1920 to 3968, the third image is intercepted from 3840 to 5888, ..., and the eighth image is intercepted from 13552 to 15600. Finally, a total of 26 images are normalized to 640pix The image is then fed into the model to get the result, and the coordinates of the output are mapped back to the original image.
[0051] The second-order defect detection process detects defect data in images in a multi-model single-scale detection manner, mainly detecting distributed, slender, and inconspicuous defects, focusing on small defects, and training multiple target detection models by selecting different sample data subsets. During detection, the image to be detected is first cropped along its width direction with a fixed width to obtain multiple sub-images of the same size to form a sub-image set; then the obtained sub-image set is input into different preset models for defect detection to obtain defect detection results, and the defect detection results output by each model are fused by non-maximum suppression. Specifically, in this embodiment, when the second-order defect detection process detects defect data in an image, the base image is traversed and cropped at 1 times the height size (i.e., 1024), and there is an overlap of 128 pixels between adjacent crops.
[0052] Among them, the models of the first-order defect detection process and the second-order defect detection process both use YOLOV8 as the target detection model. Furthermore, the second-order defect detection process compares the defect data detected by it with the defect data detected by the first-order defect detection process, and the defect data that is repeated with the defect data detected by the first-order defect detection process is screened out in the following way: the second-order defect detection process calculates the IOU intersection-over-union ratio of each defect data detected by it and each defect data detected by the first-order defect detection process, and if the calculated IOU intersection-over-union ratio is greater than 0.9, the corresponding defect data detected by the second-order defect detection process is screened out.
[0053] S3, the main process determines the category of the defect and screens the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, and stores the screened defects in the system database;
[0054] Specifically, in this embodiment, the defect classification process in the main process includes a full-category classification model and multiple groups of fine classification models; the main process determines the category of the defect and screens the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, including:
[0055] S31, the main process firstly infers the defect data in its buffer according to the preset classification standard through the full-category classification model to obtain the basic category of the defect; then judges the classified defect data, and judges whether to perform the refined classification step according to the basic defect category. If the defect data contains the preset type that needs to be further subdivided, it is further subdivided to determine the category of the defect;
[0056] Among them, the full-category classification model contains all possible categories of defects, which is used for basic qualitative analysis of the defect categories; the output category needs to be judged whether it belongs to the easily mixed defect category. If it is an easily mixed category, it enters the corresponding fine classification model for secondary classification. The easily mixed category is determined by the production site. Specifically, in this embodiment, the defect categories included in the full-category classification model include finishing roll marks, work roll marks, pits, foreign body indentations, inclusions, scars, warping, folds, scratches, longitudinal cracks, transverse cracks, star-shaped cracks, hairline cracks, edge cracks, scratches, water marks, etc. The fine classification model includes: 1) a classification model for accurately distinguishing pits, foreign body indentations, and inclusions; 2) a binary classification model for accurately distinguishing star-shaped cracks and hairline cracks; 3) a binary classification model for accurately distinguishing longitudinal cracks and water marks.
[0057] S32, after determining the defect category, refining the defect boundary through a preset semantic segmentation model;
[0058] Specifically, in this embodiment, the model used to refine the boundary of the defect is a semantic segmentation model (PIDNet segmentation model is selected in this embodiment), and the sample is annotated as a polygonal area composed of several points along the exact boundary of the defect. The area definition category of the defect is 1, and the background area definition category is 0. After the boundary refinement model, the position coordinates of the defect are replaced with the refined boundary position, such as Figure 3 shown.
[0059] S33, based on the position information and size information of the defect whose boundary is refined, combined with the contour output by the contour detection subprocess, the defects that do not need to be paid attention to are eliminated, and the retained defect data is stored in the system database;
[0060] Specifically, based on the position and size of the defect after boundary refinement, combined with the contour output by the contour detection subprocess, the defect data is screened and the process of eliminating defects that do not need to be concerned includes:
[0061] S331, by comparing the size of the defect to see if it meets the minimum size requirement, such as setting the size threshold for detecting scratches to 1mm 20mm, then the defects whose size is smaller than the set threshold will be filtered out;
[0062] S332, by comparing the relative relationship between the defect position and the steel plate contour, the defects outside the area to be detected are filtered. Specifically, in this embodiment, on the one hand, defects are filtered according to the distance from the head and tail of the defect and whether it is in the head and tail arc area, such as filtering out defects within 300mm from the head and tail of the steel plate; on the other hand, defects are filtered based on whether the distance from the defect to both sides of the steel plate is within the set threshold, such as defects within 20mm of the edge. Since the edge trimming process will be performed later, the defects in this area can be filtered out.
[0063] In summary, this embodiment designs a multi-module parallel processing method with a buffer mechanism as a link to realize metal surface defect detection. The main process and multiple sub-processes work together through data buffering, and a multi-stage model detection strategy is used to detect different types of defects in a targeted manner at different stages. The first-order detection model is used to detect large defects, and the second-order detection model is used to detect small defects. The purpose of "first coarse and then fine" is achieved during detection. First, the rapid alarm of large defects is guaranteed and accurate contours are given, and then small and subtle defects are detected to supplement the detection, thereby greatly improving the defect detection capability. In the main process, a unified defect recognition channel is used to complete cascade classification, obtain refined defect categories, and effectively filter out defects that do not need to be concerned, thereby ensuring that the recognition results meet user needs and improving the practicality of defect detection results.
[0064] Second embodiment
[0065] This embodiment provides an electronic device, such as Figure 4 As shown, the electronic device includes: a processor and a memory; wherein the processor and the memory can be connected via a communication bus; the memory stores at least one instruction, and the instruction is loaded and executed by the processor to implement the method of the first embodiment. In addition, the electronic device may also include a transceiver, the processor and the transceiver can be connected via a communication bus, and the transceiver is used to communicate with other devices.
[0066] Next, combine Figure 4 The following is a detailed introduction to the various components of the electronic device:
[0067] Among them, the processor is the control center of the electronic device, and the electronic device may include multiple processors, each of which may be a single-core processor (single-CPU) or a multi-core processor (multi-CPU). The processor here may be a processor or a general term for multiple processing elements. For example, the processor is one or more central processing units (CPUs), or other general-purpose processors, application specific integrated circuits (ASICs), or one or more integrated circuits configured to implement an embodiment of the present invention, such as one or more microprocessors (digital signal processors, DSPs), or one or more field programmable gate arrays (field programmable gate arrays, FPGAs), or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor, etc. The processor may execute various functions of the electronic device by running or executing software programs stored in the memory and calling data stored in the memory.
[0068] In a specific implementation, as an embodiment, the processor may include one or more CPUs, such as Figure 4 The CPU0 and CPU1 shown in the figure are, of course, only exemplary.
[0069] The memory is used to store the software program for executing the solution of the present invention, and the execution is controlled by the processor. The specific implementation method can refer to the above method embodiment and will not be repeated here.
[0070] Optionally, the memory may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may be integrated with the processor or exist independently and accessed through the interface circuit ( Figure 4 The processor is coupled to the processor (not shown), which is not specifically limited in this embodiment of the present invention.
[0071] The transceiver may include a receiver and a transmitter ( Figure 4 The receiver is used to implement the receiving function, and the transmitter is used to implement the sending function. The transceiver can be integrated with the processor or exist independently and communicate with the electronic device through the interface circuit ( Figure 4 (not shown) is coupled to the processor, which is not specifically limited in this embodiment of the present invention.
[0072] In addition, it should be noted that Figure 4 The structure of the electronic device shown in the figure does not constitute a limitation on the device, and the actual device may include more or fewer components than shown in the figure, or combine certain components, or arrange the components differently. In addition, the technical effects achieved by the electronic device when executing the method of the first embodiment above can refer to the technical effects described in the first embodiment above, so they are not repeated here.
[0073] Third embodiment
[0074] This embodiment provides a computer-readable storage medium, which stores at least one instruction, and the instruction is loaded and executed by a processor to implement the method of the first embodiment. The computer-readable storage medium may be a ROM, a random access memory, a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc. The instructions stored therein may be loaded by a processor in a terminal to execute the method.
[0075] In addition, it should be noted that the present invention can be provided as a method, an apparatus or a computer program product. Therefore, the embodiment of the present invention can be in the form of a full or partial hardware embodiment, a full or partial software embodiment or an embodiment combining software and hardware. Moreover, when implemented using software, the embodiment of the present invention can be in the form of a computer program product implemented on one or more computer-usable storage media containing computer-usable program codes. The computer program product includes one or more computer instructions or computer programs. When the computer instructions or computer programs are loaded or executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center by wired (e.g., infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center containing one or more available media sets. The available medium may be a magnetic medium (eg, a floppy disk, a hard disk, a magnetic tape), an optical medium (eg, a DVD), or a semiconductor medium. The semiconductor medium may be a solid state drive.
[0076] The embodiments of the present invention are described with reference to the flowcharts and / or block diagrams of the methods, terminal devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.
[0077] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device, so that a series of operation steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable terminal device provide for implementing the process in the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.
[0078] It should also be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. The terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that the process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of more restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements. In addition, the term "and / or" is only an association relationship describing the associated objects, indicating that there can be three relationships, for example, A and / or B, which can represent: A exists alone, A and B exist at the same time, and B exists alone, wherein A and B can be singular or plural. In addition, the character " / " in this article generally indicates that the objects before and after are in an "or" relationship, but it may also indicate an "and / or" relationship. Please refer to the context for specific understanding. "At least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, at least one of a, b or c can be represented by: a, b, c, ab, ac, bc, or abc, where a, b, c can be single or plural.
[0079] In addition, it can be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.
[0080] Those skilled in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.
[0081] In several embodiments provided by the present invention, it should be understood that the disclosed equipment, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of functional modules / units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point, the coupling or direct coupling or communication connection between each other shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms. The unit described as a separate component may or may not be physically separated, and the component displayed as a unit may or may not be a physical unit, that is, it may be located in one place, or it may be distributed on multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the scheme of this embodiment. In addition, each functional unit in each embodiment of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit.
[0082] If the method is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.
[0083] Finally, it should be noted that the above is only a preferred embodiment of the present invention. It should be pointed out that although the preferred embodiment of the present invention has been described, for ordinary technicians in this technical field, once the basic creative concept of the present invention is known, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the protection scope of the present invention. Therefore, the attached claims are intended to be interpreted as including the preferred embodiment and all changes and modifications that fall within the scope of the embodiments of the present invention.
Claims
1. A multi-module parallel metal surface defect detection method based on data buffering, characterized in that: Using the data buffer as a link, the image processing process is divided into a main process and multiple sub-processes, and the metal surface defect detection is completed through the collaborative work between the processes; wherein the sub-processes include a defect detection sub-process and a contour detection sub-process; the multi-module parallel metal surface defect detection method based on data buffering includes: Acquire an image of a metal plate to be detected as an image to be detected; The image to be detected is stored in the buffer of the defect detection subprocess and the buffer of the contour detection subprocess respectively; the defect detection subprocess and the contour detection subprocess are parallel; wherein the defect detection subprocess performs the defect detection task, detects the defect data on the surface of the metal plate in the image, and stores the detected defect data in the buffer of the main process; the contour detection subprocess performs the contour detection task, detects the contour of the metal plate in the image; The main process determines the defect category and filters the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, and stores the filtered defect data in the system database.
2. The multi-module parallel metal surface defect detection method based on data buffering according to claim 1, characterized in that: The step of acquiring an image of the metal plate to be detected as the image to be detected includes: Constructing a metal surface defect detection system; wherein the metal surface defect detection system comprises a plurality of linear array cameras, and the acquisition positions of the plurality of linear array cameras are parallel and collinear after calibration; The pulse signal output by the laser velocimeter is used to synchronously trigger each camera to collect the surface image of the metal plate. After the overlapping area is removed, the real-time frame images collected by each camera are horizontally spliced and used as the image to be detected.
3. The multi-module parallel metal surface defect detection method based on data buffering according to claim 1, characterized in that: The number of the defect detection subprocesses is multiple; Each defect detection subprocess runs in parallel; The process of storing the image to be inspected into the buffer of the defect detection subprocess includes: Set the index value of the image to be detected and number each defect detection subprocess; wherein the index value of the image to be detected is set sequentially starting from 0, and the number of the defect detection subprocess is numbered sequentially starting from 0; According to the index values of the images to be detected, the images to be detected are evenly distributed to multiple defect detection sub-processes.
4. The multi-module parallel metal surface defect detection method based on data buffering according to claim 3, characterized in that: The method of evenly distributing the image to be detected to multiple defect detection sub-processes based on the index value of the image to be detected is: taking the index value corresponding to the image to be distributed modulo the number of defect detection sub-processes, and assuming that the remainder obtained is i, the image to be distributed is distributed to the i-th defect detection sub-process.
5. The multi-module parallel metal surface defect detection method based on data buffering according to claim 3, characterized in that: Each defect detection sub-process includes a first-order defect detection process and a second-order defect detection process; In each defect detection sub-process, the image to be detected is first stored in the buffer corresponding to the first-order defect detection process; the first-order defect detection process first processes the image in its buffer, detects the defect data of the metal surface in the image, and stores the detected defect data in the buffer of the main process, and at the same time stores the image in its buffer and the detected defect data in the buffer corresponding to the second-order defect detection process; then the second-order defect detection process processes the image in its buffer, detects the defect data of the metal surface in the image, and compares the detected defect data with the defect data detected by the first-order defect detection process, screens out the defect data that is repeated with the defect data detected by the first-order defect detection process, and then stores the remaining defect data in the buffer of the main process.
6. The multi-module parallel metal surface defect detection method based on data buffering according to claim 5, characterized in that: The first-order defect detection process processes the image in its buffer and detects the defect data of the metal surface in the image in a single-model multi-scale detection method. During the detection, the image to be detected is first cropped multiple times along its width direction with different widths to obtain multiple sub-images; then the obtained sub-images are uniformly normalized to the size required by the model and input into the preset model to obtain the defect detection results; The second-order defect detection process processes the images in its buffer and detects the defect data of the metal surface in the image using a multi-model single-scale detection method. During the detection, the image to be detected is first cropped with a fixed width along its width direction to obtain multiple sub-images of the same size to form a sub-image set; then the obtained sub-image sets are input into different preset models for defect detection to obtain defect detection results, and the defect detection results output by each model are fused through non-maximum suppression.
7. The multi-module parallel metal surface defect detection method based on data buffering according to claim 5, characterized in that: The second-order defect detection process compares the defect data detected by it with the defect data detected by the first-order defect detection process, and screens out the defect data that are repeated with the defect data detected by the first-order defect detection process in the following way: the second-order defect detection process calculates the IOU intersection-over-union ratio of each defect data detected by it with each defect data detected by the first-order defect detection process; if the calculated IOU intersection-over-union ratio is greater than a preset threshold, the corresponding defect data detected by the second-order defect detection process is screened out.
8. The multi-module parallel metal surface defect detection method based on data buffering according to claim 1, characterized in that: The main process determines the category of the defect and screens the defect data based on the defect data output by the defect detection subprocess and the contour output by the contour detection subprocess, including: The main process first classifies the defect data in its buffer according to the preset classification standard, and then judges the classified defect data. If it contains defect data of the preset type that needs to be further classified, it is further classified to determine the category of the defect; After determining the category of the defect, the boundary of the defect is refined through the preset semantic segmentation model; Based on the location and size of the defects after boundary refinement, the defect data is screened in combination with the contour output by the contour detection subprocess.
9. The multi-module parallel metal surface defect detection method based on data buffering according to claim 8, characterized in that: The preset semantic segmentation model is a PIDNet segmentation model, and the samples during its training are labeled as polygonal areas composed of several points along the exact boundary of the defect; after the preset semantic segmentation model completes boundary refinement, the position coordinates of the defect are replaced with the refined boundary position.
10. The multi-module parallel metal surface defect detection method based on data buffering according to claim 8, characterized in that: The defect data is screened based on the position and size of the defect after boundary refinement is completed, combined with the contour output by the contour detection subprocess, including: Screen out defects whose size is smaller than a preset size threshold; Compare the relative relationship between the defect position and the metal plate contour and screen out the defects outside the area to be inspected.
Citation Information
Cited By
Real-time monitoring and early warning method and system for key impurities based on photoelectric impurity removal of tobacco lamina
CN119624962A
