Surface defect statistics method, device and storage medium for metal surface

Abstract

The present application discloses a method, device and storage medium for counting surface defects of metal surfaces. A method for counting surface defects of metal surfaces includes: performing real-time detection on the metal surface of a metal sheet to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area and coordinates of each first surface defect; and performing statistics on the defect information to determine periodic defects and construct a defect distribution map, wherein periodic defects are first surface defects of the same defect type with a predetermined distance interval between adjacent first surface defects and a number greater than or equal to three, and wherein the difference in defect length, defect width and defect area between each first surface defect corresponding to the periodic defect is less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

Description

Technical Field

[0001] The present application relates to the field of data processing technology, and in particular to a surface defect statistics method, device and storage medium for metal surfaces. Background Art

[0002] During the production process of metal sheets, various types of surface defects will appear on the metal surface of the metal sheets, and surface defects have an adverse effect on the quality and performance of industrial products. Therefore, it is necessary to count surface defects so as to inspect and control them. In the prior art, surface defects are usually counted by constructing a defect distribution map. However, the defect distribution map in the prior art only reflects the location and defect type of surface defects, which is relatively rigid and cannot be flexibly applied, and has great limitations. For example, when producing metal sheets, a failure in the production equipment may cause surface defects on the metal surface of the metal sheet. The surface defects thus generated can only be judged manually, which is time-consuming and labor-intensive, with high labor costs. In addition, due to the complexity of the types of surface defects, manual identification may result in inaccurate identification.

[0003] With respect to the technical problem in the prior art of low accuracy in determining whether a surface defect is caused by a production equipment failure, no effective solution has been proposed so far. Summary of the invention

[0004] The embodiments of the present application provide a surface defect statistics method, device and storage medium for a metal surface, so as to at least solve the technical problem in the prior art of low accuracy in determining whether a surface defect is caused by a production equipment failure.

[0005] According to one aspect of an embodiment of the present application, a surface defect statistics method for a metal surface is provided, comprising: performing real-time detection on a metal surface of a metal plate to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area and coordinates of each first surface defect; and performing statistics on the defect information to determine periodic defects and construct a defect distribution map, wherein periodic defects are first surface defects of the same defect type that are spaced at a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in defect length, defect width and defect area between each first surface defect corresponding to the periodic defect are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0006] According to another aspect of an embodiment of the present application, a storage medium is further provided, the storage medium including a stored program, wherein the above-described method is executed by a processor when the program is running.

[0007] According to another aspect of an embodiment of the present application, a surface defect statistics device for a metal surface is also provided, comprising: a first determination module, used to perform real-time detection on the metal surface of a metal plate, and determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect; and a second determination module, used to perform statistics on the defect information, determine periodic defects, and construct a defect distribution map, wherein periodic defects are first surface defects of the same defect type that are separated by a predetermined distance from adjacent first surface defects and whose number is greater than or equal to three, and wherein the differences in defect length, defect width, and defect area between each first surface defect corresponding to the periodic defect are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0008] According to another aspect of an embodiment of the present application, a surface defect statistics device for a metal surface is also provided, comprising: a processor; and a memory, connected to the processor, for providing the processor with instructions for processing the following processing steps: performing real-time detection on the metal surface of a metal sheet to determine defect information of a first surface defect on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect; and performing statistics on the defect information to determine periodic defects, and construct a defect distribution map, wherein the periodic defects are first surface defects of the same defect type that are spaced at a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in defect length, defect width, and defect area between each first surface defect corresponding to the periodic defects are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0009] In the embodiment of the present application, the surface defect statistics system uses the statistics module to perform real-time statistics on the surface defects on the metal surface of the metal sheet, so that in the process of producing the metal sheet, the statistical information of the metal surface can be obtained, and the real-time nature of the statistical data is ensured. In addition, in the process of statistical surface defects, the technical solution counts periodic defects. Since periodic defects are caused by the failure of production equipment during the production of metal sheets, the technical solution selects periodic defects from the surface defects to determine whether the production equipment is faulty, repair the production equipment in time, and reduce the surface defects of the metal sheet. Compared with the prior art, there is no need for manual judgment, which reduces manpower and material resources, and improves the accuracy of distinguishing surface defects caused by production equipment failure. In addition, the defect distribution map with periodic defects added is more flexible and has fewer limitations. In addition, the technical problem of low accuracy in judging whether the surface defects are caused by production equipment failure in the prior art is solved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0011] Figure 1 is a hardware structure block diagram of a computing device for implementing the method according to Embodiment 1 of the present application;

[0012] Figure 2 is a schematic diagram of a surface defect statistics system according to Example 1 of the present application;

[0013] Figure 3 It is a schematic flow chart of the surface defect statistics method of a metal surface according to the first aspect of Example 1 of the present application;

[0014] Figure 4 is a schematic diagram of a partial defect distribution diagram according to Example 1 of the present application;

[0015] Figure 5 is another schematic diagram of a partial defect distribution diagram according to Example 1 of the present application;

[0016] Figure 6 is a schematic diagram of periodic defects according to a partial defect distribution diagram described in Example 1 of the present application;

[0017] Figure 7 is a schematic diagram of a partial defect trend statistical diagram according to Example 1 of the present application;

[0018] Fig. 8A is a schematic diagram of the surface defect detection model according to Example 1 of the present application;

[0019] Figure 8B is another architecture diagram of the surface defect detection model according to Example 1 of the present application;

[0020] Fig. 9 is a schematic diagram of a process of extracting features from a first tensor to be encoded according to Embodiment 1 of the present application;

[0021] Fig. 10A is a schematic diagram of image data corresponding to pixel points in a partial image area of ​​a binary image according to Embodiment 1 of the present application;

[0022] Fig. 10B is a schematic diagram of coordinates corresponding to image data in a partial image area of ​​a binary image according to Embodiment 1 of the present application;

[0023] Fig. 10C is a schematic diagram of part of tensor element data of the first tensor feature to be encoded according to Embodiment 1 of the present application;

[0024] Fig. 10D is a schematic diagram of part of tensor element data of the third tensor feature to be encoded according to Embodiment 1 of the present application;

[0025] Fig.11 is a schematic diagram of the pooling module according to Example 1 of the present application;

[0026] Fig.12 is a schematic diagram of an upsampling layer of the pooling module according to Example 1 of the present application;

[0027] Fig.13 is a schematic diagram of the unified attention fusion module according to Example 1 of the present application;

[0028] Fig.14 is a schematic diagram of a surface defect counting device for a metal surface according to Example 2 of the present application; and

[0029] Fig.15 It is a schematic diagram of a surface defect counting device for a metal surface according to Example 3 of the present application. DETAILED DESCRIPTION

[0030] In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0031] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0032] Example 1

[0033] According to this embodiment, a method embodiment of a surface defect statistics method for a metal surface is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0034] The method embodiment provided in this embodiment can be executed in a mobile terminal, a computer terminal, a server or a similar computing device. Figure 1 FIG. 1 shows a hardware structure block diagram of a computing device for implementing a surface defect statistics method for a metal surface. Figure 1 As shown, the computing device may include one or more processors (the processor may include but is not limited to a processing device such as a microprocessor MCU or a programmable logic device FPGA), a memory for storing data, and a transmission device for communication functions. In addition, it may also include: a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which may be included as one of the ports of the I / O interface), a network interface, a power supply and / or a camera. It can be understood by those skilled in the art that Figure 1 The structure shown is only for illustration and does not limit the structure of the above electronic device. Figure 1 More or fewer components as shown, or with Figure 1 Different configurations shown.

[0035] It should be noted that the one or more processors and / or other data processing circuits described above may generally be referred to herein as "data processing circuitry". The data processing circuitry may be embodied in whole or in part as software, hardware, firmware, or any other combination thereof. In addition, the data processing circuitry may be a single independent processing module, or may be incorporated in whole or in part into any of the other components in the computing device. As described in the embodiments of the present application, the data processing circuitry serves as a processor control (e.g., selection of a variable resistor terminal path connected to an interface).

[0036] The memory can be used to store software programs and modules of application software, such as program instructions / data storage devices corresponding to the surface defect statistics method of the metal surface in the embodiment of the present application. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, the surface defect statistics method of the metal surface of the above-mentioned application is realized. The memory may include a high-speed random access memory, and may also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memory. In some instances, the memory may further include a memory remotely arranged relative to the processor, and these remote memories may be connected to the computing device via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and combinations thereof.

[0037] The transmission device is used to receive or send data via a network. The specific example of the above network may include a wireless network provided by a communication provider of the computing device. In one example, the transmission device includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station so as to communicate with the Internet. In one example, the transmission device can be a radio frequency (Radio Frequency, RF) module, which is used to communicate with the Internet wirelessly.

[0038] The display may be, for example, a touch screen liquid crystal display (LCD) that may enable a user to interact with a user interface of the computing device.

[0039] It should be noted that, in some optional embodiments, the above Figure 1 The computing device shown may include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. Figure 1 This is merely one example of a particular embodiment and is intended to illustrate the types of components that may be present in the computing devices described above.

[0040] Figure 2Schematic diagram of the surface defect statistics system according to this embodiment. Figure 2 As shown, the system includes: a statistical module, a display module and an alarm module.

[0041] The statistical module is used to collect statistics on the defect information of the surface defects of the metal plate, thereby generating a defect distribution map, a defect trend statistical map and a defect statistical table, and then sending the defect distribution map, defect trend statistical map and defect statistical table to the display module.

[0042] The display module is used to receive and display the defect distribution graph, defect trend statistical graph and defect statistical table sent by the statistical module.

[0043] The alarm module is used to receive the alarm information sent by the statistics module and issue an alarm.

[0044] It should be noted that the statistical module, display module and alarm module in the system can all be applied to the hardware structure described above.

[0045] Under the above operating environment, according to the first aspect of this embodiment, a method for counting surface defects of a metal surface is provided. The method comprises: Figure 2 The surface defect statistics system shown in is implemented. Figure 3 A schematic diagram showing the process of the method is shown in FIG. Figure 3 As shown, the method includes:

[0046] S302: Performing real-time detection on the metal surface of the metal plate to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect; and

[0047] S304: Count the defect information, determine periodic defects, and construct a defect distribution map, wherein the periodic defects are first surface defects of the same defect type that are spaced at a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in defect length, defect width, and defect area between each first surface defect corresponding to the periodic defects are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0048] Specifically, in the process of producing metal sheets, it is necessary to count the surface defects of the metal surface of the metal sheets. A plurality of cameras, for example, cameras 1 to L, are arranged above the metal sheets.

[0049] During the production process of metal sheets, the metal sheets move forward, and the cameras 1 to L fixed above the metal sheets take photos of the metal sheets according to the moving distance of the metal sheets. For example, when the metal sheets move forward 0.0001 meters, the cameras 1 to L take photos once. The moving distance of the metal sheets can be determined by a meter.

[0050] Therefore, after the metal sheet is produced, the cameras 1 to L take multiple shots of the metal sheet to generate multiple metal surface images. The number of cameras can be determined according to the actual required metal surface image width and image accuracy.

[0051] Furthermore, cameras 1 to L transmit the generated metal surface images to the surface defect statistics system. The surface defect statistics system receives the metal surface images and detects the metal surface images in real time through the statistics module, thereby determining the defect information of the surface defects (i.e., first surface defects) on the metal surface of the metal sheet. The defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect.

[0052] Furthermore, the statistical module performs statistics on the defect information to generate a defect distribution map corresponding to the metal surface of the metal sheet. Figure 4 FIG. 1 shows a partial defect distribution diagram of the metal surface of the metal sheet in this embodiment, referring to FIG. Figure 4 As shown, the transverse width and longitudinal length of the defect distribution diagram are the same as the actual width and actual length of the metal sheet.

[0053] More specifically, the statistical module maps the position of each surface defect on the metal sheet to the defect distribution map, so that the coordinates of the surface defect on the metal sheet are used as the coordinates of the surface defect on the defect distribution map, and the statistical module sets different graphic identifiers for each defect type to distinguish them. For example, the graphic identifier corresponding to the surface defect "black spot" is The graphic mark corresponding to the surface defect "peeling" is "◆". There are 12 types of surface defects, namely: dirt, abrasion, corrosion, black spots, black stripes, scratches, cracked edges, water marks, prints, sticking marks, peeling and bulges, and there are 12 corresponding graphic marks.

[0054] Further, the statistical module determines the periodic defects according to the defect information. The periodic defects are surface defects of the same defect type with a predetermined distance interval between adjacent surface defects and a number greater than or equal to three. And the difference between the defect length, defect width and defect area between the first surface defects corresponding to the periodic value defect is less than the preset threshold. For example, the statistical module obtains surface defects 1, surface defects 2 and surface defects 3, wherein surface defects 1, surface defects 2 and surface defects 3 are all "black spot" type surface defects, and the difference between the length, width and area of ​​surface defects 1, surface defects 2 and surface defects 3 is less than the predetermined threshold, and surface defects 1 and surface defects 2 are adjacent surface defects, and the distance between them is within the predetermined distance interval range (for example, 0.95m~1m). And surface defects 2 and surface defects 3 are adjacent surface defects, and the distance between them is within the predetermined distance interval range (for example, 0.95m~1m). Thus, the statistical module determines that surface defects 1, surface defects 2 and surface defects 3 are periodic defects. Then, the statistical module connects surface defect 1, surface defect 2 and surface defect 3 in series through straight line marks in the defect distribution diagram, thereby indicating that they are periodic defects.

[0055] Furthermore, the statistics module sends the defect distribution map to the display module, which displays the defect distribution map through the display module, so that the user can view the distribution of each defect through the defect distribution map, thereby performing statistics and displaying the surface defects.

[0056] In addition, when the user places the mouse on the graphic mark of the defect distribution map, the defect distribution map will display a floating frame corresponding to the graphic mark, wherein the floating frame displays the defect type, coordinates and camera information corresponding to the graphic mark.

[0057] For example, refer to Figure 5 As shown, the user places the mouse on a graphic icon of the defect distribution map The defect type "black spot" corresponding to the image mark, coordinates "x:125.4mm, y:458.4m", and camera information "camera 1" are displayed. "Camera 1" is the image mark that was captured Camera corresponding to surface defects.

[0058] As described in the background technology, during the production process of metal sheets, various types of surface defects will appear on the metal surface of the metal sheets, and the surface defects have an adverse effect on the quality and performance of industrial products. Therefore, it is necessary to count the surface defects so as to inspect and control the surface defects. In the prior art, surface defects are usually counted by constructing a defect distribution map. However, the defect distribution map in the prior art only reflects the location and defect type of the surface defects, which is relatively rigid and cannot be flexibly applied, and has great limitations. For example, when producing metal sheets, a failure in the production equipment may cause surface defects on the metal surface of the metal sheet. The surface defects thus generated can only be judged manually, which is time-consuming and labor-intensive, with high labor costs. In addition, due to the complexity of the types of surface defects, manual identification may result in inaccurate identification.

[0059] In view of the technical problems described above, through the technical solutions of the embodiments of the present application, the surface defect statistics system uses the statistics module to perform real-time statistics on the surface defects on the metal surface of the metal sheet, so that in the process of producing the metal sheet, the statistical information of the metal surface can be obtained, and the real-time nature of the statistical data is ensured. In addition, in the process of statistical surface defects, the technical solution counts periodic defects. Since periodic defects are caused by the failure of production equipment during the production of metal sheets, the technical solution selects periodic defects from the surface defects to determine whether the production equipment is faulty, repair the production equipment in time, and reduce the surface defects of the metal sheet. Compared with the prior art, there is no need for manual judgment, which reduces manpower and material resources, and improves the accuracy of distinguishing surface defects caused by production equipment failure. In addition, the defect distribution map with periodic defects added is more flexible and has fewer limitations. In addition, the technical problem of low accuracy in judging whether the surface defects are caused by production equipment failure in the prior art is solved.

[0060] Optionally, the defect information is statistically analyzed to determine the operation of periodic defects, including: obtaining the abscissa and ordinate of a second surface defect, wherein the second surface defect is used to indicate a newly added surface defect during real-time detection; obtaining the abscissa and ordinate of a third surface defect that is the same as the abscissa or ordinate of the second surface defect and the same defect type; when the first distance between the second surface defect and the third surface defect is within a predetermined distance interval, obtaining the abscissa and ordinate of a fourth surface defect that is the same as the abscissa or ordinate of the third surface defect and the same defect type; and when the second distance between the third surface defect and the fourth surface defect is within a predetermined distance interval, treating the second surface defect, the third surface defect and the fourth surface defect as periodic defects.

[0061] Specifically, the statistical module also counts periodic defects in the process of counting defect information. The periodic defects are multiple surface defects of the same type with a number greater than or equal to three, and the distance between adjacent surface defects is within a predetermined distance interval. The difference in defect length, defect width and defect area between each first surface defect corresponding to the periodic defect is less than a preset threshold. The periodic defects can be multiple surface defects arranged horizontally or multiple surface defects arranged vertically.

[0062] In this embodiment, a periodic defect is a plurality of surface defects generated in the longitudinal direction with the same horizontal coordinate and different vertical coordinates as an example for explanation. Furthermore, the periodic defect is a surface defect of the same type with a number greater than or equal to three, and the difference in the vertical coordinates between adjacent surface defects is within a predetermined distance interval range, and the horizontal coordinates are the same.

[0063] More specifically, the statistical module obtains a new metal surface image of the metal sheet taken in real time at the current moment, and detects a newly added surface defect (i.e., a second surface defect), such as surface defect 1, compared with the metal surface image taken at the previous moment.

[0064] Further, the statistical module determines the abscissa, ordinate, defect type, defect length, defect width and defect area of ​​the newly added surface defect, and obtains other surface defects with the same abscissa as the newly added surface defect.

[0065] Further, the statistical module screens the other surface defects to determine the surface defect of the same defect type as the newly added surface defect (ie, the third surface defect). The statistical module then obtains the defect length, defect width, and defect area of ​​the surface defect (ie, the third surface defect).

[0066] Further, the statistical module calculates the difference between the defect length, defect width and defect area between the newly added surface defect (ie, the second surface defect) and the plurality of third surface defects.

[0067] When the difference between the defect length, defect width and defect area between the newly added surface defect (i.e., the second surface defect) and the plurality of third surface defects is less than a preset threshold, the statistical module obtains the abscissa and ordinate of the surface defect (i.e., the third surface defect), wherein the number of the third surface defects may be multiple.

[0068] Further, the statistical module calculates the distance between the newly added surface defect (ie, the second surface defect) and the plurality of third surface defects, that is, the difference between the ordinate of the newly added surface defect (ie, the second surface defect) and the ordinate of the plurality of third surface defects (ie, the first distance).

[0069] The statistical module then determines whether the difference (i.e., the first distance) is within a predetermined distance interval range (e.g., the predetermined distance interval range is 0.95-1 m). For example, when the first distance between the ordinate of surface defect 1 (i.e., the second surface defect) and the ordinate of a third surface defect (e.g., surface defect 2) is within the predetermined distance interval range, other surface defects with the same abscissa as the surface defect 2 (i.e., the third surface defect) are obtained, and a surface defect with the same defect type as the surface defect 2 (i.e., the third surface defect) (i.e., the fourth surface defect) is obtained from the other surface defects.

[0070] Preferably, when the first distance between surface defect 2 (i.e., the third surface defect) and the second surface defect is within a predetermined distance interval, the statistical module obtains other third surface defects except surface defect 2 as fourth surface defects, wherein the horizontal coordinate of the fourth surface defect is the same as the horizontal coordinate of the third surface defect, and the defect type of the fourth surface defect is also the same as the defect type of the third surface defect.

[0071] Further, the statistical module obtains the ordinate of the fourth surface defect, thereby calculating the distance between the plurality of fourth surface defects and the third surface defect, that is, the difference between the ordinate of the fourth surface defect and the ordinate of the third surface defect (that is, the second distance).

[0072] The statistical module then determines whether the difference (i.e., the second distance) is within a predetermined distance interval (i.e., 0.95 to 1 m). For example, when the second distance between the ordinate of surface defect 2 (i.e., the third surface defect) and the ordinate of a fourth surface defect (e.g., surface defect 3) is within a predetermined distance interval, the statistical module determines that surface defect 1 (i.e., the second surface defect), surface defect 2 (i.e., the third surface defect), and surface defect 3 (i.e., the fourth surface defect) are periodic defects.

[0073] The difference between surface defect 1 (i.e., the second surface defect) and surface defect 2 (i.e., the third surface defect) and the difference between surface defect 2 (i.e., the third surface defect) and surface defect 3 (i.e., the fourth surface defect) are both within a predetermined distance interval, wherein the predetermined distance interval is determined based on empirical values.

[0074] Furthermore, when the statistical module obtains a new metal surface image of the metal sheet actually shot in real time at the next moment, a newly added surface defect (ie, a second surface defect), such as surface defect 4, is detected therefrom.

[0075] Afterwards, the statistical module determines, according to the above-mentioned method for determining the third surface defect, that surface defect 4 has the same horizontal coordinate and defect type as surface defect 3, and the difference between the vertical coordinates is within a predetermined distance interval. In this case, surface defect 4 is directly regarded as a periodic defect in a group with surface defect 1 (i.e., the second surface defect), surface defect 2 (i.e., the third surface defect) and surface defect 3 (i.e., the fourth surface defect).

[0076] The defect distribution diagram shows periodic defects, see Figure 6 As shown, the periodic defects are displayed as straight line marks on the defect distribution diagram. The starting and ending points of the straight line mark are the coordinates of the first surface defect and the last surface defect of the periodic defect, respectively. For example, the straight line mark can be set to a position offset by 4 pixels to the left of the graphic mark of the periodic defect, and the width of the straight line mark is 2 pixels.

[0077] If there are multiple periodic defects in the defect distribution diagram, multiple straight line marks of the periodic defects are displayed on the defect distribution diagram.

[0078] When the user moves the mouse onto the straight line mark of a periodic defect, the straight line mark of the current periodic defect and the associated graphic mark of the periodic defect are highlighted. Specifically, the width of the straight line mark increases to 4 pixels, and the size of the associated defect mark increases by 2 pixels in length and width with the center as the origin.

[0079] When the user moves the mouse onto the straight line mark of the periodic defect, the defect distribution map will display a floating box corresponding to the periodic defect, wherein the floating box displays the defect type, coordinates and defect quantity of the periodic defect corresponding to the periodic defect.

[0080] For example, refer to Figure 6 As shown, the user moves the mouse onto the straight line mark of the periodic defect, thereby displaying the defect type "black dot" corresponding to the periodic defect, the coordinates "x:145mm, y:13.4m~24.3m", and the number of defects "6". The coordinate x is the horizontal coordinate of the periodic defect, and y is the vertical coordinate of the periodic defect. The "y:13.4m~24.3m" represents the vertical coordinate of the first surface defect to the last surface defect in the periodic defect.

[0081] Since periodic defects are caused by faults in production equipment during the production of metal sheets, this technical solution screens periodic defects from surface defects to determine whether the production equipment is faulty, repair the production equipment in a timely manner, and reduce surface defects of metal sheets.

[0082] In addition, it should be noted that this embodiment takes the periodic defects generated in the longitudinal direction with the same horizontal coordinate and different vertical coordinates as an example, but it is not limited to periodic defects. It can also be multiple surface defects generated in the transverse direction with the same horizontal coordinate and different vertical coordinates, which will not be repeated here.

[0083] Optionally, the method also includes: obtaining the length of the metal sheet, separating the metal sheets according to a preset value, and determining the metal sheet intervals; and performing statistics on defect information based on the metal sheet intervals to construct a defect trend statistical graph, wherein the defect trend statistical graph is used to indicate the number of defects of each defect type in each metal sheet interval.

[0084] Specifically, the statistical module obtains the actual length of the metal sheet, for example, the length of the metal sheet is 2000mm, and the preset value is 100mm, then the length of the metal sheet is divided according to the preset value, so that every 100mm is a metal sheet interval. Thus, the metal sheet intervals obtained after division are: 0-100mm, 100mm-200mm, ..., 1900mm-2000mm.

[0085] The defect information of the surface defects also includes the defect quantity of the surface defects included in each defect type of the metal sheet.

[0086] Thus, the statistical module counts the number of defects in each metal sheet interval, generates a corresponding defect trend statistical graph, and sends the defect trend statistical graph to the display module for display.

[0087] In addition, users can also select the required defect type and count the corresponding defect quantity. Figure 7 Some defect trend statistics are shown, see Figure 7 As shown in the figure, the user counted the number of defects of four defect types: "black spots", "peeling", "bumps" and "scratches".

[0088] Therefore, the technical solution generates a defect trend statistical graph by counting the number of defects, so that the change in the number of defects in the metal sheet interval can be analyzed intuitively, and the number of defects corresponding to various defect types can be compared, so as to easily determine the metal sheet interval with the largest number of defects and the defect type. Therefore, the metal sheet can be processed in a targeted manner to reduce defects.

[0089] Optionally, the method also includes: obtaining the area, length and width of each first surface defect, and determining the defect level of each first surface defect based on the area, length and width; separately counting the number of defects of the first surface defects of each defect type on the upper surface and lower surface of the metal plate, as well as the total number of defects of the first surface defects of the metal plate; and generating a defect statistics table based on the defect level, defect number and total defect number.

[0090] Specifically, after detecting the surface defects of the metal sheet, the statistical module determines the area, length and width of the surface defects (i.e., the first surface defects), and then determines the defect level of each surface defect (i.e., the first surface defects) according to the area, length and width of the surface defects. The defect levels include minor, general and severe.

[0091] Table 1 shows some defect statistics. Referring to Table 1, the area corresponding to "minor" is 0-5mm 2 , length (or width) is 0 to 5 mm; the area corresponding to "generally" is 5 to 10 mm 2 , length (or width) is 5 to 10 mm; "severe" corresponds to an area > 10 mm 2 , length (or width) is >10mm.

[0092] Table 1

[0093]

[0094] In Table 1, a1 to a6 represent the number of defects corresponding to different defect levels of each defect type in the upper surface of the metal plate, b1 to b6 represent the number of defects corresponding to different defect levels of each defect type in the lower surface of the metal plate, c1 to c6 represent the number of defects corresponding to different defect levels of each defect type in the metal surface (i.e., upper surface + lower surface) of the metal plate, d1 represents the total number of defects on the upper surface of the metal plate, d2 represents the total number of defects on the lower surface of the metal plate, and d3 represents the total number of defects on the metal plate.

[0095] And the statistical module determines the priority of the defect level of the surface defects as follows: severe>general>minor, so when one of the area or length (or width) meets the defect level, priority is given to the defect level.

[0096] For example, when the area of ​​a surface defect is between 0 and 5 mm 2 If the length (or width) is between 0 and 5 mm, the statistical module determines that the defect level of the surface defect is minor.

[0097] For example, when the area of ​​a surface defect is between 0 and 5 mm 2 If the length (or width) is between 5 and 10 mm, the statistical module determines that the defect level of the surface defect is general.

[0098] Furthermore, when the statistical module counts the surface defects of the metal sheet, it will count them separately according to the different surfaces of the metal surface. For example, the metal surface includes an upper surface and a lower surface. Thus, the statistical module counts the number of defects of each defect type in the upper surface and the lower surface (i.e., the first surface defects), and counts the total number of defects on the upper surface or the lower surface, as well as the total number of defects on the metal surface of the metal sheet (i.e., the upper surface + the lower surface). Thus, the statistical module generates a defect statistical table (i.e., Table 1) based on data such as defect level, defect number, and total defect number, and sends the defect statistical table to the display module for display.

[0099] In addition, according to actual needs, the number of surface defects of only a single surface may be counted, where the single surface refers to the upper surface or the lower surface.

[0100] Therefore, the technical solution uses a defect statistical table in a tabular form to reasonably arrange the statistical data, which can clearly and concisely reflect the distribution of the defect data. The defect statistical table is also convenient for checking, comparing and analyzing the statistical data, making the statistical data clearer.

[0101] Optionally, the method further includes: sending alarm information when the defect information meets the alarm rule during the statistical process.

[0102] Specifically, when the statistical module is in the process of counting surface defects, if the statistical data meets the alarm rules, the alarm module will send an alarm message to the alarm module, so that the alarm module will alarm according to the alarm message. The alarm rules include: (1) the surface defect statistical system is running abnormally; (2) the statistical module or the display module is running abnormally; (3) the camera is abnormal, for example, the camera does not take real-time photos to generate metal surface images at the specified time; (4) the statistical module detects periodic defects; (5) the total number of defects of the metal sheet exceeds the first preset value; (6) during the production process of the metal sheet, if the number of interval defects exceeds the second preset value, the statistical module will send an alarm message to the alarm module. The number of interval defects refers to the number of defects in different areas of the metal sheet. For example, the statistical module starts to count the number of defects of the metal sheet from 0mm of the metal sheet until the cumulative number of defects detected at 50mm (i.e., the number of interval defects) exceeds the second preset value, then sends an alarm message to the alarm module, and then the statistical module starts to count the number of defects of the metal sheet from 50mm of the metal sheet until the cumulative number of defects detected at 70mm exceeds the second preset value, then sends an alarm message to the alarm module. Thus, the metal sheet detection is completed in the above manner. (7) The statistics module sends the total number of defects to the alarm module according to the preset time interval.

[0103] Therefore, the technical solution alarms the user through the alarm module, thereby reminding the user of system abnormalities or abnormal number of defects, so that the abnormal problems can be solved in time.

[0104] Optionally, the operation of performing real-time detection on the metal surface of the metal sheet includes: detecting the metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of the first surface defect of the metal surface; and analyzing the first surface defect of the metal surface through a large language model to generate a corresponding detection report.

[0105] Specifically, the statistical module is provided with a defect detection model. After acquiring the metal surface image, the statistical module inputs the metal surface image into the defect detection model, detects the metal surface image through the defect detection model, and outputs the corresponding defect type and the coordinates of the defect.

[0106] Furthermore, after determining the defect type, coordinates, size and periodicity of the surface defect (i.e., the first surface defect) on the metal plate, the statistical module inputs the defect type, coordinates, size and periodicity of the surface defect into the large language model, thereby outputting a test report corresponding to the metal plate through the large language model. The test report includes at least: (1) a test overview; (2) defect test results, such as the main defects of the metal plate; (3) recommended treatment measures for the defects found; and (4) a summary.

[0107] Further, the statistical module determines the cause of the surface defect (i.e., the first surface defect) according to the defect type of the surface defect (i.e., the first surface defect). The cause is used to indicate the cause of the surface defect (i.e., the first surface defect) in the process of producing the metal sheet. And the cause of each type of surface defect is preset by the staff.

[0108] For example, the staff pre-set the cause of the defect type "black spot" as "residues in the emulsion or unclean rollers, trenches, oil pipes, oil tanks, etc."

[0109] Therefore, the statistical module determines that the cause of the surface defect with the defect type of "black spot" is "residue in the emulsion or uncleanness of the roller, ditch, oil pipe, oil tank, etc." according to the pre-set cause of the defect type "black spot".

[0110] Furthermore, when the user places the mouse on a surface defect in the defect statistics table, the defect statistics table will display a floating box (not shown in the figure) corresponding to the surface defect, wherein the floating box displays the cause corresponding to the surface defect.

[0111] Therefore, the present technical solution detects surface defects on the metal surface through a defect detection model, and determines the detection report of the surface defects through a large language model. Therefore, the present technical solution utilizes a deep learning method to accurately and easily identify and judge defects under complex conditions, thereby improving the accuracy of metal surface defect detection.

[0112] Optionally, the operation of detecting a metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of a first surface defect on the metal surface includes: inputting the metal surface image into an encoding network of the defect detection model, and generating a first feature tensor corresponding to the metal surface image through an encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; generating a second feature tensor based on the first feature tensor using a decoding network of the defect detection model corresponding to the encoding network; and classifying the second feature tensor using a softmax classifier of the defect detection model to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at a corresponding position on the metal surface.

[0113] Specifically, the staff uses a camera to shoot the metal surface to obtain the metal surface image, and then uses the defect detection model to detect the metal surface image to determine whether there are defects on the metal surface and determine the type of defects.

[0114] refer to Fig. 8A As shown, the defect detection model includes an encoding network, a decoding network, a pooling module and a softmax classifier, the encoding network includes encoding modules 1 to 2, and the decoding network includes multiple decoding modules 1 to 2.

[0115] Thus, the statistical module inputs the metal surface image into the encoding network, and then the encoding module 1 in the encoding network receives the metal surface image and performs an encoding operation on the metal surface image, thereby generating a corresponding feature map 101. The statistical module then inputs the feature map 101 into the encoding module 2, and the encoding module 2 performs an encoding operation on the received feature map 101, thereby outputting a corresponding feature map 102 (i.e., a first feature tensor).

[0116] Furthermore, the statistical module position-encodes the feature map 102 (ie, the first feature tensor) and performs a pooling operation through a pooling unit, thereby outputting the pooled feature map 102 .

[0117] Furthermore, the statistical module inputs the pooled feature map 102 output by the encoding module 2 to the pooling module. The pooling module processes the received pooled feature map 102 and outputs a corresponding feature map.

[0118] Furthermore, the statistical module inputs the feature map output by the pooling module into the decoding network, so that the deconvolution unit of the decoding module 1 in the decoding network extracts features from the feature map, thereby outputting a feature map 301. The statistical module then inputs the feature map 301 into the unified attention fusion module 1, and the unified attention fusion module 1 extracts features from the feature map 301, thereby outputting a feature map 302.

[0119] Furthermore, the statistical module inputs the feature map 302 to the decoding module 2, and the decoding module 2 uses a deconvolution unit to extract features from the feature map 302, thereby outputting a feature map 303.

[0120] The statistical module then inputs the feature map 303 to the deconvolution module in the decoding network, and the deconvolution module performs an upsampling operation on the feature map 303 to generate a feature map 304, so that the encoding network outputs the feature map 304 as a second feature tensor.

[0121] Further, the statistical module inputs the feature map 304 (i.e., the second feature tensor) into the softmax classifier. The softmax classifier performs classification processing on the feature map 304 (i.e., the second feature tensor) to generate a prediction tensor corresponding to the metal surface image. The prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position of the metal surface.

[0122] For example, this embodiment includes 12 types of surface defects and 1 type of surface background, for a total of 13 surface types. Thus, the prediction tensor has a total of 13 prediction matrices. Each prediction matrix corresponds to one surface type. And the elements of each prediction matrix correspond to the pixel points at the corresponding position of the metal surface image, so that each element in each prediction matrix represents the probability value of the corresponding pixel point being the corresponding surface type. The prediction tensor order is a tensor greater than or equal to 2.

[0123] For example, prediction matrix 1 corresponds to a surface defect of the dirt type. Thus, in the prediction matrix 1, each element corresponds to a probability value that a pixel point of the metal surface image is a surface defect of the dirt type.

[0124] For example, the pixels of the metal surface image are 4096*1000*3, and the defect detection model outputs a defect map of 4096*1000*13.

[0125] In another embodiment, reference Figure 8B As shown, the defect detection model includes an encoding network, a decoding network, a pooling module and a softmax classifier, the encoding network includes encoding modules 1 to n, and the decoding network includes multiple decoding modules 1 to n.

[0126] Thus, the statistical module inputs the metal surface image into the encoding network, and then the encoding module 1 in the encoding network receives the metal surface image and performs encoding operation on the metal surface image, thereby generating a corresponding feature map 101. The statistical module then inputs the feature map 101 into the encoding module 2, and the encoding module 2 performs encoding operation on the received feature map 101, thereby outputting a corresponding feature map 102, and so on, until the encoding module n-1 outputs the corresponding feature map ( Fig. 8A (not shown), and the feature map is input to the encoding module n, and then the encoding module n outputs the corresponding feature map 10n (ie, the first feature tensor) according to the feature map.

[0127] Furthermore, the statistical module performs position encoding on the feature map 10n (ie, the first feature tensor), and performs a pooling operation through a pooling unit, thereby outputting a pooled feature map 10n.

[0128] Furthermore, the statistical module inputs the pooled feature map 10n output by the encoding module n to the pooling module. The pooling module processes the received pooled feature map 10n and outputs a corresponding feature map.

[0129] Furthermore, the statistical module inputs the feature map output by the pooling module into the decoding network, so that the decoding network uses the deconvolution unit of the decoding module 1 to extract features from the feature map, thereby outputting a feature map 301. The statistical module then inputs the feature map 301 into the unified attention fusion module 1, and the unified attention fusion module 1 extracts features from the feature map 301, thereby outputting a feature map 302.

[0130] Furthermore, the statistical module inputs the feature map 302 to the decoding module 2, and the decoding module 2 uses a deconvolution unit to extract features from the feature map 302, thereby outputting a feature map 303.

[0131] And so on, until the decoding module n outputs the corresponding feature map 30m.

[0132] in Figure 8B The process of unified attention fusion module n-2, decoding module n-1, and unified attention fusion module n-1 outputting feature maps is the same as above ( Figure 8B The feature graph output by the above module is not shown in FIG, and will not be described here.

[0133] The statistical module then inputs the feature map 30m to the deconvolution module in the decoding network. The deconvolution module performs an upsampling operation on the feature map 30m to generate a feature map 401, so that the encoding network outputs the feature map 401 as a second feature tensor.

[0134] Further, the statistical module inputs the feature map 401 (i.e., the second feature tensor) into the softmax classifier. The softmax classifier performs classification processing on the feature map 401 (i.e., the second feature tensor) to generate a prediction tensor corresponding to the metal surface image. The prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position of the metal surface.

[0135] Thus, the defects on the metal surface can be identified and detected through the defect detection model constructed by the encoding network and the decoding network, so that the defects under complex conditions can be accurately and easily identified and judged by the deep learning method. Compared with the prior art, this technical solution does not require manual parameter adjustment, and uses the deep learning method to improve the accuracy of the parameters, thereby improving the accuracy of metal surface defect detection.

[0136] Optionally, the encoding operation of at least one encoding module in the encoding network includes: receiving an input tensor to be encoded; using at least one convolutional layer to convolve the tensor to be encoded to generate a first tensor feature to be encoded corresponding to the tensor to be encoded; using a feature extraction module based on a global attention mechanism and a local attention mechanism to generate a second tensor feature to be encoded corresponding to the first tensor feature to be encoded; and merging the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coded tensor as the encoding result.

[0137] Specifically, refer to Fig. 8A As shown, the encoding module 1 receives the input tensor to be encoded corresponding to the metal surface image, and uses at least one convolutional layer (such as Fig. 8A The convolution unit in the encoding module 1 performs a convolution operation on the tensor to be encoded, thereby generating a first tensor feature to be encoded corresponding to the tensor to be encoded.

[0138] Furthermore, the statistics module inputs the first tensor feature to be encoded into the feature extraction module, wherein the feature extraction module includes a feature extraction unit based on global attention and a feature extraction unit based on local attention mechanism.

[0139] Therefore, the feature extraction module extracts features of the first tensor feature to be encoded through a feature extraction unit based on global attention and a feature extraction unit based on a local attention mechanism, thereby generating corresponding second tensor features to be encoded.

[0140] Furthermore, the statistical module adds the elements of the first tensor feature to be encoded and the elements of the second tensor feature to be encoded at corresponding positions, thereby fusing the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coded tensor as the encoding result.

[0141] In addition, other encoding modules process the received feature map (i.e., the tensor to be encoded) according to the method of processing the tensor to be encoded by the above-mentioned encoding module 1 to generate the encoding tensor, so as to generate a new feature map (i.e., the encoding tensor), which will not be repeated here.

[0142] The tensor to be encoded received by encoding module 1 is a metal surface image, and the tensors to be encoded received by other encoding modules are feature maps output by the previous encoding module.

[0143] Therefore, the traditional image processing method in the prior art only processes the image through the convolution layer and the pooling layer. On this basis, the present technical solution adds a method for extracting semantic features, thereby extracting the semantic features of the image through the feature extraction module, thereby improving the accuracy of metal surface defect detection.

[0144] Optionally, the operation of generating a second tensor feature to be encoded corresponding to the first tensor feature to be encoded using a feature extraction module based on a global attention mechanism and a local attention mechanism includes: generating a binary image corresponding to the first tensor feature to be encoded, wherein a connected domain of the binary image corresponds to a metal surface defect area of ​​the metal surface; generating a third tensor feature to be encoded based on the binary image and the first tensor feature to be encoded, wherein the tensor element values ​​corresponding to the connected domain in the third tensor feature to be encoded are the same as the corresponding tensor element values ​​in the first tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0; performing a block embedding operation on the first tensor feature to be encoded to obtain a first sequence, and performing feature extraction on the first sequence using a feature extraction unit based on a global attention mechanism to determine a global semantic feature corresponding to the first tensor feature to be encoded; performing a block embedding operation on the third tensor feature to be encoded to obtain a second sequence, and performing feature extraction on the second sequence using a feature extraction unit based on a local attention mechanism to determine a local semantic feature corresponding to the third tensor feature to be encoded; and merging the global semantic feature and the local semantic feature to determine the second tensor feature to be encoded.

[0145] Specifically, see Fig. 8A and Figure 8BAs shown, the defect detection model also includes an image segmentation model, which segments and binarizes the metal surface image to generate a binary image, wherein the binary image includes a connected domain corresponding to the defect area in the metal surface image. In addition, the generated binary image is input to the feature extraction module of each encoding module. After the image segmentation model generates the binary image, the binary image can also be patch-embedding operated to generate a corresponding sequence, and then the corresponding sequence is input to the feature extraction module of each encoding module.

[0146] Fig. 9 A schematic diagram of the feature extraction module of each encoding module is further shown, referring to Fig. 9 As shown, the feature extraction module scales the binarized image through the image scaling module so that the generated binary image is consistent with the size of the first tensor feature to be encoded.

[0147] Further, among them Fig. 10A Image data corresponding to pixel points in a partial image area of ​​a binary image is shown. Fig. 10B The coordinates corresponding to the image data in the partial image area of ​​the binary image are shown. Fig. 10A and Fig. 10B As shown in the figure, the pixel value of the defective area in the binary image is 1, and the pixel value of the background area is 0. Therefore, the statistical module determines the area with a pixel value of 1 as a connected domain, where the coordinates of the area with a pixel value of 1 are x(1,1), x(1,2), x(2,1) and x(2,2), respectively. The defective area is determined based on the connected domain, that is, the area with a pixel value of 1 is a connected domain and also a defective area.

[0148] Further, Fig. 10C shows part of the tensor element data of the first tensor feature to be encoded, Fig. 10D Part of the tensor element data of the third tensor feature to be encoded is shown. The statistical module maps the first tensor feature to be encoded to the binary image, so that each tensor element in the first tensor feature to be encoded corresponds to the pixel point at the same position of the binary image. Thus, the statistical module uses the tensor element value corresponding to the connected domain in the first tensor feature to be encoded as the tensor element value corresponding to the connected domain in the third tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0, thereby obtaining the third tensor feature to be encoded. The tensor element value corresponding to the connected domain in the third tensor feature to be encoded is the same as the corresponding tensor element value in the first tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0.

[0149] by FIG. 10A to FIG. 10DFor example, the tensor element values ​​at the positions x(1,1), x(1,2), x(2,1) and x(2,2) in the third tensor feature to be encoded are determined according to the tensor element values ​​at the positions x(1,1), x(1,2), x(2,1) and x(2,2) in the third tensor feature to be encoded. The tensor element values ​​at the remaining positions in the third tensor feature to be encoded are determined according to the pixel values ​​at the corresponding positions in the binary image.

[0150] For example, at the corresponding positions, the tensor element values ​​of the first tensor feature to be encoded are multiplied by the pixel values ​​of the binary image to obtain the tensor element values ​​of each position of the third tensor feature to be encoded. Thus, at the positions x(1,1), x(1,2), x(2,1) and x(2,2), the tensor element values ​​of the third tensor feature to be encoded are the same as the tensor element values ​​of the first tensor feature to be encoded, which are x 1,1 、x 1,2 、x 2,1 and x 2,2 At other positions, the tensor element value of the third tensor feature to be encoded is 0.

[0151] Further, the statistical module performs a block embedding operation on the first tensor feature to be encoded to obtain a first sequence, and the statistical module performs a block embedding operation on the third tensor feature to be encoded to obtain a second sequence.

[0152] Further, the feature extraction module includes a feature extraction unit based on a global attention mechanism and a feature extraction unit based on a local attention mechanism. Thus, the statistical module inputs the first sequence into the feature extraction unit based on the global attention mechanism, and inputs the second sequence into the feature extraction unit based on the local attention mechanism.

[0153] Therefore, the feature extraction unit based on the global attention mechanism calculates the weight value corresponding to each element in the first sequence, and then calculates the global semantic feature according to each element and the corresponding weight value.

[0154] The feature extraction unit based on the local attention mechanism calculates the weight value corresponding to each element in the second sequence, and then calculates the local semantic feature according to each element and the corresponding weight value, wherein the number of elements in the first sequence is the same as the number of elements in the second sequence.

[0155] Thus, the feature extraction unit based on the global attention mechanism outputs the global semantic feature corresponding to the first tensor feature to be encoded, and the feature extraction unit based on the local attention mechanism outputs the local semantic feature corresponding to the third tensor feature to be encoded. The global attention mechanism and the local attention mechanism can be a multi-head attention mechanism. In this embodiment, the number of global attention heads is the same as that of local attention heads.

[0156] Furthermore, the statistical module combines the global semantic features and the local semantic features to determine the second tensor feature to be encoded.

[0157] In addition, the feature extraction unit outputs the formula for the element value of each element of the second tensor feature to be encoded as follows:

[0158] [γ1,γ2,...,γ t ] T =[x1, x2, ..., x t ] T +[y1,y2,...,y t ] T ;

[0159]

[0160]

[0161] as well as

[0162]

[0163] [x1, x2, ..., x t ] T Represents the global semantic features, [y1, y2, ..., y t ] T Represents local semantic features, represents the first sequence corresponding to the first tensor feature to be encoded, represents the second sequence corresponding to the third tensor feature to be encoded, V parameter of the global attention mechanism, kg i represents the K parameter of the global attention mechanism, Vl i V parameter represents the local attention mechanism, kl i represents the K parameter of the local attention mechanism. d is or The vector dimension of .

[0164] Therefore, the technical solution extracts the global semantic features of the first tensor feature to be encoded through the global attention mechanism, and extracts the local semantic features of the third tensor feature to be encoded through the local attention mechanism, and then fuses the global semantic features and the local semantic features. Therefore, the technical solution not only considers the feature level of the entire feature map, but also measures the local edge features of the feature map, making the detection of metal surface defects more accurate and improving the detection accuracy.

[0165] Optionally, the operation of generating a second feature tensor based on the first feature tensor using a decoding network corresponding to the encoding network includes: inputting the first feature tensor into a pooling module, performing a downsampling operation on the first feature tensor layer by layer through the pooling module to generate a downsampled tensor; upsampling the downsampled tensor layer by layer through the pooling module to generate a third feature tensor corresponding to the first feature tensor; and inputting the third feature tensor into a decoding network, and generating a second feature tensor corresponding to the third feature tensor through a decoding operation of at least one decoding module in the decoding network.

[0166] Specifically, refer to Fig.11 As shown, the pooling module includes multiple adjacent downsampling layers and multiple adjacent upsampling layers.

[0167] After the encoding network outputs the first feature tensor, the statistical module inputs the first feature tensor to the upsampling layer of the pooling module, so that multiple upsampling layers downsample the received first feature tensor layer by layer to generate a downsampled tensor.

[0168] Furthermore, the statistical module inputs the downsampled tensor into the upsampled layer of the pooling module, so that the multiple upsampled layers upsample the received downsampled tensors layer by layer, thereby outputting a third feature tensor.

[0169] Furthermore, the statistical module inputs the third feature tensor into the decoding network. The decoding network includes multiple decoding modules. Thus, the multiple decoding modules perform a decoding operation on the third feature tensor to obtain a corresponding feature map, and then the deconvolution module in the decoding network performs an upsampling operation on the feature map to generate a corresponding decoding feature map, so that the encoding network outputs the decoding feature map as the second feature tensor.

[0170] Therefore, the technical solution decodes the feature map through a simplified pooling module and a decoding module, which reduces the decoding process and time and improves the decoding speed.

[0171] Optionally, the downsampled tensor is upsampled layer by layer through a pooling module to generate an operation of a third feature tensor corresponding to the first feature tensor, including: receiving an input tensor to be upsampled; performing an average pooling operation on the tensor to be upsampled through a first average pooling unit of the pooling module to generate a first pooling tensor; performing average pooling operations on the first pooling tensor through multiple second average pooling units of different sizes to generate corresponding multiple second pooling tensors; and merging the multiple second pooling tensors after convolution and upsampling operations to generate an upsampled tensor.

[0172] Specifically, refer to Fig.12 As shown, each upsampling layer includes a first average pooling unit, multiple second average pooling units, multiple first convolution units, a second convolution unit and multiple upsampling units.

[0173] For example, the statistical module inputs the tensor to be upsampled into the average pooling unit 1 (ie, the first average pooling unit) of the upsampling layer, and the average pooling unit 1 performs an average pooling operation on the tensor to be upsampled to generate a corresponding feature map 201 (first pooling tensor).

[0174] Further, the statistical module inputs the feature map 201 into multiple average pooling units of different sizes (i.e., second average pooling units). For example, there are 3 average pooling units (i.e., average pooling units 2 to 4), with sizes of 1×1, 2×2, and 4×4, respectively. Thus, the 3 average pooling units perform average pooling operations on the feature map 201 (first pooling tensor), thereby generating feature maps 202 to 204 (i.e., multiple second pooling tensors), respectively.

[0175] Furthermore, the statistical module inputs the feature maps 202 to 204 (i.e., multiple second pooling tensors) into the corresponding first convolution units, respectively. The convolution kernel in the first convolution unit is 1×1, and the output channel is smaller than the input channel. And one of the second average pooling units corresponds to a first convolution unit and an upsampling unit, so that when the second average pooling units include 3, the first convolution unit and the upsampling unit also include 3 respectively. Thus, the three first convolution units (i.e., first convolution units 1 to 3) generate feature maps 205 to 207 respectively.

[0176] Furthermore, the statistical module inputs the feature maps 205-207 into corresponding upsampling units respectively, so that the three upsampling units (ie, upsampling units 1-3) output feature maps 208-210. The feature maps 208-210 have the same size.

[0177] Furthermore, the statistical module adds the feature elements at corresponding positions of the feature graphs 208 to 210 to generate a feature graph 211 .

[0178] Furthermore, the statistical module inputs the feature map 211 into the second convolution unit. The convolution kernel in the second convolution unit is 1×1. Thus, the second convolution unit outputs a feature map 212 (ie, an upsampled tensor).

[0179] Furthermore, the statistical module inputs the feature map 212 (i.e., the upsampled tensor) as the tensor to be upsampled into the next upsampled layer, so that the next upsampled layer (i.e., the current upsampled layer) performs an upsample operation on the received feature map 212 according to the method of generating the upsampled tensor by the previous upsampled layer, and generates a corresponding upsampled tensor.

[0180] Therefore, other upsampling layers perform upsampling processing on the upsampled tensor according to the above method to generate corresponding upsampled tensors.

[0181] Therefore, the pooling module in this technical solution reduces the number of channels through a 1×1 convolution kernel, and uses a more concise and efficient addition operation to add the feature elements of multiple feature maps, making the pooling module more efficient and ensuring the real-time performance of the model.

[0182] Optionally, the third feature tensor is input into the decoding network, and the operation of generating a second feature tensor corresponding to the third feature tensor through the decoding operation of at least one decoding module in the decoding network includes: receiving the input first tensor to be decoded through the decoding module of the decoding network; performing a deconvolution operation on the first tensor to be decoded through the decoding module using at least one deconvolution layer to generate a decoding tensor corresponding to the first tensor to be decoded as a decoding result; inputting the decoding tensor as the first tensor to be fused into the unified attention fusion module; inputting the first tensor to be encoded feature output by the corresponding encoding module as the second tensor to be fused into the unified attention fusion module; fusing the first tensor to be fused and the second tensor to be fused through the unified attention fusion module to generate a fused tensor; and inputting the fused tensor as the second tensor to be decoded into a decoding module that is shallower than the unified attention fusion module.

[0183] Specifically, refer to Fig. 8A As shown, the decoding network includes multiple decoding modules. For example, decoding module 1 receives the first tensor to be decoded. Decoding module 1 is the deepest decoding module in the decoding network, and the first tensor to be decoded received by the deepest decoding module is the third feature tensor output by the pooling module.

[0184] Furthermore, the decoding module 1 includes multiple deconvolution layers (such as Fig. 8A The decoding module 1 inputs the first tensor to be decoded into the deconvolution unit, and the deconvolution unit performs a deconvolution operation on the first tensor to be decoded. Then, the decoding module 1 outputs a feature map 301 (i.e., a decoding tensor) corresponding to the first tensor to be decoded.

[0185] Furthermore, the statistical module inputs the feature map 301 (ie, the decoded tensor) as the first tensor to be fused into the unified attention fusion module 1.

[0186] The encoding module 1 corresponds to the unified attention fusion module 1, so that the statistical module inputs the first tensor to be encoded output by the encoding module 1 into the unified attention fusion module 1 as the second tensor to be fused.

[0187] Furthermore, the unified attention fusion module 1 calculates the weights corresponding to the first tensor to be fused and the second tensor to be fused respectively through the attention module, and merges the first tensor to be fused and the second tensor to be fused and the corresponding weights after calculating, thereby generating a feature map 302 (i.e., a fused tensor).

[0188] Afterwards, the statistical module sends the feature map 302 (ie, the fusion tensor) to a decoding module at a shallower level than the unified attention fusion module. The decoding module at a shallower level than the unified attention fusion module 1 is the decoding module 2.

[0189] Thus, the statistical module sends the feature map 302 (i.e., fused tensor) output by the unified attention module 1 as the second tensor to be decoded to the decoding module 2. In this case, relative to the decoding module 2, the feature map 302 is the first tensor to be decoded, and similar operations described above are continued. The decoding module 2 includes multiple deconvolution units. The decoding module 2 receives the second tensor to be decoded and inputs it into the deconvolution unit. The deconvolution unit performs a deconvolution operation on the second tensor to be decoded, and then the decoding module 2 outputs a feature map 303 corresponding to the second tensor to be decoded.

[0190] Therefore, this technical solution strengthens the feature representation of the feature map by adding a unified attention fusion module in the decoding module, and screens important features so that only important features can be calculated, reducing the amount of calculation and improving the decoding speed.

[0191] Optionally, the operation of fusing the first tensor to be fused and the second tensor to be fused through a unified attention fusion module to generate a fused tensor includes: performing an upsampling operation on the first tensor to be fused through an upsampling unit of the unified attention fusion module to generate a third tensor to be fused, wherein the third tensor to be fused has the same size as the second tensor to be fused; inputting the second tensor to be fused and the third tensor to be fused into the attention unit respectively, and calculating the first weight corresponding to the second tensor to be fused and the second weight corresponding to the third tensor to be fused through the attention unit respectively; calculating the first attention weighted feature according to the second tensor to be fused and the first weight; calculating the second attention weighted feature according to the third tensor to be fused and the second weight; and fusing the first attention weighted feature and the second attention weighted feature to generate a fused tensor.

[0192] Specifically, refer to Fig.13 As shown, the unified attention fusion module includes an upsampling unit and an attention unit.

[0193] For example, the unified attention fusion module 1 receives the first tensor to be fused F output by the decoding module 1 respectively high, and the second tensor to be fused output by the convolution unit of the encoding module 1, and then the first tensor to be decoded F high Input upsampling unit, upsampling unit through bilinear interpolation operation, the first to be decoded tensor F high Upsample to the same size as the second tensor to be decoded, thereby generating the third tensor to be fused F up The third tensor to be fused has the same size as the second tensor to be fused. up The expression is:

[0194] F up =Upsample(F high ).

[0195] Furthermore, the statistics module converts the third tensor to be fused F up and the second tensor to be fused F low are input to the attention unit respectively, so that the attention unit calculates and the second tensor to be fused F low The corresponding first weight and second tensor to be fused F low The corresponding second weight. The sum of the first weight and the second weight is 1, so the first weight is expressed as r, and the second weight is expressed as 1-r. The method for the attention unit to calculate the first weight and the second weight can be, for example, through a spatial attention mechanism or a channel attention mechanism. The expression for calculating the first weight r is:

[0196] r = Attention(F up ,F low ).

[0197] Furthermore, the statistical module multiplies the tensor elements in the second tensor to be fused by the first weight respectively, so as to obtain the first attention weighted feature.

[0198] Furthermore, the statistical module multiplies the tensor elements of the third tensor to be fused by the second weight respectively, so as to obtain the second attention weighted feature.

[0199] Furthermore, the statistical module adds the tensor elements of the first attention-weighted feature and the tensor elements at the corresponding positions of the second attention-weighted feature to generate a fused tensor F out The formula for calculating the fused tensor is:

[0200] F out =F up ·r+F low ·(1-r).

[0201] Therefore, this technical solution strengthens the feature representation of the feature map through a unified attention fusion module, and screens important features, so that only important features can be calculated, reducing the amount of calculation and improving the decoding speed.

[0202] In addition, reference Figure 1 As shown, according to the second aspect of this embodiment, a storage medium is provided, wherein the storage medium includes a stored program, wherein when the program is run, the processor executes the above method.

[0203] Therefore, according to this embodiment, the surface defect statistics system uses the statistics module to perform real-time statistics on the surface defects on the metal surface of the metal sheet, so that in the process of producing the metal sheet, the statistical information of the metal surface can be obtained, ensuring the real-time nature of the statistical data. In addition, in the process of statistical surface defects, the technical solution counts periodic defects. Since periodic defects are caused by the failure of production equipment during the production of metal sheets, the technical solution determines whether the production equipment is faulty by screening periodic defects from the surface defects, repairs the production equipment in time, and reduces the surface defects of the metal sheet. Compared with the prior art, there is no need for manual judgment, which reduces manpower and material resources, and improves the accuracy of distinguishing surface defects caused by production equipment failure. In addition, the defect distribution map with periodic defects added is more flexible and has fewer limitations. In addition, the technical problem of low accuracy in judging whether the surface defects are caused by production equipment failure in the prior art is solved.

[0204] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described order of actions, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0205] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0206] Example 2

[0207] Fig.14 FIG. 1 shows a surface defect counting device 1400 for a metal surface according to this embodiment, and the device 1400 corresponds to the method according to the first aspect of Embodiment 1. Fig.14 As shown, the device 1400 includes: a first determination module 1410, which is used to perform real-time detection on the metal surface of the metal plate and determine the defect information of the first surface defects on the metal surface, wherein the defect information includes the defect type, defect length, defect width, defect area and coordinates of each first surface defect; and a second determination module 1420, which is used to count the defect information, determine periodic defects, and construct a defect distribution map, wherein the periodic defects are first surface defects of the same defect type that are separated by a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in defect length, defect width and defect area between each first surface defect corresponding to the periodic defect are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0208] Optionally, the second determination module 1420 includes: a first acquisition submodule, used to acquire the horizontal coordinate and the vertical coordinate of the second surface defect, wherein the second surface defect is used to indicate a newly added surface defect during the real-time detection process; a second acquisition submodule, used to acquire the horizontal coordinate and the vertical coordinate of a third surface defect that is the same as the horizontal coordinate or the vertical coordinate of the second surface defect and the same defect type; a third acquisition submodule, used to acquire the horizontal coordinate and the vertical coordinate of a fourth surface defect that is the same as the horizontal coordinate or the vertical coordinate of the third surface defect and the same defect type when the first distance between the second surface defect and the third surface defect is within a predetermined distance interval; and the first determination submodule, used to treat the second surface defect, the third surface defect and the fourth surface defect as periodic defects when the second distance between the third surface defect and the fourth surface defect is equal to within a predetermined distance interval.

[0209] Device 1400 also includes: a first acquisition module, used to obtain the length of the metal plate, separate the metal plates according to a preset value, and determine the metal plate interval; and a first statistical module, used to count the defect information according to the metal plate interval, and construct a defect trend statistical graph, wherein the defect trend statistical graph is used to indicate the number of defects of each defect type in each metal plate interval.

[0210] The device 1400 also includes: a second acquisition module, used to obtain the area, length and width of each first surface defect, and determine the defect level of each first surface defect based on the area, length and width; a second statistical module, used to respectively count the defect number of first surface defects of each defect type on the upper surface and lower surface of the metal plate, as well as the total defect number of first surface defects of the metal plate; and a statistical table generation module, used to generate a defect statistical table based on the defect level, defect number and total defect number.

[0211] The device 1400 also includes: an alarm module, which is used to send alarm information when the defect information meets the alarm rules during the statistical process.

[0212] Optionally, the first determination module 1410 includes: a first determination submodule, used to detect the metal surface image corresponding to the metal surface through a defect detection model, and determine the defect type of the first surface defect of the metal surface; and a second determination submodule, used to analyze the first surface defect of the metal surface through a large language model to generate a corresponding detection report.

[0213] Optionally, the first determination submodule includes: a first generating unit, used to input the metal surface image into the encoding network of the defect detection model, and generate a first feature tensor corresponding to the metal surface image through the encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; a second generating unit, used to generate a second feature tensor based on the first feature tensor using the decoding network of the defect detection model corresponding to the encoding network; and a third generating unit, used to classify the second feature tensor using the softmax classifier of the defect detection model, to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position of the metal surface.

[0214] Therefore, according to this embodiment, the surface defect statistics system uses the statistics module to perform real-time statistics on the surface defects on the metal surface of the metal sheet, so that in the process of producing the metal sheet, the statistical information of the metal surface can be obtained, ensuring the real-time nature of the statistical data. In addition, in the process of statistical surface defects, the technical solution counts periodic defects. Since periodic defects are caused by the failure of production equipment during the production of metal sheets, the technical solution determines whether the production equipment is faulty by screening periodic defects from the surface defects, repairs the production equipment in time, and reduces the surface defects of the metal sheet. Compared with the prior art, there is no need for manual judgment, which reduces manpower and material resources, and improves the accuracy of distinguishing surface defects caused by production equipment failure. In addition, the defect distribution map with periodic defects added is more flexible and has fewer limitations. In addition, the technical problem of low accuracy in judging whether the surface defects are caused by production equipment failure in the prior art is solved.

[0215] Example 3

[0216] Fig.15 FIG. 1 shows a surface defect counting device 1500 for a metal surface according to the first aspect of this embodiment, and the device 1500 corresponds to the method according to the first aspect of embodiment 1. Fig.15 As shown, the device 1500 includes: a processor 1510; and a memory 1520, which is connected to the processor 1510 and is used to provide the processor 1510 with instructions for processing the following processing steps: real-time detection of the metal surface of the metal plate to determine defect information of the first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area and coordinates of each first surface defect; and statistics the defect information to determine periodic defects, and construct a defect distribution map, wherein the periodic defects are first surface defects of the same defect type that are separated by a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in defect length, defect width and defect area between each first surface defect corresponding to the periodic defect are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type.

[0217] Optionally, the defect information is statistically analyzed to determine the operation of periodic defects, including: obtaining the horizontal coordinate and vertical coordinate of the second surface defect, wherein the second surface defect is used to indicate a newly added surface defect during the real-time detection process; obtaining the horizontal coordinate and vertical coordinate of a third surface defect that is the same as the horizontal coordinate or vertical coordinate of the second surface defect and the same defect type; when the first distance between the second surface defect and the third surface defect is within a predetermined distance interval, obtaining the horizontal coordinate and vertical coordinate of a fourth surface defect that is the same as the horizontal coordinate or vertical coordinate of the third surface defect and the same defect type; and when the second distance between the third surface defect and the fourth surface defect is equal to within a predetermined distance interval, treating the second surface defect, the third surface defect and the fourth surface defect as periodic defects.

[0218] Optionally, the memory 1520 is also used to provide instructions for the processor 1510 to process the following processing steps: obtaining the length of the metal sheet, separating the metal sheets according to preset values, and determining the metal sheet intervals; and performing statistics on defect information based on the metal sheet intervals to construct a defect trend statistical graph, wherein the defect trend statistical graph is used to indicate the number of defects of each defect type in each metal sheet interval.

[0219] Optionally, the memory 1520 is also used to provide instructions for the processor 1510 to process the following processing steps: obtaining the area, length and width of each first surface defect, and determining the defect level of each first surface defect based on the area, length and width; separately counting the number of defects of the first surface defects of each defect type on the upper and lower surfaces of the metal sheet, as well as the total number of defects of the first surface defects of the metal sheet; and generating a defect statistics table based on the defect level, defect number and total defect number.

[0220] Optionally, the memory 1520 is further configured to provide the processor 1510 with instructions for processing the following processing steps: sending alarm information when the defect information meets the alarm rule during the statistical process.

[0221] Optionally, the operation of performing real-time detection on the metal surface of the metal sheet includes: detecting the metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of the first surface defect of the metal surface; and analyzing the first surface defect of the metal surface through a large language model to generate a corresponding detection report.

[0222] Optionally, the operation of detecting a metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of a first surface defect on the metal surface includes: inputting the metal surface image into an encoding network of the defect detection model, and generating a first feature tensor corresponding to the metal surface image through an encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; generating a second feature tensor based on the first feature tensor using a decoding network of the defect detection model corresponding to the encoding network; and classifying the second feature tensor using a softmax classifier of the defect detection model to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at a corresponding position on the metal surface.

[0223] Therefore, according to this embodiment, the surface defect statistics system uses the statistics module to perform real-time statistics on the surface defects on the metal surface of the metal sheet, so that in the process of producing the metal sheet, the statistical information of the metal surface can be obtained, ensuring the real-time nature of the statistical data. In addition, in the process of statistical surface defects, the technical solution counts periodic defects. Since periodic defects are caused by the failure of production equipment during the production of metal sheets, the technical solution determines whether the production equipment is faulty by screening periodic defects from the surface defects, repairs the production equipment in time, and reduces the surface defects of the metal sheet. Compared with the prior art, there is no need for manual judgment, which reduces manpower and material resources, and improves the accuracy of distinguishing surface defects caused by production equipment failure. In addition, the defect distribution map with periodic defects added is more flexible and has fewer limitations. In addition, the technical problem of low accuracy in judging whether the surface defects are caused by production equipment failure in the prior art is solved.

[0224] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0225] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0226] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0227] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0228] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0229] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0230] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for counting surface defects of metal surfaces, characterized in that: include: Performing real-time detection on the metal surface of the metal sheet to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect; as well as The defect information is statistically analyzed to determine periodic defects, and a defect distribution map is constructed, wherein the periodic defects are first surface defects of the same defect type, adjacent first surface defects are separated by a predetermined distance, and the number of the first surface defects is greater than or equal to three, and the differences in the defect length, the defect width, and the defect area between the first surface defects corresponding to the periodic defects are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type, and wherein The operation of real-time detection of the metal surface of the metal sheet comprises: detecting the metal surface image corresponding to the metal surface by a defect detection model to determine the defect type of the first surface defect of the metal surface; and analyzing the first surface defect of the metal surface by a large language model to generate a corresponding detection report, and wherein The operation of detecting a metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of a first surface defect on the metal surface includes: inputting the metal surface image into an encoding network of the defect detection model, and generating a first feature tensor corresponding to the metal surface image through an encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; generating a second feature tensor based on the first feature tensor using a decoding network of the defect detection model corresponding to the encoding network; and classifying the second feature tensor using a softmax classifier of the defect detection model to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value that a corresponding position on the metal surface exists in the surface defect corresponding to the prediction matrix, and wherein, The encoding operation of at least one encoding module in the encoding network includes: receiving an input tensor to be encoded; using at least one convolutional layer to convolve the tensor to be encoded to generate a first tensor feature to be encoded corresponding to the tensor to be encoded; using a feature extraction module based on a global attention mechanism and a local attention mechanism to generate a second tensor feature to be encoded corresponding to the first tensor feature to be encoded; and merging the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coding tensor as an encoding result.

2. The method according to claim 1, characterized in that The operation of collecting statistics on the defect information and determining periodic defects includes: Acquire a horizontal coordinate and a vertical coordinate of a second surface defect, wherein the second surface defect is used to indicate a surface defect newly added during the real-time detection process; Obtaining the abscissa and ordinate of a third surface defect having the same abscissa or ordinate as the second surface defect and the same defect type; When the first distance between the second surface defect and the third surface defect is within a predetermined distance interval, obtaining the abscissa and ordinate of a fourth surface defect having the same abscissa or ordinate as the third surface defect and the same defect type as the third surface defect; and In the case where a second distance between the third surface defect and the fourth surface defect is within the predetermined distance interval range, the second surface defect, the third surface defect and the fourth surface defect are regarded as periodic defects.

3. The method according to claim 1, characterized in that Also includes: Obtaining the length of the metal sheet, dividing the metal sheet according to a preset value, and determining the interval between the metal sheets; as well as The defect information is counted according to the metal sheet intervals to construct a defect trend statistical graph, wherein the defect trend statistical graph is used to indicate the defect quantity of each defect type in each metal sheet interval.

4. The method according to claim 1, characterized in that: Also includes: Acquire the area, length and width of each first surface defect, and determine the defect grade of each first surface defect according to the area, length and width; Counting the number of first surface defects of each defect type on the upper surface and the lower surface of the metal plate, respectively, and the total number of first surface defects of the metal plate; as well as A defect statistics table is generated according to the defect level, the defect quantity and the total defect quantity.

5. The method according to claim 1, characterized in that Also includes: When the defect information meets the alarm rules during the statistical process, an alarm message is sent.

6. A storage medium, characterized in that: The storage medium includes a stored program, wherein when the program is run, the processor executes the method according to any one of claims 1 to 5.

7. A device for counting surface defects of metal surfaces, characterized in that: include: A first determination module is used to perform real-time detection on the metal surface of the metal sheet to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area and coordinates of each first surface defect; as well as The second determination module is used to count the defect information, determine periodic defects, and construct a defect distribution map, wherein the periodic defects are first surface defects of the same defect type that are separated by a predetermined distance from adjacent first surface defects and are greater than or equal to three in number, and wherein the differences in the defect length, the defect width, and the defect area between the first surface defects corresponding to the periodic defects are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type, and wherein The first determination module includes: a first determination submodule, which is used to detect the metal surface image corresponding to the metal surface through the defect detection model to determine the defect type of the first surface defect of the metal surface; and a second determination submodule, which is used to analyze the first surface defect of the metal surface through the large language model to generate a corresponding detection report, and wherein The first determination submodule includes: a first generation unit, which is used to input the metal surface image into the encoding network of the defect detection model, and generate a first feature tensor corresponding to the metal surface image through the encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; a second generation unit, which is used to generate a second feature tensor based on the first feature tensor using the decoding network of the defect detection model corresponding to the encoding network; and a third generation unit, which is used to classify the second feature tensor using the softmax classifier of the defect detection model to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position of the metal surface, and wherein The encoding operation of at least one encoding module in the encoding network includes: receiving an input tensor to be encoded; using at least one convolutional layer to convolve the tensor to be encoded to generate a first tensor feature to be encoded corresponding to the tensor to be encoded; using a feature extraction module based on a global attention mechanism and a local attention mechanism to generate a second tensor feature to be encoded corresponding to the first tensor feature to be encoded; and merging the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coding tensor as an encoding result.

8. A device for counting surface defects of metal surfaces, characterized in that: include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Performing real-time detection on the metal surface of the metal sheet to determine defect information of first surface defects on the metal surface, wherein the defect information includes defect type, defect length, defect width, defect area, and coordinates of each first surface defect; as well as The defect information is statistically analyzed to determine periodic defects, and a defect distribution map is constructed, wherein the periodic defects are first surface defects of the same defect type, adjacent first surface defects are separated by a predetermined distance, and the number of the first surface defects is greater than or equal to three, and the differences in the defect length, the defect width, and the defect area between the first surface defects corresponding to the periodic defects are all less than a preset threshold, and the defect distribution map is used to indicate the location of each first surface defect on the metal surface and the corresponding defect type, and wherein The operation of real-time detection of the metal surface of the metal sheet comprises: detecting the metal surface image corresponding to the metal surface by a defect detection model to determine the defect type of the first surface defect of the metal surface; and analyzing the first surface defect of the metal surface by a large language model to generate a corresponding detection report, and wherein The operation of detecting a metal surface image corresponding to the metal surface through a defect detection model to determine the defect type of a first surface defect on the metal surface includes: inputting the metal surface image into an encoding network of the defect detection model, and generating a first feature tensor corresponding to the metal surface image through an encoding operation of at least one encoding module in the encoding network, wherein the metal surface image is obtained by photographing the metal surface; generating a second feature tensor based on the first feature tensor using a decoding network of the defect detection model corresponding to the encoding network; and classifying the second feature tensor using a softmax classifier of the defect detection model to generate a prediction tensor corresponding to the metal surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value that a corresponding position on the metal surface exists in the surface defect corresponding to the prediction matrix, and wherein, The encoding operation of at least one encoding module in the encoding network includes: receiving an input tensor to be encoded; using at least one convolutional layer to convolve the tensor to be encoded to generate a first tensor feature to be encoded corresponding to the tensor to be encoded; using a feature extraction module based on a global attention mechanism and a local attention mechanism to generate a second tensor feature to be encoded corresponding to the first tensor feature to be encoded; and merging the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coding tensor as an encoding result.

Images