Surface defect management system, method, device and storage medium for industrial materials

Abstract

The present application discloses a surface defect management system, method, device and storage medium for industrial materials. The method includes: sending instruction information for controlling the detection control module to the detection control module through the management platform module; collecting detection information related to industrial materials through the synchronization card management module, and sending the detection information to the detection control module; sending image acquisition control instructions to the image acquisition module according to production information and detection information through the detection control module; performing corresponding operations according to the image acquisition control instructions through the image acquisition module; and segmenting the surface image according to the position of the surface defect corresponding to the collected surface image through the surface defect detection module to generate a segmented image.

Description

Technical Field

[0001] The present application relates to the field of information processing technology, and in particular to a surface defect management system, method, device and storage medium for industrial materials. Background Art

[0002] When producing industrial materials such as metal sheets or plastic sheets, surface defects such as scratches or bubbles may occur on the surface of the material due to production equipment failure or processing problems. Surface defects have an adverse effect on the quality and performance of industrial materials. Therefore, it is necessary to detect surface defects and manage surface defects so as to inspect and control surface defects.

[0003] However, there is no surface defect management system for managing surface defects in the prior art. Surface defects are usually counted manually to generate corresponding data tables. However, this method is extremely inconvenient for managing surface defects, resulting in low management efficiency and technical problems with large limitations. Summary of the invention

[0004] The embodiments of the present application provide a surface defect management system, method, device and storage medium for industrial materials to at least solve the technical problems of low efficiency and great limitations in the management of surface defects in the prior art.

[0005] According to one aspect of an embodiment of the present application, a surface defect management method for industrial materials is provided, including: sending instruction information for controlling a detection control module to the detection control module through a management platform module; collecting detection information related to industrial materials through a synchronization card management module, and sending the detection information to the detection control module; sending image acquisition control instructions to an image acquisition module according to production information and detection information through the detection control module; executing corresponding operations according to the image acquisition control instructions through the image acquisition module; and segmenting the surface image according to the positions of surface defects corresponding to the collected surface image through the surface defect detection module to generate a segmented image.

[0006] According to another aspect of an embodiment of the present application, a storage medium is further provided, the storage medium including a stored program, wherein the above-described method is executed by a processor when the program is running.

[0007] According to another aspect of an embodiment of the present application, a surface defect management system for industrial materials is also provided, including: a management platform module, a detection control module, an image acquisition module, a surface defect detection module and a synchronization card management module, the management platform module is configured to perform the following operations: sending instruction information for controlling the detection control module to the detection control module; the synchronization card management module is configured to perform the following operations: collecting detection information related to industrial materials, and sending the detection information to the detection control module; the detection control module is configured to perform the following operations: sending image acquisition control instructions to the image acquisition module according to production information and detection information; the image acquisition module is configured to perform the following operations: executing corresponding operations according to the image acquisition control instructions; and the surface defect detection module is configured to perform the following operations: segmenting the surface image according to the position of the surface defect corresponding to the acquired surface image to generate a segmented image.

[0008] According to another aspect of an embodiment of the present application, a surface defect management device for industrial materials is also provided, including: a first sending module, used to send instruction information for controlling the detection control module to the detection control module through the management platform module; an information acquisition module, used to collect detection information related to industrial materials through the synchronization card management module, and send the detection information to the detection control module; a second sending module, used to send image acquisition control instructions to the image acquisition module according to production information and detection information through the detection control module; an instruction execution module, which performs corresponding operations according to the image acquisition control instructions through the image acquisition module; and an image generation module, which is used to segment the surface image according to the position of the surface defect corresponding to the acquired surface image through the surface defect detection module to generate a segmented image.

[0009] According to another aspect of an embodiment of the present application, a surface defect management method for industrial materials is also provided, including: a processor; and a memory connected to the processor, for providing the processor with instructions for processing the following processing steps: sending instruction information for controlling the detection control module to the detection control module through the management platform module; collecting detection information related to industrial materials through the synchronization card management module, and sending the detection information to the detection control module; sending image acquisition control instructions to the image acquisition module according to production information and detection information through the detection control module; performing corresponding operations according to the image acquisition control instructions through the image acquisition module; and segmenting the surface image according to the position of the surface defect corresponding to the collected surface image through the surface defect detection module to generate a segmented image.

[0010] In the embodiment of the present application, a surface defect management system for industrial materials is constructed, so that the present technical solution manages the surface defects of industrial materials through a complete surface defect management system for industrial materials, thereby making the management of surface defects simpler, reducing the waste of manpower and material resources, and making the management more efficient. And the surface defect management system for industrial materials in the present technical solution includes at least a management platform module, a detection control module, an image acquisition module, a surface defect detection module, and a synchronization card management module, so that corresponding functional modules are constructed according to different functions, so that the coupling of the surface defect management system for industrial materials is low, so that during the later maintenance, the modification of a certain functional module will not affect other functional modules. This makes the later maintenance more convenient, improves the simplicity of maintenance, makes the efficiency higher, and has fewer limitations. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0012] Figure 1 is a schematic diagram of a surface defect management system for industrial materials according to Example 1 of the present application;

[0013] Figure 2 is a schematic diagram of a module of a surface defect management system for industrial materials according to Example 1 of the present application;

[0014] Figure 3A is a schematic diagram of the surface defect detection model according to Example 1 of the present application;

[0015] Figure 3B is another architecture diagram of the surface defect detection model according to Example 1 of the present application;

[0016] Figure 4 is a schematic diagram of a process of extracting features from a first tensor to be encoded according to Embodiment 1 of the present application;

[0017] Figure 5A is a schematic diagram of image data corresponding to pixel points in a partial image area of ​​a binary image according to Embodiment 1 of the present application;

[0018] Figure 5B is a schematic diagram of coordinates corresponding to image data in a partial image area of ​​a binary image according to Embodiment 1 of the present application;

[0019] Figure 5C is a schematic diagram of part of tensor element data of the first tensor feature to be encoded according to Embodiment 1 of the present application;

[0020] Figure 5D is a schematic diagram of part of tensor element data of the third tensor feature to be encoded according to Embodiment 1 of the present application;

[0021] Figure 6 is a schematic diagram of the pooling module according to Example 1 of the present application;

[0022] Figure 7 is a schematic diagram of an upsampling layer of the pooling module according to Example 1 of the present application;

[0023] Figure 8 is a schematic diagram of the unified attention fusion module according to Example 1 of the present application;

[0024] Fig. 9 is a schematic flow chart of a method for managing surface defects of industrial materials according to the first aspect of Example 1 of the present application;

[0025] Fig.10 is a schematic diagram of a surface defect management device for industrial materials according to Example 2 of the present application; and

[0026] Fig.11 This is a schematic diagram of a surface defect management device for industrial materials according to Example 3 of the present application. DETAILED DESCRIPTION

[0027] In order to enable those skilled in the art to better understand the technical solutions of the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below in conjunction with the drawings in the embodiments of the present application. Obviously, the described embodiments are only embodiments of a part of the present application, not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present application.

[0028] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device comprising a series of steps or units is not necessarily limited to those steps or units clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0029] Example 1

[0030] Figure 1 Schematic diagram of the surface defect management system for industrial materials according to this embodiment. Figure 1 As shown, the system includes: a server 100 , an industrial computer 210 and an industrial computer 220 .

[0031] The server 100 is connected to the industrial computer 210 and the industrial computer 220 through the switch 300. The industrial computer 210 and the industrial computer 220 are connected to multiple cameras through the acquisition network port. The multiple cameras connected to the industrial computer 210 are upper surface camera groups, which are used to collect surface images of the upper surface of industrial materials. The multiple cameras connected to the industrial computer 220 are lower surface camera groups, which are used to collect surface images of the lower surface of industrial materials. The number of industrial computers can be set according to actual conditions.

[0032] refer to Figure 2 As shown, the server 100 is provided with a management platform module, a synchronization card management module, a light source control module, a detection control module and a storage module.

[0033] The industrial computer 210 and the industrial computer 220 are respectively provided with an image acquisition module and a surface defect detection module, and the industrial computer 210 is also provided with a model training management module.

[0034] The management platform module is used to receive instruction information sent by the user through the terminal device, and send the instruction information to the corresponding functional module.

[0035] The synchronization card management module is used to collect detection information and send the detection information to the detection control module, wherein the detection information includes the detection length, detection speed and light sensing state corresponding to the industrial materials when the industrial materials are produced.

[0036] The detection control module is used to obtain production information from the production line system and detection information from the synchronization card management module. The production line system is used to indicate the system for producing industrial materials. The production information includes the production line speed and the scissors status. The production line speed is used to indicate the speed of producing industrial materials. The scissors status is used to indicate the state of cutting industrial materials. After that, the detection control module sends an image acquisition control instruction to the image acquisition module according to the production information and the synchronization card management module. The image acquisition control instruction includes starting acquisition, pausing acquisition, continuing acquisition and stopping acquisition.

[0037] The image acquisition module is used to perform corresponding operations according to the image acquisition control instructions sent by the detection control module, and send the collected surface image of the industrial material to the surface defect detection module.

[0038] The surface defect detection module is used to detect surface defects based on the surface image, determine the location of the surface defects, and then segment the surface image according to the location of the surface defects to generate a segmented image. The segmented image of the surface defect detection module is then sent to the storage module for storage.

[0039] The light source is arranged around the industrial material. Thus, the user can select to turn on or off the light source in the management platform client, thereby sending the instruction information for controlling the light source to the management platform module, and then the management platform module sends the instruction information for controlling the light source to the light source control module. The light source control module controls the light source to turn on or off.

[0040] The model training management module is used to train the surface defect detection model and store the trained surface defect detection model in the storage module. After the surface defect detection model is put into use, the model training management module will also use the defect data obtained after it is put into use as a sample to train and update the surface defect detection model regularly.

[0041] According to a first aspect of this embodiment, referring to Figure 1 as well as Figure 2 As shown, a surface defect management system for industrial materials is provided, comprising: a management platform module, a detection control module, an image acquisition module, a surface defect detection module and a synchronization card management module, wherein the management platform module is configured to perform the following operations: sending instruction information for controlling the detection control module to the detection control module; the synchronization card management module is configured to perform the following operations: collecting detection information related to industrial materials, and sending the detection information to the detection control module; the detection control module is configured to perform the following operations: sending image acquisition control instructions to the image acquisition module according to production information and detection information; the image acquisition module is configured to perform the following operations: executing corresponding operations according to the image acquisition control instructions; and the surface defect detection module is configured to perform the following operations: segmenting the surface image according to the position of the surface defect corresponding to the acquired surface image to generate a segmented image.

[0042] Specifically, the management platform module is configured to perform the following operations: receive instruction information for controlling the detection control module sent by the user through the terminal device, and send the instruction information to the detection control module; and receive response information returned by the detection control module, and the synchronization card management module is configured to perform the following operations: collect detection information related to industrial materials, and send the detection information to the detection control module, wherein the detection information includes the detection length and light sensing state corresponding to the industrial materials when the industrial materials are produced, wherein the detection length is used to indicate the production length of the industrial materials detected during the production of the industrial materials, and the light sensing state is used to indicate whether the light is blocked by the industrial materials during the production of the industrial materials, and the detection control module is configured to perform the following operations: obtain production information related to the industrial materials from the production line system; information, and obtains detection information from the synchronization card management module, wherein the production line system is used to produce industrial materials, and the production information includes the production line speed; and sends an image acquisition control instruction to the image acquisition module according to the production information and the detection information, wherein the image acquisition control instruction includes starting image acquisition, pausing image acquisition, continuing image acquisition or stopping image acquisition, and the image acquisition module is configured to perform the following operations: receiving the image acquisition control instruction sent by the detection control module; performing corresponding operations according to the image acquisition control instruction; and sending the collected surface image of the industrial material to the surface defect detection module, and the surface defect detection module is configured to perform the following operations: detecting surface defects according to the surface image and determining the location of the surface defects; and segmenting the surface image according to the location of the surface defects to generate a segmented image.

[0043] More specifically, the user's terminal device is installed with a management platform client, so that the user controls the surface defect management system of industrial materials through the management platform client.

[0044] For example, the user selects "start detection" on the management platform client, so that the user's terminal device sends the instruction information of "start detection" to the management platform module set in the server 100. After receiving the instruction information of "start detection", the management platform module sends the instruction information of "start detection" to the detection control module. The detection control module generates the instruction information of "start image acquisition" for starting to collect surface images and the instruction information of "start detection information acquisition" for starting to collect detection information according to the received instruction information of "start detection". After that, the detection control module sends the instruction information of "start image acquisition" to the image acquisition module set in the industrial computer 210 and the industrial computer 220. And the detection control module sends the instruction information of "start detection information acquisition" to the synchronization card management module.

[0045] Furthermore, each image acquisition module controls the camera to start according to the received instruction information of "start image acquisition".

[0046] Thus, the image acquisition module acquires the surface image of the industrial material captured by the camera, and then sends the acquired surface image to the surface defect detection module.

[0047] Furthermore, the synchronization card management module starts to collect detection information related to industrial materials according to the received instruction information of "starting detection information collection". The detection information includes the detection speed, detection length and light sensing state corresponding to the industrial materials when the industrial materials are produced. The synchronization card management module is connected to a detection device for obtaining the detection length and a light sensing device for obtaining the light sensing state.

[0048] The detection length is the production length of the industrial material detected during the production of the industrial material, and the detection speed is the production speed of the industrial material detected during the production of the industrial material. The light sensing state is used to indicate whether the light is blocked by the industrial material during the production of the industrial material. When the light sensing state is blocked, it means that the industrial material is in production, and when the light sensing state is not blocked, it means that the industrial material is not in production.

[0049] The detection length = (current pulse number - base point pulse number) * detection device roller circumference / detection device output pulse number per rotation,

[0050] The detection device is, for example, a roller-shaped encoder, which measures the production length of the industrial material through its circumference.

[0051] When the synchronization card management module receives the instruction information to start collecting detection information, it first resets the detection speed and detection length to 0 and obtains the pulse signal of the detection device, thereby obtaining the current pulse number of the meter serial port as the base point pulse number. The current pulse number is obtained in real time at different times.

[0052] The detection speed = (current detection length - last detection length) / detection length collection period. The detection length collection period is the time length for collecting the detection length.

[0053] Further, the detection control module sends an acquisition request for acquiring detection information to the synchronization card management module, and the synchronization card management module sends the collected detection information to the detection control module according to the acquisition request. After acquiring the detection information, the detection control module acquires production information related to industrial materials from the production line system. The production line system is used to indicate a system for producing industrial materials. And the production information includes a production line speed. The production line speed is used to indicate the speed of producing industrial materials.

[0054] Furthermore, the synchronization card management module determines whether to control the camera to shoot according to the detection length in the detection information.

[0055] More specifically, during the production process of industrial materials, the industrial materials are constantly moving, and the camera is in a fixed state. The synchronization card management module obtains the detection length in real time through the detection device, and sends the camera a command message for shooting at a predetermined length interval, thereby controlling the camera to shoot the surface of the industrial material being produced. Therefore, the synchronization card management module can control the camera to continuously shoot the industrial material according to the detection length, so that the surface of the industrial material can be shot by the camera.

[0056] Furthermore, the detection control module generates corresponding image acquisition control instructions according to the production information and the detection information, and sends the image acquisition control instructions to the image acquisition module, wherein the image acquisition control instructions include starting image acquisition, pausing image acquisition, continuing image acquisition or stopping image acquisition.

[0057] Furthermore, after the image acquisition module starts to acquire the surface image and sends the surface image to the surface defect detection module, the surface defect detection module performs surface defect detection on the surface image through the surface defect detection model and starts the process of surface defect detection. The process is determined by the number of GPUs, and multiple processes share one GPU.

[0058] Each process separately obtains a set of surface images to be inspected from the inspection queue, wherein a set of surface images is the surface images collected by the upper surface camera group or the lower surface camera group at the same time.

[0059] Each group of surface images will carry global vertex coordinates (ie, the coordinates of the four vertices of each surface image in the entire industrial material) and group ID and other information, so that the global coordinates of the surface defects can be calculated through the global vertex coordinates of the surface image.

[0060] If the surface defect detection module detects a surface defect, the contour of the surface defect is determined by the findcontours() function, the length and width of the surface defect are determined by the minimum circumscribed rectangle, the center coordinates of the surface defect are determined by the minimum circumscribed circle, and the area of ​​the surface defect is calculated by counting the number of non-zero pixels in the surface defect.

[0061] Furthermore, the surface defect detection module divides the surface image according to the contour, length, width and center coordinates of the surface defect to generate a rectangular segmented image, wherein the segmented image includes the surface defect.

[0062] Furthermore, after the production of industrial materials is completed, the detection control module sends a response message of production completion to the management platform module, and then the management platform module sends the received response message to the management platform client, so that the management platform client displays the response message.

[0063] In addition, a light source control module for controlling the light source is also provided in the server 100, wherein the light source is provided around the industrial material. Thus, the user can select to turn on or off the light source in the management platform client, thereby sending the instruction information for controlling the light source to the management platform module, and then the management platform module sends the instruction information for controlling the light source to the light source control module. The light source is controlled to turn on or off by the light source control module.

[0064] In addition, a model training management module is also provided in the industrial computer 210. The model training management module is used to train the surface defect detection model and store the trained surface defect detection model in the storage module. After the surface defect detection model is put into use, the model training management module will also use the defect data obtained after the use as a sample to train and update the surface defect detection model regularly.

[0065] As described in the background technology, when producing industrial materials such as metal sheets or plastic sheets, surface defects such as scratches or bubbles may be caused on the surface of the material due to production equipment failure or processing technology problems. Surface defects have an adverse effect on the quality and performance of industrial materials. Therefore, it is necessary to detect surface defects and manage surface defects, so as to inspect and control surface defects. However, there is no surface defect management system for managing surface defects in the prior art. Usually, surface defects are manually counted to generate corresponding data tables. However, such a method is extremely inconvenient for the management of surface defects, resulting in low management efficiency and technical problems with large limitations.

[0066] In view of the technical problems described above, a surface defect management system for industrial materials is constructed through the technical solution of the embodiment of the present application, so that the present technical solution manages the surface defects of industrial materials through a complete surface defect management system for industrial materials, thereby making the management of surface defects simpler, reducing the waste of manpower and material resources, and making the management more efficient. And the surface defect management system for industrial materials in the present technical solution includes at least a management platform module, a detection control module, an image acquisition module, a surface defect detection module, and a synchronization card management module, so that corresponding functional modules are constructed according to different functions, so that the coupling of the surface defect management system for industrial materials is low, so that during the later maintenance, the modification of a certain functional module will not affect other functional modules. This makes the later maintenance more convenient, improves the simplicity of maintenance, and makes the efficiency higher and the limitations smaller.

[0067] Optionally, the surface defect detection module is further configured to perform the following operations: sending the segmented image to a storage module for storage. The storage module is a cloud storage. Thus, the technical solution stores the segmented image in the cloud, so that it does not occupy too much memory and is more convenient to access. In addition, the surface defect detection module also sends information such as the defect type, size, and coordinates of the surface defect to the management platform module for storage.

[0068] Optionally, the operation of sending an image acquisition control instruction to the image acquisition module based on the production information and the detection information includes: generating instruction information for starting image acquisition when the production status is in production and the detection length in the detection information is greater than or equal to a first preset threshold; and sending the instruction information for starting image acquisition to the image acquisition module.

[0069] Specifically, after receiving the "start image acquisition" instruction information sent by the management platform module, the detection control module first determines whether the "start image acquisition" instruction information should be sent to the image acquisition module based on the production information and the detection information.

[0070] Then the detection control module determines the production status of the industrial product and obtains the detection length in the detection information. The production status includes stopped, in production, in detection, paused and waiting to stop.

[0071] When the production of industrial products begins, the production status is in production. When the production status is in production and the detection length is greater than or equal to the first preset threshold, an instruction message for starting image acquisition is generated, and the production status is modified to in detection, and the production line system will continue to produce industrial materials. The first preset threshold is the head margin length of the industrial material, that is, before starting to collect the surface image, the head of the industrial material must be moved to the range that the camera can capture. Then the detection starts from the head of the industrial material, so that when the detection length is greater than or equal to the head margin length (that is, the first preset threshold), it means that the head of the industrial material has moved to the range that the camera can capture.

[0072] Furthermore, the detection control module sends instruction information for starting image acquisition to the image acquisition module.

[0073] In addition, the production information also includes a scissor state, wherein the scissor state is used to indicate the state of cutting the industrial material. The scissor state includes: cut or uncut. When the industrial material is still being produced, the scissor state is uncut; when the industrial material is completed, the scissor state is cut.

[0074] When the production status is stopped, the scissors status is uncut, the production line speed is greater than 0, and the light sensing status is blocked, the detection control module sends the instruction information of "start detection information collection" to the synchronization card management module.

[0075] Therefore, the present technical solution automatically determines whether image acquisition should be started according to production information and detection information, and determines image acquisition control instructions according to production status, thereby reducing human intervention errors and improving data acquisition efficiency.

[0076] Optionally, the operation of sending image acquisition control instructions to the image acquisition module based on production information and detection information also includes: when the production status is paused or waiting to stop, and the production line speed in the production information is greater than a second preset threshold, generating instruction information for continuing image acquisition; and sending the instruction information for continuing image acquisition to the image acquisition module.

[0077] Specifically, after receiving the instruction information of "starting image acquisition" or "starting detection information acquisition", the detection control module will acquire the production information and detection information at a predetermined time interval.

[0078] Therefore, when the production status is paused or waiting to stop, and the production line speed is greater than the second preset threshold, the detection control module generates instruction information to continue image acquisition, and the production status is changed to being detected. The second preset threshold is 0.

[0079] Furthermore, the detection control module sends instruction information for continuing image acquisition to the image acquisition module.

[0080] Therefore, the technical solution automatically determines whether image acquisition should continue based on production information and detection information, and determines image acquisition control instructions based on production status, thereby reducing manual intervention errors and improving data acquisition efficiency.

[0081] Optionally, the operation of sending image acquisition control instructions to the image acquisition module based on production information and detection information also includes: when the production status is under detection and the production line speed in the production information is equal to a second preset threshold, generating instruction information for pausing image acquisition; and sending the instruction information for pausing image acquisition to the image acquisition module.

[0082] Specifically, after receiving the instruction information of "start detection", the detection control module will obtain production information and detection information at predetermined time intervals.

[0083] Therefore, when the production status is under inspection and the production line speed is equal to the second preset threshold, the inspection control module generates instruction information for pausing image acquisition, and the production status is modified to being suspended. The second preset threshold is 0.

[0084] Furthermore, the detection control module sends instruction information for pausing image acquisition to the image acquisition module.

[0085] Therefore, the technical solution automatically determines whether image acquisition should be suspended according to production information and detection information, and determines image acquisition control instructions according to the production status, thereby reducing manual intervention errors and improving data acquisition efficiency.

[0086] Optionally, the operation of sending an image acquisition control instruction to the image acquisition module based on the production information and the detection information also includes: when the production status is detection, the detection length is greater than or equal to a third preset threshold, and the light sensitivity state in the detection information is unobstructed, generating instruction information for stopping image acquisition; and sending the instruction information for stopping image acquisition to the image acquisition module.

[0087] Specifically, when the production state is paused and the scissor state value is cut, the end length is generated and recorded, and the production state is modified to waiting for stop, wherein the end length is the detection length.

[0088] When the production state is in detection, the detection length is greater than or equal to the third preset threshold, and the light sensing state is unobstructed, the detection control module generates instruction information for stopping image acquisition and generates instruction information for stopping detection information acquisition. The third preset threshold is the end length.

[0089] Furthermore, the detection control module sends instruction information for stopping image acquisition to the image acquisition module, and sends instruction information for stopping detection information acquisition to the synchronization card management module.

[0090] Therefore, the technical solution automatically determines whether image acquisition should be stopped according to production information and detection information, and determines image acquisition control instructions according to production status, thereby reducing manual intervention errors and improving data acquisition efficiency.

[0091] Optionally, the operation of detecting surface defects based on a surface image includes: inputting the surface image into an encoding network of a surface defect detection model, and generating a first feature tensor corresponding to the surface image through an encoding operation of at least one encoding module in the encoding network, wherein the surface image is obtained by photographing the surface of the industrial material; generating a second feature tensor based on the first feature tensor using a decoding network of the surface defect detection model corresponding to the encoding network; and classifying the second feature tensor using a softmax classifier of the surface defect detection model to generate a prediction tensor corresponding to the surface image, wherein the prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at a corresponding position on the surface of the industrial material.

[0092] Specifically, a surface defect detection model is provided in the surface defect detection module, so that the surface defect detection module detects the surface image using the surface defect detection model, thereby determining whether there are surface defects on the surface of the industrial material and determining the type of the surface defects.

[0093] refer to Figure 3A As shown, the surface defect detection model includes an encoding network, a decoding network, a pooling module and a softmax classifier, the encoding network includes encoding modules 1 to 2, and the decoding network includes multiple decoding modules 1 to 2.

[0094] Thus, the surface defect detection module inputs the surface image into the encoding network, and then the encoding module 1 in the encoding network receives the surface image and performs encoding operation on the surface image, thereby generating a corresponding feature map 101. The surface defect detection module then inputs the feature map 101 into the encoding module 2, and the encoding module 2 performs encoding operation on the received feature map 101, thereby outputting a corresponding feature map 102 (i.e., a first feature tensor). Before performing encoding operation on the surface image, the surface defect detection module may first perform convolution operation on the surface image through a CNN network to obtain a corresponding feature map. The surface defect detection module then performs encoding operation on the feature map through the encoding module in the encoding network.

[0095] Furthermore, the surface defect detection module position-encodes the feature map 102 (ie, the first feature tensor) and performs a pooling operation through a pooling unit, thereby outputting the pooled feature map 102 .

[0096] Furthermore, the surface defect detection module inputs the pooled feature map 102 output by the encoding module 2 to the pooling module. The pooling module processes the received pooled feature map 102 and outputs a corresponding feature map.

[0097] Furthermore, the surface defect detection module inputs the feature map output by the pooling module into the decoding network, so that the deconvolution unit of the decoding module 1 in the decoding network extracts features from the feature map, thereby outputting a feature map 301. The surface defect detection module then inputs the feature map 301 into the unified attention fusion module 1, and the unified attention fusion module 1 extracts features from the feature map 301, thereby outputting a feature map 302.

[0098] Furthermore, the surface defect detection module inputs the feature map 302 to the decoding module 2 , and the decoding module 2 uses a deconvolution unit to extract features from the feature map 302 , thereby outputting a feature map 303 .

[0099] Then, the surface defect detection module inputs the feature map 303 to the deconvolution module in the decoding network, and the deconvolution module performs an upsampling operation on the feature map 303 to generate a feature map 304, so that the encoding network outputs the feature map 304 as the second feature tensor.

[0100] Further, the surface defect detection module inputs the feature map 304 (i.e., the second feature tensor) into a softmax classifier. The softmax classifier performs classification processing on the feature map 304 (i.e., the second feature tensor) to generate a prediction tensor corresponding to the surface image. The prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position on the surface of the industrial material.

[0101] For example, this embodiment includes 12 types of surface defects and 1 type of surface background, for a total of 13 surface types. Thus, the prediction tensor has a total of 13 prediction matrices. Each prediction matrix corresponds to one surface type. And the elements of each prediction matrix correspond to the pixel points at the corresponding position of the surface image, so that each element in each prediction matrix represents the probability value of the corresponding pixel point being the corresponding surface type. The prediction tensor order is a tensor greater than or equal to 2.

[0102] The 12 types of surface defects are: dirt, scratches, corrosion, black spots, black stripes, scratches, cracked edges, water marks, prints, stickiness, peeling and bubbles.

[0103] For example, prediction matrix 1 corresponds to a surface defect of the dirt type. Thus, in the prediction matrix 1, each element corresponds to a probability value that a pixel point of the surface image is a surface defect of the dirt type.

[0104] For example, the pixels of the surface image are 4096*1000*3, and the surface defect detection model outputs a surface defect map of 4096*1000*13.

[0105] In another embodiment, reference Figure 3B As shown, the surface defect detection model includes an encoding network, a decoding network, a pooling module and a softmax classifier, the encoding network includes encoding modules 1 to n, and the decoding network includes multiple decoding modules 1 to n.

[0106] Thus, the surface defect detection module inputs the surface image into the encoding network, and then the encoding module 1 in the encoding network receives the surface image and performs encoding operation on the surface image, thereby generating a corresponding feature map 101. The surface defect detection module then inputs the feature map 101 into the encoding module 2, and the encoding module 2 performs encoding operation on the received feature map 101, thereby outputting a corresponding feature map 102, and so on, until the encoding module n-1 outputs the corresponding feature map ( Figure 3A The feature map is input to the encoding module n, and then the encoding module n outputs the corresponding feature map according to the feature map. Fig.10 n (i.e., the first eigentensor).

[0107] Furthermore, the surface defect detection module will feature Fig.10 n (i.e., the first feature tensor) is positionally encoded and pooled through the pooling unit to output the pooled feature Fig.10 n.

[0108] Furthermore, the surface defect detection module uses the pooled features output by the encoding module n Fig.10 n is input to the pooling module. The pooling module receives the pooled features Fig.10 n is processed and the corresponding feature map is output.

[0109] Furthermore, the surface defect detection module inputs the feature map output by the pooling module into the decoding network, so that the decoding network uses the deconvolution unit of the decoding module 1 to extract features from the feature map, thereby outputting a feature map 301. The surface defect detection module then inputs the feature map 301 into the unified attention fusion module 1, and the unified attention fusion module 1 extracts features from the feature map 301, thereby outputting a feature map 302.

[0110] Furthermore, the surface defect detection module inputs the feature map 302 to the decoding module 2 , and the decoding module 2 uses a deconvolution unit to extract features from the feature map 302 , thereby outputting a feature map 303 .

[0111] And so on, until the decoding module n outputs the corresponding feature map 30m.

[0112] in Figure 3B The process of unified attention fusion module n-2, decoding module n-1, and unified attention fusion module n-1 outputting feature maps is the same as above ( Figure 3B The feature graph output by the above module is not shown in FIG, and will not be described here.

[0113] Then, the surface defect detection module inputs the feature map 30m to the deconvolution module in the decoding network. The deconvolution module performs an upsampling operation on the feature map 30m to generate a feature map 401, so that the encoding network outputs the feature map 401 as the second feature tensor.

[0114] Further, the surface defect detection module inputs the feature map 401 (i.e., the second feature tensor) into a softmax classifier. The softmax classifier performs classification processing on the feature map 401 (i.e., the second feature tensor) to generate a prediction tensor corresponding to the surface image. The prediction tensor includes at least one prediction matrix, wherein each prediction matrix corresponds to a different type of surface defect, and the elements of the prediction matrix are used to indicate the probability value of the surface defect corresponding to the prediction matrix existing at the corresponding position on the surface of the industrial material.

[0115] Therefore, the present technical solution uses a surface defect detection model constructed by an encoding network and a decoding network to identify and detect surface defects on the surface of industrial materials, so that the deep learning method can accurately and easily identify and judge surface defects under complex conditions. Compared with the prior art, the present technical solution does not require manual parameter adjustment, and uses the deep learning method to improve the accuracy of parameters, thereby improving the accuracy of surface defect detection. This solves the technical problem of inaccurate detection results when detecting surface defects through traditional image processing methods and machine learning methods based on manual features or shallow learning technology in the prior art.

[0116] Optionally, the encoding operation of at least one encoding module in the encoding network includes: receiving an input tensor to be encoded; using at least one convolutional layer to convolve the tensor to be encoded to generate a first tensor feature to be encoded corresponding to the tensor to be encoded; using a feature extraction module based on a global attention mechanism and a local attention mechanism to generate a second tensor feature to be encoded corresponding to the first tensor feature to be encoded; and merging the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coded tensor as the encoding result.

[0117] Specifically, refer to Figure 3A As shown, the encoding module 1 receives the input tensor to be encoded corresponding to the surface image, and uses at least one convolutional layer (such as Figure 3A The convolution unit in the encoding module 1 performs a convolution operation on the tensor to be encoded, thereby generating a first tensor feature to be encoded corresponding to the tensor to be encoded.

[0118] Furthermore, the surface defect detection module inputs the first tensor feature to be encoded into a feature extraction module, wherein the feature extraction module includes a feature extraction unit based on global attention and a feature extraction unit based on a local attention mechanism.

[0119] Therefore, the feature extraction module extracts features of the first tensor feature to be encoded through a feature extraction unit based on global attention and a feature extraction unit based on a local attention mechanism, thereby generating corresponding second tensor features to be encoded.

[0120] Furthermore, the surface defect detection module adds the elements of the first tensor feature to be encoded and the second tensor feature to be encoded at corresponding positions, thereby fusing the first tensor feature to be encoded and the second tensor feature to be encoded to generate a coded tensor as the encoding result.

[0121] In addition, other encoding modules process the received feature map (i.e., the tensor to be encoded) according to the method of processing the tensor to be encoded by the above-mentioned encoding module 1 to generate the encoding tensor, so as to generate a new feature map (i.e., the encoding tensor), which will not be repeated here.

[0122] The tensor to be encoded received by encoding module 1 is a surface image, and the tensors to be encoded received by other encoding modules are feature maps output by the previous encoding module.

[0123] Therefore, the traditional image processing method in the prior art only processes the image through the convolution layer and the pooling layer. On this basis, the present technical solution adds a method for extracting semantic features, thereby extracting the semantic features of the image through the feature extraction module, thereby improving the accuracy of surface defect detection.

[0124] Optionally, the operation of generating a second tensor feature to be encoded corresponding to the first tensor feature to be encoded using a feature extraction module based on a global attention mechanism and a local attention mechanism includes: generating a binary image corresponding to the first tensor feature to be encoded, wherein a connected domain of the binary image corresponds to a surface defect area on the surface of the industrial material; generating a third tensor feature to be encoded based on the binary image and the first tensor feature to be encoded, wherein the tensor element values ​​corresponding to the connected domain in the third tensor feature to be encoded are the same as the corresponding tensor element values ​​in the first tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0; performing a block embedding operation on the first tensor feature to be encoded to obtain a first sequence, and performing feature extraction on the first sequence using a feature extraction unit based on a global attention mechanism to determine a global semantic feature corresponding to the first tensor feature to be encoded; performing a block embedding operation on the third tensor feature to be encoded to obtain a second sequence, and performing feature extraction on the second sequence using a feature extraction unit based on a local attention mechanism to determine a local semantic feature corresponding to the third tensor feature to be encoded; and merging the global semantic feature and the local semantic feature to determine the second tensor feature to be encoded.

[0125] Specifically, see Figure 3A and Figure 3BAs shown, the surface defect detection model also includes an image segmentation model, which segments and binarizes the surface image to generate a binary image, wherein the binary image includes a connected domain corresponding to the defect area in the surface image. In addition, the generated binary image is input to the feature extraction module of each encoding module. After the image segmentation model generates the binary image, the binary image can also be subjected to a patch-embedding operation to generate a corresponding sequence, and then the corresponding sequence is input to the feature extraction module of each encoding module.

[0126] Figure 4 Further diagrams of feature extraction modules of various encoding modules are shown, see Figure 4 As shown, the feature extraction module scales the binarized image through the image scaling module so that the generated binary image is consistent with the size of the first tensor feature to be encoded.

[0127] Further, among them Figure 5A Image data corresponding to pixel points in a partial image area of ​​a binary image is shown. Figure 5B The coordinates corresponding to the image data in the partial image area of ​​the binary image are shown. Figure 5A and Figure 5B As shown in the figure, the pixel value of the defective area in the binary image is 1, and the pixel value of the background area is 0. Therefore, the surface defect detection module determines the area with a pixel value of 1 as a connected domain, where the coordinates of the area with a pixel value of 1 are x(1, 1), x(1, 2), x(2, 1) and x(2, 2), respectively. The defective area is determined based on the connected domain, that is, the area with a pixel value of 1 is a connected domain and also a defective area.

[0128] Further, Figure 5C shows part of the tensor element data of the first tensor feature to be encoded, Figure 5D Part of the tensor element data of the third tensor feature to be encoded is shown. The surface defect detection module maps the first tensor feature to be encoded to the binary image, so that each tensor element in the first tensor feature to be encoded corresponds to the pixel point at the same position of the binary image. Therefore, the surface defect detection module uses the tensor element value corresponding to the connected domain in the first tensor feature to be encoded as the tensor element value corresponding to the connected domain in the third tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0, thereby obtaining the third tensor feature to be encoded. The tensor element value corresponding to the connected domain in the third tensor feature to be encoded is the same as the corresponding tensor element value in the first tensor feature to be encoded, and the remaining tensor element values ​​in the third tensor feature to be encoded are 0.

[0129] by FIG. 5A to FIG. 5DFor example, the tensor element values ​​at the positions x(1,1), x(1,2), x(2,1) and x(2,2) in the third tensor feature to be encoded are determined according to the tensor element values ​​at the positions x(1,1), x(1,2), x(2,1) and x(2,2) in the third tensor feature to be encoded. The tensor element values ​​at the remaining positions in the third tensor feature to be encoded are determined according to the pixel values ​​at the corresponding positions in the binary image.

[0130] For example, at the corresponding positions, the tensor element values ​​of the first tensor feature to be encoded are multiplied by the pixel values ​​of the binary image to obtain the tensor element values ​​of each position of the third tensor feature to be encoded. Thus, at the positions x(1,1), x(1,2), x(2,1) and x(2,2), the tensor element values ​​of the third tensor feature to be encoded are the same as the tensor element values ​​of the first tensor feature to be encoded, which are x 1，1 、x 1，2 、x 2，1 and x 2，2 At other positions, the tensor element value of the third tensor feature to be encoded is 0.

[0131] Further, the surface defect detection module performs a block embedding operation on the first tensor feature to be encoded to obtain a first sequence, and the surface defect detection module performs a block embedding operation on the third tensor feature to be encoded to obtain a second sequence.

[0132] Further, the feature extraction module includes a feature extraction unit based on a global attention mechanism and a feature extraction unit based on a local attention mechanism. Thus, the surface defect detection module inputs the first sequence into the feature extraction unit based on the global attention mechanism, and inputs the second sequence into the feature extraction unit based on the local attention mechanism.

[0133] Therefore, the feature extraction unit based on the global attention mechanism calculates the weight value corresponding to each element in the first sequence, and then calculates the global semantic feature according to each element and the corresponding weight value.

[0134] The feature extraction unit based on the local attention mechanism calculates the weight value corresponding to each element in the second sequence, and then calculates the local semantic feature according to each element and the corresponding weight value, wherein the number of elements in the first sequence is the same as the number of elements in the second sequence.

[0135] Thus, the feature extraction unit based on the global attention mechanism outputs the global semantic feature corresponding to the first tensor feature to be encoded, and the feature extraction unit based on the local attention mechanism outputs the local semantic feature corresponding to the third tensor feature to be encoded. The global attention mechanism and the local attention mechanism can be a multi-head attention mechanism. In this embodiment, the number of global attention heads is the same as that of local attention heads.

[0136] Furthermore, the surface defect detection module combines the global semantic features and the local semantic features to determine the second tensor features to be encoded.

[0137] In addition, the feature extraction unit outputs the formula for the element value of each element of the second tensor feature to be encoded as follows:

[0138] [γ1,γ2,...,γ t ] T =[x1, x2, ..., x t ] T +[y1,y2,...,y t ] T ;

[0139]

[0140]

[0141] as well as

[0142]

[0143] [x1, x2, ..., x t ] T Represents the global semantic features, [y1, y2, ..., y t ] T Represents local semantic features, represents the first sequence corresponding to the first tensor feature to be encoded, represents the second sequence corresponding to the third tensor feature to be encoded, Vg i V parameter of the global attention mechanism, kg i represents the K parameter of the global attention mechanism, Vl i V parameter represents the local attention mechanism, kl i represents the K parameter of the local attention mechanism. d is or The vector dimension of .

[0144] Therefore, the technical solution extracts the global semantic features of the first tensor feature to be encoded through the global attention mechanism, and extracts the local semantic features of the third tensor feature to be encoded through the local attention mechanism, and then fuses the global semantic features and the local semantic features. Therefore, the technical solution not only considers the feature level of the entire feature map, but also measures the local edge features of the feature map, making the detection of surface defects more accurate and improving the detection accuracy.

[0145] Optionally, the operation of generating a second feature tensor based on the first feature tensor using a decoding network corresponding to the encoding network includes: inputting the first feature tensor into a pooling module, performing a downsampling operation on the first feature tensor layer by layer through the pooling module to generate a downsampled tensor; upsampling the downsampled tensor layer by layer through the pooling module to generate a third feature tensor corresponding to the first feature tensor; and inputting the third feature tensor into a decoding network, and generating a second feature tensor corresponding to the third feature tensor through a decoding operation of at least one decoding module in the decoding network.

[0146] Specifically, refer to Figure 6 As shown, the pooling module includes multiple adjacent downsampling layers and multiple adjacent upsampling layers.

[0147] After the encoding network outputs the first feature tensor, the surface defect detection module inputs the first feature tensor to the upsampling layer of the pooling module, so that multiple upsampling layers downsample the received first feature tensor layer by layer to generate a downsampled tensor.

[0148] Furthermore, the surface defect detection module inputs the downsampled tensor into the upsampled layer of the pooling module, so that the multiple upsampled layers upsample the received downsampled tensor layer by layer, thereby outputting a third feature tensor.

[0149] Furthermore, the surface defect detection module inputs the third feature tensor into the decoding network. The decoding network includes multiple decoding modules. Thus, the multiple decoding modules perform decoding operations on the third feature tensor to obtain corresponding feature maps, and then the deconvolution module in the decoding network performs upsampling operations on the feature maps to generate corresponding decoding feature maps, so that the encoding network outputs the decoding feature maps as the second feature tensor.

[0150] Therefore, the technical solution decodes the feature map through a simplified pooling module and a decoding module, which reduces the decoding process and time and improves the decoding speed.

[0151] Optionally, the downsampled tensor is upsampled layer by layer through a pooling module to generate an operation of a third feature tensor corresponding to the first feature tensor, including: receiving an input tensor to be upsampled; performing an average pooling operation on the tensor to be upsampled through a first average pooling unit of the pooling module to generate a first pooling tensor; performing average pooling operations on the first pooling tensor through multiple second average pooling units of different sizes to generate corresponding multiple second pooling tensors; and merging the multiple second pooling tensors after convolution and upsampling operations to generate an upsampled tensor.

[0152] Specifically, refer to Figure 7As shown, each upsampling layer includes a first average pooling unit, multiple second average pooling units, multiple first convolution units, a second convolution unit and multiple upsampling units.

[0153] For example, the surface defect detection module inputs the tensor to be upsampled into the average pooling unit 1 (ie, the first average pooling unit) of the upsampling layer, and the average pooling unit 1 performs an average pooling operation on the tensor to be upsampled to generate a corresponding feature map 201 (first pooling tensor).

[0154] Further, the surface defect detection module inputs the feature map 201 into multiple average pooling units of different sizes (i.e., second average pooling units). For example, there are 3 average pooling units (i.e., average pooling units 2 to 4), with sizes of 1×1, 2×2, and 4×4, respectively. Thus, the 3 average pooling units perform average pooling operations on the feature map 201 (first pooling tensor), respectively, thereby generating feature maps 202 to 204 (i.e., multiple second pooling tensors), respectively.

[0155] Furthermore, the surface defect detection module inputs the feature maps 202 to 204 (i.e., multiple second pooling tensors) into the corresponding first convolution units, respectively. The convolution kernel in the first convolution unit is 1×1, and the output channel is smaller than the input channel. And one of the second average pooling units corresponds to a first convolution unit and an upsampling unit, so that when the second average pooling units include 3, the first convolution unit and the upsampling unit also include 3 respectively. Thus, the three first convolution units (i.e., first convolution units 1 to 3) generate feature maps 205 to 207, respectively.

[0156] Further, the surface defect detection module inputs the feature maps 205-207 into corresponding upsampling units respectively, so that the three upsampling units (ie, upsampling units 1-3) output feature maps 208-210. The feature maps 208-210 have the same size.

[0157] Furthermore, the surface defect detection module adds the feature elements at corresponding positions of the feature maps 208 to 210 to generate a feature map 211 .

[0158] Further, the surface defect detection module inputs the feature map 211 into the second convolution unit. The convolution kernel in the second convolution unit is 1×1. Thus, the second convolution unit outputs a feature map 212 (ie, an upsampled tensor).

[0159] Furthermore, the surface defect detection module inputs the feature map 212 (i.e., the upsampled tensor) as the tensor to be upsampled into the next upsampled layer, so that the next upsampled layer (i.e., the current upsampled layer) performs an upsample operation on the received feature map 212 according to the method of generating the upsampled tensor by the previous upsampled layer, and generates a corresponding upsampled tensor.

[0160] Therefore, other upsampling layers perform upsampling processing on the upsampled tensor according to the above method to generate corresponding upsampled tensors.

[0161] Therefore, the pooling module in this technical solution reduces the number of channels through a 1×1 convolution kernel, and uses a more concise and efficient addition operation to add the feature elements of multiple feature maps, making the pooling module more efficient and ensuring the real-time performance of the model.

[0162] Optionally, the third feature tensor is input into the decoding network, and the operation of generating a second feature tensor corresponding to the third feature tensor through the decoding operation of at least one decoding module in the decoding network includes: receiving the input first tensor to be decoded through the decoding module of the decoding network; performing a deconvolution operation on the first tensor to be decoded through the decoding module using at least one deconvolution layer to generate a decoding tensor corresponding to the first tensor to be decoded as a decoding result; inputting the decoding tensor as the first tensor to be fused into the unified attention fusion module; inputting the first tensor to be encoded feature output by the corresponding encoding module as the second tensor to be fused into the unified attention fusion module; fusing the first tensor to be fused and the second tensor to be fused through the unified attention fusion module to generate a fused tensor; and inputting the fused tensor as the second tensor to be decoded into a decoding module that is shallower than the unified attention fusion module.

[0163] Specifically, refer to Figure 3A As shown, the decoding network includes multiple decoding modules. For example, decoding module 1 receives the first tensor to be decoded. Decoding module 1 is the deepest decoding module in the decoding network, and the first tensor to be decoded received by the deepest decoding module is the third feature tensor output by the pooling module.

[0164] Furthermore, the decoding module 1 includes multiple deconvolution layers (such as Figure 3A The decoding module 1 inputs the first tensor to be decoded into the deconvolution unit, and the deconvolution unit performs a deconvolution operation on the first tensor to be decoded. Then, the decoding module 1 outputs a feature map 301 (i.e., a decoding tensor) corresponding to the first tensor to be decoded.

[0165] Furthermore, the surface defect detection module inputs the feature map 301 (ie, the decoded tensor) as the first tensor to be fused into the unified attention fusion module 1.

[0166] The encoding module 1 corresponds to the unified attention fusion module 1, so that the surface defect detection module inputs the first tensor to be encoded output by the encoding module 1 into the unified attention fusion module 1 as the second tensor to be fused.

[0167] Furthermore, the unified attention fusion module 1 calculates the weights corresponding to the first tensor to be fused and the second tensor to be fused respectively through the attention module, and merges the first tensor to be fused and the second tensor to be fused and the corresponding weights after calculating, thereby generating a feature map 302 (i.e., a fused tensor).

[0168] Afterwards, the surface defect detection module sends the feature map 302 (ie, the fusion tensor) to a decoding module at a shallower level than the unified attention fusion module. The decoding module at a shallower level than the unified attention fusion module 1 is the decoding module 2.

[0169] Thus, the surface defect detection module sends the feature map 302 (i.e., fused tensor) output by the unified attention module 1 as the second tensor to be decoded to the decoding module 2. In this case, relative to the decoding module 2, the feature map 302 is the first tensor to be decoded, and similar operations described above are continued. The decoding module 2 includes multiple deconvolution units. The decoding module 2 receives the second tensor to be decoded and inputs it into the deconvolution unit. The deconvolution unit performs a deconvolution operation on the second tensor to be decoded, and then the decoding module 2 outputs a feature map 303 corresponding to the second tensor to be decoded.

[0170] Therefore, this technical solution strengthens the feature representation of the feature map by adding a unified attention fusion module in the decoding module, and screens important features so that only important features can be calculated, reducing the amount of calculation and improving the decoding speed.

[0171] Therefore, in the decoding network, the process of taking the third feature tensor as input and generating the second feature tensor corresponding to the third feature tensor through the decoding operation of at least one decoding module includes a series of complex operations.

[0172] First, the decoding module receives an input first tensor to be decoded, and then uses at least one deconvolution layer to perform a deconvolution operation on the first tensor to be decoded. This process is called a decoding operation, and the purpose is to convert the first tensor to be decoded into a decoding tensor.

[0173] Then, the decoded tensor is input into the unified attention fusion module as the first tensor to be fused. At the same time, the first tensor feature to be encoded output by the corresponding encoding module is also input into the unified attention fusion module as the second tensor to be fused.

[0174] In the unified attention fusion module, the first tensor to be fused and the second tensor to be fused are fused through a specific fusion algorithm to generate a fused tensor. This step is a key link in the decoding process, because through the attention mechanism, the decoding network can selectively focus on the output of the encoding network, thereby introducing richer context information in the decoding process.

[0175] Finally, the fused tensor is input as the second tensor to be decoded into a decoding module that is shallower than the unified attention fusion module. This step further decodes the fused tensor to generate the final second feature tensor.

[0176] This decoding process allows the model to gradually introduce more contextual information during the decoding process when processing sequence data or image data, thereby generating more accurate and meaningful outputs.

[0177] Optionally, the operation of fusing the first tensor to be fused and the second tensor to be fused through a unified attention fusion module to generate a fused tensor includes: performing an upsampling operation on the first tensor to be fused through an upsampling unit of the unified attention fusion module to generate a third tensor to be fused, wherein the third tensor to be fused has the same size as the second tensor to be fused; inputting the second tensor to be fused and the third tensor to be fused into the attention unit respectively, and calculating the first weight corresponding to the second tensor to be fused and the second weight corresponding to the third tensor to be fused through the attention unit respectively; calculating the first attention weighted feature according to the second tensor to be fused and the first weight; calculating the second attention weighted feature according to the third tensor to be fused and the second weight; and fusing the first attention weighted feature and the second attention weighted feature to generate a fused tensor.

[0178] Specifically, refer to Figure 8 As shown, the unified attention fusion module includes an upsampling unit and an attention unit.

[0179] For example, the unified attention fusion module 1 receives the first tensor to be fused F output by the decoding module 1 respectively high , and the second tensor to be fused output by the convolution unit of the encoding module 1, and then the first tensor to be decoded F high Input upsampling unit, upsampling unit through bilinear interpolation operation, the first to be decoded tensor F high Upsample to the same size as the second tensor to be decoded, thereby generating the third tensor to be fused F up The third tensor to be fused has the same size as the second tensor to be fused. up The expression is:

[0180] F up =Upsample(Fhigh ).

[0181] Furthermore, the surface defect detection module converts the third tensor to be fused F up and the second tensor to be fused F low are input to the attention unit respectively, so that the attention unit calculates and the second tensor to be fused F low The corresponding first weight and second tensor to be fused F low The corresponding second weight. The sum of the first weight and the second weight is 1, so the first weight is expressed as r, and the second weight is expressed as 1-r. The method for the attention unit to calculate the first weight and the second weight can be, for example, through a spatial attention mechanism or a channel attention mechanism. The expression for calculating the first weight r is:

[0182] r = Attention(F up ,F low ).

[0183] Furthermore, the surface defect detection module multiplies the tensor elements in the second tensor to be fused by the first weight respectively, so as to obtain the first attention weighted feature.

[0184] Furthermore, the surface defect detection module multiplies the tensor elements of the third tensor to be fused by the second weights respectively, thereby obtaining a second attention weighted feature.

[0185] Furthermore, the surface defect detection module adds the tensor elements of the first attention weighted feature and the tensor elements at the corresponding positions of the second attention weighted feature to generate a fused tensor F out The formula for calculating the fused tensor is:

[0186] F out =F up ·r+F low ·(1-r).

[0187] Therefore, this technical solution strengthens the feature representation of the feature map through a unified attention fusion module, and screens important features, so that only important features can be calculated, reducing the amount of calculation and improving the decoding speed.

[0188] In addition, according to a second aspect of this embodiment, a surface defect management method for industrial materials is provided, the method comprising: Figure 1 Implementation of a surface defect management system for industrial materials as shown in . Fig. 9 A schematic diagram showing the process of the method is shown in FIG. Fig. 9 As shown, the method includes:

[0189] S902: Sending instruction information for controlling the detection control module to the detection control module through the management platform module;

[0190] S904: Collecting detection information related to industrial materials through the synchronization card management module, and sending the detection information to the detection control module;

[0191] S906: Sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information through the detection control module;

[0192] S908: executing corresponding operations according to the image acquisition control instruction through the image acquisition module; and

[0193] S910: Segmenting the surface image according to the positions of the surface defects corresponding to the collected surface image by the surface defect detection module to generate a segmented image.

[0194] In addition, reference Figure 1 As shown, according to the third aspect of this embodiment, a storage medium is provided. The storage medium includes a stored program, wherein when the program is run, the processor executes the above method.

[0195] Therefore, according to this embodiment, a surface defect management system for industrial materials is constructed, and thus this technical solution manages the surface defects of industrial materials through a complete surface defect management system for industrial materials, thereby making the management of surface defects simpler, reducing the waste of manpower and material resources, and making the management more efficient. And the surface defect management system for industrial materials in this technical solution includes at least a management platform module, a detection control module, an image acquisition module, a surface defect detection module, and a synchronization card management module, so that corresponding functional modules are constructed according to different functions, so that the coupling of the surface defect management system for industrial materials is low, so that during later maintenance, the modification of a certain functional module will not affect other functional modules. This makes later maintenance more convenient, improves the simplicity of maintenance, and makes it more efficient and less limited.

[0196] It should be noted that, for the above-mentioned method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present invention.

[0197] Through the description of the above implementation methods, those skilled in the art can clearly understand that the method according to the above embodiment can be implemented by means of software plus a necessary general hardware platform, and of course can also be implemented by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, a magnetic disk, or an optical disk), and includes a number of instructions for enabling a terminal device (which can be a mobile phone, a computer, a server, or a network device, etc.) to execute the methods described in each embodiment of the present invention.

[0198] Example 2

[0199] Fig.10 The surface defect management device 1000 of industrial materials according to the first aspect of this embodiment is shown, and the device 1000 corresponds to the method according to the first aspect of embodiment 1. Fig.10 As shown, the device 1000 includes: a first sending module 1010, which is used to send instruction information for controlling the detection control module to the detection control module through the management platform module; an information acquisition module 1020, which is used to collect detection information related to industrial materials through the synchronization card management module, and send the detection information to the detection control module; a second sending module 1030, which is used to send image acquisition control instructions to the image acquisition module according to production information and detection information through the detection control module; an instruction execution module 1040, which performs corresponding operations according to the image acquisition control instructions through the image acquisition module; and an image generation module 1050, which is used to segment the surface image according to the position of the surface defect corresponding to the acquired surface image through the surface defect detection module to generate a segmented image.

[0200] Therefore, according to this embodiment, a surface defect management system for industrial materials is constructed, and thus this technical solution manages the surface defects of industrial materials through a complete surface defect management system for industrial materials, thereby making the management of surface defects simpler, reducing the waste of manpower and material resources, and making the management more efficient. And the surface defect management system for industrial materials in this technical solution includes at least a management platform module, a detection control module, an image acquisition module, a surface defect detection module, and a synchronization card management module, so that corresponding functional modules are constructed according to different functions, so that the coupling of the surface defect management system for industrial materials is low, so that during later maintenance, the modification of a certain functional module will not affect other functional modules. This makes later maintenance more convenient, improves the simplicity of maintenance, and makes it more efficient and less limited.

[0201] Example 3

[0202] Fig.11 The surface defect management device 1100 of industrial materials according to the first aspect of this embodiment is shown, and the device 1100 corresponds to the method according to the first aspect of embodiment 1. Fig.11 As shown, the device 1100 includes: a processor 1110; and a memory 1120, which is connected to the processor 1110 and is used to provide the processor 1110 with instructions for processing the following processing steps: sending instruction information for controlling the detection control module to the detection control module through the management platform module; collecting detection information related to industrial materials through the synchronization card management module, and sending the detection information to the detection control module; sending image acquisition control instructions to the image acquisition module according to production information and detection information through the detection control module; executing corresponding operations according to the image acquisition control instructions through the image acquisition module; and segmenting the surface image according to the position of the surface defects corresponding to the collected surface image through the surface defect detection module to generate a segmented image.

[0203] Therefore, according to this embodiment, a surface defect management system for industrial materials is constructed, and thus this technical solution manages the surface defects of industrial materials through a complete surface defect management system for industrial materials, thereby making the management of surface defects simpler, reducing the waste of manpower and material resources, and making the management more efficient. And the surface defect management system for industrial materials in this technical solution includes at least a management platform module, a detection control module, an image acquisition module, a surface defect detection module, and a synchronization card management module, so that corresponding functional modules are constructed according to different functions, so that the coupling of the surface defect management system for industrial materials is low, so that during later maintenance, the modification of a certain functional module will not affect other functional modules. This makes later maintenance more convenient, improves the simplicity of maintenance, and makes it more efficient and less limited.

[0204] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0205] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0206] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0207] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0208] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0209] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, RandomAccess Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0210] The above is only a preferred embodiment of the present invention. It should be pointed out that for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principle of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A surface defect management system for industrial materials, It is characterized in that include: Management platform module, detection control module, image acquisition module, surface defect detection module and synchronization card management module, among which The management platform module is configured to perform the following operations: sending instruction information for controlling the detection control module to the detection control module; The synchronization card management module is configured to perform the following operations: collect detection information related to the industrial material, and send the detection information to the detection control module; The detection control module is configured to perform the following operations: sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information; The image acquisition module is configured to perform the following operations: perform corresponding operations according to the image acquisition control instruction; and The surface defect detection module is configured to perform the following operations: segmenting the surface image according to the position of the surface defect corresponding to the acquired surface image to generate a segmented image, wherein The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information includes: generating instruction information for starting image acquisition when the production status is in production and the detection length in the detection information is greater than or equal to a first preset threshold; And the instruction information for starting image acquisition is sent to the image acquisition module.

2. The system according to claim 1, It is characterized in that The surface defect detection module is also configured to perform the following operations: The segmented image is sent to a storage module for storage.

3. The system according to claim 1, It is characterized in that The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information also includes: When the production status is paused or waiting to stop, and the production line speed in the production information is greater than a second preset threshold, generating instruction information for continuing image acquisition; and The instruction information for continuing image acquisition is sent to the image acquisition module.

4. The system according to claim 1, It is characterized in that The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information also includes: When the production status is under detection and the production line speed in the production information is equal to a second preset threshold, generating instruction information for pausing image acquisition; and The instruction information for pausing image acquisition is sent to the image acquisition module.

5. The system according to claim 1, It is characterized in that The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information also includes: When the production state is detection, the detection length is greater than or equal to a third preset threshold, and the light sensing state in the detection information is unobstructed, generating instruction information for stopping image acquisition; and The instruction information for stopping image acquisition is sent to the image acquisition module.

6. A method for managing surface defects of industrial materials, It is characterized in that include: Sending instruction information for controlling the detection control module to the detection control module through the management platform module; Collecting detection information related to the industrial material through a synchronization card management module, and sending the detection information to the detection control module; Sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information by the detection control module; Executing corresponding operations according to the image acquisition control instruction by the image acquisition module; as well as The surface defect detection module segments the surface image according to the position of the surface defects corresponding to the collected surface image to generate a segmented image, wherein The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information includes: generating instruction information for starting image acquisition when the production status is in production and the detection length in the detection information is greater than or equal to a first preset threshold; And the instruction information for starting image acquisition is sent to the image acquisition module.

7. A storage medium, It is characterized in that The storage medium includes a stored program, wherein when the program is executed, the method of claim 6 is executed by a processor.

8. A surface defect management device for industrial materials, It is characterized in that include: A first sending module, used to send instruction information for controlling the detection control module to the detection control module through the management platform module; An information collection module, used for collecting detection information related to the industrial material through a synchronization card management module, and sending the detection information to the detection control module; A second sending module, used for sending an image acquisition control instruction to the image acquisition module through the detection control module according to the production information and the detection information; an instruction execution module, which executes corresponding operations according to the image acquisition control instruction through the image acquisition module; and The image generation module is used to segment the surface image according to the position of the surface defects corresponding to the collected surface image through the surface defect detection module to generate a segmented image, wherein The second sending module includes: generating instruction information for starting image acquisition when the production status is in production and the detection length in the detection information is greater than or equal to a first preset threshold; And the instruction information for starting image acquisition is sent to the image acquisition module.

9. A method for managing surface defects of industrial materials. It is characterized in that include: processor; as well as A memory, connected to the processor, configured to provide the processor with instructions for processing the following processing steps: Sending instruction information for controlling the detection control module to the detection control module through the management platform module; Collecting detection information related to the industrial material through a synchronization card management module, and sending the detection information to the detection control module; Sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information by the detection control module; Executing corresponding operations according to the image acquisition control instruction by the image acquisition module; as well as The surface defect detection module segments the surface image according to the position of the surface defects corresponding to the collected surface image to generate a segmented image, wherein The operation of sending an image acquisition control instruction to the image acquisition module according to the production information and the detection information includes: generating instruction information for starting image acquisition when the production status is in production and the detection length in the detection information is greater than or equal to a first preset threshold; And the instruction information for starting image acquisition is sent to the image acquisition module.

Images