Steel detection method, electronic equipment and computer readable medium

By combining an image acquisition device and a transmission device with an infrared sensor and an ambient light sensor, the system automatically adjusts the supplementary lighting and the camera to acquire images, performing image stitching and defect detection. This solves the problems of high cost and low accuracy in steel surface defect detection in existing technologies, achieving low-cost and high-efficiency detection results.

CN118914217AActive Publication Date: 2024-11-08CHINA CONSTR SEVENTH ENG DIVISION CORP LTD

Patent Information

Application Number
CN202411033389.X
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-30
Publication Date
2024-11-08
Estimated Expiration
2044-07-30

AI Technical Summary

Technical Problem

Existing methods for detecting defects on steel surfaces suffer from high costs and low accuracy in manual inspection, while laser scanning inspection is costly and difficult to promote.

Method used

Using an image acquisition device and a steel conveying device, combined with an infrared sensor and an ambient light sensor, the brightness of the supplementary light and the image acquired by the camera are automatically adjusted. Defect detection is performed through image stitching and a steel surface defect detection model.

Benefits of technology

It enables high-accuracy steel surface defect detection at low cost, reduces the risks of manual handling and image acquisition, and improves detection efficiency and accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118914217A_ABST
    Figure CN118914217A_ABST
Patent Text Reader

Abstract

The embodiment of the invention discloses a steel detection method, electronic equipment and a computer readable medium. A specific embodiment of the method comprises the following steps: in response to infrared shielding detected by a first infrared sensor and a main conveying track in a steel conveying state, determining a current ambient light state through an ambient light sensor; in response to a weak ambient light state represented by the current ambient light state, controlling the light supplement lamp to be turned on, and adjusting the brightness of the light supplement lamp according to the current ambient light state; controlling a first camera, a second camera and a third camera to carry out image acquisition in response to the condition that the adjustment is finished or the current ambient light state represents a strong ambient light state and a second infrared sensor detects infrared shielding; performing image splicing on each target image contained in the target image group to generate a spliced image; and for the spliced image, defect detection is carried out on the spliced image through the steel surface defect detection model. According to the embodiment, high-accuracy steel surface defect detection is realized.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] Embodiments of the present disclosure relate to the field of computer technology, and in particular to a steel material detection method, an electronic device, and a computer-readable medium. Background Art

[0002] As an important basic material, steel plays a vital role in the development of the national economy. Steel is widely used in various fields, including but not limited to: construction, manufacturing, transportation, energy, etc. Especially in the construction industry, as the height of the building increases, the amount of steel required increases greatly. The surface defects of steel produced during the production process, such as inclusions, plaques, pitting, etc., affect the quality of steel and also affect the quality of the building. At present, when detecting surface defects of steel, methods such as manual inspection and laser scanning inspection are usually used. However, the manual inspection method has a high labor cost, and manual inspection is affected by differences in subjective judgment, which can easily lead to low detection accuracy and efficiency; while the laser scanning detection method has a high real-time cost, so it is difficult to promote its use. Summary of the invention

[0003] The content of this disclosure is used to introduce concepts in a brief form, which will be described in detail in the detailed implementation section below. The content of this disclosure is not intended to identify the key features or essential features of the technical solution claimed for protection, nor is it intended to limit the scope of the technical solution claimed for protection.

[0004] Some embodiments of the present disclosure provide a steel material detection method, an electronic device, and a computer-readable medium to solve one or more of the technical problems mentioned in the above background technology section.

[0005] In a first aspect, some embodiments of the present disclosure provide a steel inspection method, which is applied to a steel surface defect inspection device, wherein the steel surface defect inspection device comprises: an image acquisition device and a steel conveying device, the image acquisition device comprises: a first camera, a second camera, a third camera, a fill light and an ambient light sensor, the steel conveying device comprises: a main conveying track, a first infrared sensor, a second infrared sensor and a third infrared sensor, the second infrared sensor is arranged before the image acquisition device, and the method comprises: in response to the first infrared sensor detecting infrared shading and the main conveying track being in a steel conveying state, determining a current ambient light state through the ambient light sensor; in response to the current ambient light state representing a weak ambient light state, controlling the fill light to turn on, and according to the current The method comprises the steps of: adjusting the brightness of the fill light according to the ambient light state; in response to the adjustment being completed or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, controlling the first camera, the second camera and the third camera to acquire images to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera and / or the fill light to be turned off; performing image stitching on each target image group in the target image group set to generate a stitched image to obtain a stitched image set; for each stitched image in the stitched image set, performing defect detection on the stitched image through a steel surface defect detection model to generate detection information.

[0006] In a second aspect, some embodiments of the present disclosure provide an electronic device comprising: one or more processors; a storage device on which one or more programs are stored, and when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation manner of the above-mentioned first aspect.

[0007] In a third aspect, some embodiments of the present disclosure provide a computer-readable medium having a computer program stored thereon, wherein when the program is executed by a processor, the method described in any implementation manner of the above-mentioned first aspect is implemented.

[0008] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the steel material detection method of some embodiments of the present disclosure, high-accuracy steel material surface defect detection is achieved under the premise of low implementation cost. Specifically, the present disclosure realizes the movement and image acquisition of steel materials to be subjected to steel material surface defect detection through a steel material surface defect detection device. In practice, the weight of construction steel materials is large, and the handling cost is high if they are transported manually, and the danger is high during the image acquisition process. Therefore, the present disclosure realizes the movement and image acquisition of steel materials through a steel material surface defect detection device, and realizes automated movement and image acquisition. The specific analysis is as follows: First, in response to the above-mentioned first infrared sensor detecting infrared shielding and the above-mentioned main conveying track being in the steel material conveying state, the current ambient light state is determined through the above-mentioned ambient light sensor. Secondly, in response to the above-mentioned current ambient light state representing a weak ambient light state, the above-mentioned fill light is controlled to be turned on, and according to the above-mentioned current ambient light state, the brightness of the above-mentioned fill light is adjusted to collect the current ambient light state through the ambient light sensor, so as to determine whether to turn on the fill light for fill light, so as to avoid the problem of poor image quality obtained in a dark light environment. Then, in response to the adjustment being completed or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the first camera, the second camera and the third camera are controlled to perform image acquisition to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera and / or the fill light to turn off. Further, each target image contained in each target image group in the target image group set is image stitched to generate a stitched image to obtain a stitched image set. Through image stitching, the images collected by the first camera, the second camera and the third camera at different image acquisition positions at the same time are stitched. Finally, for each stitched image in the stitched image set, the steel surface defect detection model is used to perform defect detection on the stitched image to generate detection information. That is, defect detection is performed by image recognition. In summary, the present disclosure realizes high-accuracy steel surface defect detection by combining a steel surface defect detection device and image recognition at a low implementation cost. BRIEF DESCRIPTION OF THE DRAWINGS

[0009] The above and other features, advantages and aspects of the embodiments of the present disclosure will become more apparent with reference to the following detailed description in conjunction with the accompanying drawings. Throughout the accompanying drawings, the same or similar reference numerals represent the same or similar elements. It should be understood that the drawings are schematic and that components and elements are not necessarily drawn to scale.

[0010] Figure 1 is a flow chart of some embodiments of the steel material detection method according to the present disclosure;

[0011] Figure 2 It is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. DETAILED DESCRIPTION

[0012] Embodiments of the present disclosure will be described in more detail below with reference to the accompanying drawings. Although certain embodiments of the present disclosure are shown in the accompanying drawings, it should be understood that the present disclosure can be implemented in various forms and should not be construed as being limited to the embodiments set forth herein. On the contrary, these embodiments are provided to provide a more thorough and complete understanding of the present disclosure. It should be understood that the drawings and embodiments of the present disclosure are only for exemplary purposes and are not intended to limit the scope of protection of the present disclosure.

[0013] It should also be noted that, for ease of description, only the parts related to the invention are shown in the drawings. In the absence of conflict, the embodiments and features in the embodiments of the present disclosure can be combined with each other.

[0014] It should be noted that the concepts such as "first" and "second" mentioned in the present disclosure are only used to distinguish different devices, modules or units, and are not used to limit the order or interdependence of the functions performed by these devices, modules or units.

[0015] It should be noted that the modifications of "one" and "plurality" mentioned in the present disclosure are illustrative rather than restrictive, and those skilled in the art should understand that unless otherwise clearly indicated in the context, it should be understood as "one or more".

[0016] The names of the messages or information exchanged between multiple devices in the embodiments of the present disclosure are only used for illustrative purposes and are not used to limit the scope of these messages or information.

[0017] The present disclosure will be described in detail below with reference to the accompanying drawings and in conjunction with embodiments.

[0018] refer to Figure 1 , shows a process 100 of some embodiments of the steel material detection method according to the present disclosure. The steel material detection method comprises the following steps:

[0019] Step 101, in response to the first infrared sensor detecting infrared shielding and the main conveying track being in a steel conveying state, the current ambient light state is determined by the ambient light sensor.

[0020] In some embodiments, the execution subject (e.g., a computing device) of the steel material detection method may determine the current ambient light state through an ambient light sensor in response to the first infrared sensor detecting infrared blocking and the main conveying track being in a steel material conveying state. The above steel material detection method is applied to a steel material surface defect detection device. The above steel material surface defect detection device includes: an image acquisition device and a steel material conveying device. The above image acquisition device includes: a first camera, a second camera, a third camera, a fill light and an ambient light sensor. The above steel material conveying device includes: a main conveying track, a first infrared sensor, a second infrared sensor and a third infrared sensor. The above second infrared sensor is arranged before the above image acquisition device. The above third infrared sensor is arranged at the end of the above main conveying track. The above first infrared sensor is arranged before the above second infrared sensor and at the front end of the above main conveying track. The main conveying track may include a plurality of conveying track shafts. When the main conveying track is opened, the conveying track shafts will rotate accordingly, thereby driving the objects on the main conveying track to move. The above ambient light sensor may be arranged on both sides of the main conveying track, facing the inside of the main conveying track, to detect the ambient light state inside the main conveying track. Specifically, the execution subject can map the ambient light state according to the signal value detected by the ambient light sensor as the current ambient light state. Since the current ambient light state is the basis for determining whether to turn on the fill light brightness, the ambient light state can include: a weak ambient light state and a strong ambient light state.

[0021] It should be noted that the computing device 101 can be hardware or software. When the computing device is hardware, it can be implemented as a single terminal device. When the computing device is embodied as software, it can be installed in the hardware devices listed above. Specifically, the steel surface defect detection device can also include a microcomputer. Among them, the microcomputer can be used to schedule and control the steel conveying device, as well as to perform image stitching and defect detection. Therefore, the execution subject can be the microcomputer.

[0022] Optionally, the steel conveying device further includes: an auxiliary conveying track and a dragging device. The track angle of the auxiliary conveying track is adjustable. The auxiliary conveying track is arranged before the main conveying track. The auxiliary conveying track is connected to the main conveying track. The dragging device includes: a magnetic suction cup, a dragging cable and an electric winch. The dragging device is arranged above the main conveying track. The dragging cable is connected to the electric winch and the magnetic suction cup respectively. The dragging cable is used to drag the steel to be inspected for surface defects of the steel. The dragging cable is retracted and released by the electric winch. The first infrared sensor is arranged at the junction of the main conveying track and the auxiliary conveying track. Specifically, the auxiliary conveying track can rotate counterclockwise with the junction with the main conveying track as the axis to adjust the track angle. A coil can be arranged in the magnetic suction cup, which will generate magnetic force when powered on, so as to control the magnetic suction cup to be adsorbed on the surface of the steel. The first infrared sensor can detect whether there is steel moving into the main conveying track.

[0023] In some optional implementations of some embodiments, in response to the first infrared sensor detecting infrared shielding and the main conveying track being in a steel conveying state, before determining the current ambient light state through the ambient light sensor, the method further includes:

[0024] The first step is to drive the auxiliary transmission track to change the track angle.

[0025] In practice, when the steel surface defect detection device is turned on, the above-mentioned execution body can control the motor to drive the auxiliary conveying track to rotate counterclockwise with the joint with the main conveying track as the axis to achieve track angle adjustment.

[0026] In the second step, in response to the auxiliary transfer track touching the ground or the track angle of the auxiliary transfer track being greater than or equal to a preset track angle, the track angle change for the auxiliary transfer track is stopped.

[0027] In practice, the execution subject can detect the torque of the motor to determine whether the auxiliary transmission track touches the ground. In addition, there may be uneven ground. In this case, only judging whether the auxiliary transmission track touches the ground to stop the track angle change for the auxiliary transmission track may cause the auxiliary transmission track to change too much. Therefore, the restriction of track angle change with preset track angle is added.

[0028] The third step is to wind up the towing cable by the electric winch in response to the electric winch being turned on.

[0029] In practice, when the magnetic suction cup is powered on and the electric winch is turned on, it indicates that the steel has been fixed. Therefore, the above-mentioned execution body can control the electric winch to reel in the towing cable to achieve the purpose of dragging the steel onto the secondary transmission track.

[0030] The fourth step is to start the main transport track in response to the infrared occlusion detected by the first infrared sensor.

[0031] In practice, when the first infrared sensor detects infrared shielding, it indicates that the steel has been transported to the junction of the main conveying track and the auxiliary conveying track, and the main conveying track can be started at this time. By controlling the opening of the main conveying track through the first infrared sensor, the idling of the main conveying track can be avoided, and unnecessary power consumption can be avoided.

[0032] The fifth step is to release the magnetic force of the magnetic suction cup in response to the preset duration of starting the main transmission track, and to tighten the towing cable through the electric winch.

[0033] In practice, since the towing device is arranged above the above-mentioned main transmission track, when the steel passes the towing device, the towing cable restricts the movement of the steel. In addition, if the magnetic suction cup is immediately released magnetically when the first infrared sensor detects infrared obstruction, it may cause the steel to be stuck on the auxiliary transmission track and unable to enter the main transmission track normally. Therefore, the preset time length set in the present invention is to ensure that the steel enters the main transmission track normally. Specifically, since the distance between the junction of the main transmission track and the auxiliary transmission track and the towing device is fixed, when the transmission speed of the main transmission track changes, the preset time length may also change accordingly. For example, when the transmission speed increases, the preset time length is shortened. When the transmission speed decreases, the preset time length increases. In addition, the above-mentioned towing cable is tightened by the above-mentioned electric winch to avoid possible wear of the towing cable and the magnetic suction cup on the main transmission track.

[0034] Step 102, in response to the current ambient light state indicating a weak ambient light state, controlling the fill light to turn on, and adjusting the brightness of the fill light according to the current ambient light state.

[0035] In some embodiments, the execution subject may control the fill light to turn on in response to the current ambient light state characterizing a weak ambient light state, and adjust the fill light brightness according to the current ambient light state. In practice, the execution subject may map the current ambient light state to a fill light level. For example, when the fill light level is 1-5, it corresponds to a strong ambient light state, and when the fill light level is 6-10, it corresponds to a weak ambient light state. Therefore, when the current ambient light state characterizes a weak ambient light state, the execution subject may adjust the fill light to a fill light brightness that matches the fill light level according to the fill light level corresponding to the current ambient light state. In practice, for example, the fill light may be respectively arranged on the lower side of the main conveying track and the upper side of the main conveying track to realize fill light on the upper and lower sides of the steel conveyed by the main conveying track. For another example, in order to realize full fill light on the surface of the steel, the image acquisition device may include: 3 fill lights, such as: fill light A, fill light B and fill light C. Among them, fill light A is arranged on the directly lower side of the main conveying track. Fill light B and fill light C are arranged on the upper side of the main conveying track. Fill light A, fill light B and fill light C form an angle of 120 degrees. Fill light A, fill light B and fill light C are arranged on the same plane, and the normal vector of the plane where fill light A, fill light B and fill light C are located is parallel to the running direction of the above-mentioned main conveying track.

[0036] Step 103 , in response to the adjustment being completed or the current ambient light state indicating a strong ambient light state, and the second infrared sensor detecting infrared shielding, the first camera, the second camera, and the third camera are controlled to acquire images to obtain a target image group set.

[0037] In some embodiments, in response to the adjustment being completed or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the execution subject can control the first camera, the second camera, and the third camera to collect images to obtain a target image group set. Among them, each target image in the target image group corresponds to the same central axis. The third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the fill light to turn off. Specifically, since the third infrared sensor is arranged at the end of the main conveying track, when the third infrared sensor detects infrared occlusion, it is converted to no infrared occlusion is detected, indicating that there is no steel on the main conveying track. Therefore, the first camera, the second camera, and the third camera can be controlled to be turned off. By controlling the camera to turn off, first, unnecessary power consumption can be reduced. Second, it can avoid the camera collecting images that do not contain the surface of the steel, thereby eliminating the need for image screening, reducing processing steps, and reducing the computing pressure of the microcomputer.

[0038] In practice, the first camera, the second camera and the third camera are all facing the center of the plane. Specifically, the execution subject can control the acquisition timing synchronization of the first camera, the second camera and the third camera through a clock synchronizer.

[0039] Optionally, the main conveying track includes: an image acquisition area. Specifically, the image acquisition area may be an area in the middle of the main conveying track that does not include the conveying track axis. By setting the image acquisition area, the camera is avoided from being blocked by the conveying track axis, thereby improving the imaging quality. The first camera is arranged directly below the image acquisition area. The second camera and the third camera are arranged above the image acquisition area. The angles between the first camera, the second camera and the third camera are all 120 degrees.

[0040] In some optional implementations of some embodiments, the execution subject controls the first camera, the second camera, and the third camera to collect images to obtain a target image group set, which may include the following steps:

[0041] The first step is to determine the current track speed of the main transmission track.

[0042] In practice, for example, since the transmission track shaft in the main transmission track is driven by a motor, the current track speed can be obtained by mapping according to the real-time rotation speed of the motor. For another example, the rotation speed of the transmission track shaft can be detected by a speedometer as the current track speed. For another example, in order to improve the accuracy of the current track speed obtained, the rotation speeds of different transmission track shafts in the main transmission track can be detected by multiple speedometers, and the average value can be taken as the current track speed.

[0043] The second step is to synchronize the focal lengths of the first camera, the second camera, and the third camera.

[0044] In practice, first, the execution subject may send a current focal length confirmation request to the first camera, the second camera, and the third camera by broadcasting. Then, when the current focal lengths of the first camera, the second camera, and the third camera are inconsistent, a camera focal length adjustment request is sent to the second camera and the third camera based on the current focal length of the first camera.

[0045] The third step is to determine the current camera focal length in response to successful synchronization.

[0046] The current camera focal length may be the current focal length of the first camera.

[0047] The fourth step is to determine the image acquisition time interval according to the current camera focal length and the current track speed.

[0048] In practice, due to the size limitation of the camera's photosensitive element, the size of the image captured by the camera is limited. Image acquisition with a lower acquisition density may result in missed acquisitions, while image acquisition with a higher density may result in a higher degree of overlap between images. Specifically, it can be simplified to a fixed image size. Therefore, under the premise of pre-calibration, there is a mapping relationship between the image coordinate system where the image captured by the camera is located and the geodetic coordinate system where the steel surface defect detection device is located. Therefore, the area corresponding to the image in the geodetic coordinate system can be determined. When the current track speed is known, the shooting interval between every two images can be determined by the speed formula as the acquisition time interval.

[0049] The fifth step is to control the first camera, the second camera and the third camera to perform synchronous image acquisition at the image acquisition time interval as the time interval to generate a target image group and obtain the target image group set.

[0050] In practice, for example, the execution subject can send a time interval to the first camera, the second camera, and the third camera so that the first camera, the second camera, and the third camera automatically perform image acquisition. Since the time interval and the acquisition start time are the same, the purpose of synchronous image acquisition can be achieved, that is, one-time control to achieve synchronous image acquisition. For another example, the execution subject can use the image acquisition time interval as the time interval to control the first camera, the second camera, and the third camera to perform synchronous image acquisition through timed broadcasting, that is, multiple controls to achieve synchronous image acquisition, which has higher control accuracy than the one-time control method.

[0051] Step 104 , performing image stitching on each target image contained in each target image group in the target image group set to generate a stitched image, thereby obtaining a stitched image set.

[0052] In some embodiments, the execution subject may stitch the target images contained in each target image group in the target image group set to generate a stitched image and obtain a stitched image set. In practice, since the target images in the target image group are images of the same steel area captured by different cameras at different viewing angles, the execution subject may stitch the images in sequence in a clockwise order to obtain a stitched image.

[0053] In some optional implementations of some embodiments, the execution subject performs image stitching on each target image included in each target image group in the target image group set to generate a stitched image, which may include the following steps:

[0054] In the first step, for each of the above target images, the following processing steps are performed:

[0055] In the first sub-step, feature points are extracted from the edge region of the first image and the edge region of the second image respectively to generate a first feature point group and a second feature point group.

[0056] Among them, the first image edge area and the second image edge area are symmetrical image edge areas in the target image. In practice, the target image is defined in a clockwise direction, the left side can be defined as side A, the upper side can be defined as side B, the right side is defined as side C, and the lower side is defined as side D. Since image stitching only involves two sides, specifically, the target image collected by the first camera involves side A and side C, that is, side A is used as the first image edge area, and side C is used as the second image edge area. The target image collected by the second camera involves side B and side D, that is, side B is used as the first image edge, and side D is used as the second image edge. In practice, the above-mentioned execution subject can extract feature points through the SIFT algorithm. Specifically, at the same time, there is an image overlap area between images collected by different cameras. The simple method of image stitching based on the image edge will result in repeated detection of repeated image content in the subsequent process. In addition, image stitching only involves two sides of the image, and the other two sides do not involve stitching. Therefore, if feature point extraction is performed on the entire image, it will cause a waste of computing resources.

[0057] The second sub-step is to perform regional grid division on the first image edge region and the second image edge region respectively to obtain the divided first image edge region and the divided second image edge region.

[0058] In practice, the principle of voxelization may be referred to to perform regional grid division on the edge region of the first planar image and the edge region of the second planar image.

[0059] The third sub-step is to fuse the first feature points in the first feature point group that are located in the same area grid within the edge area of ​​the first image after the above division, obtain the fused first feature points as the third feature points, and obtain the third feature point group.

[0060] In practice, the execution entity may average at least one first feature point located in the same regional grid to obtain a third feature point.

[0061] The fourth sub-step is to fuse the second feature points in the second feature point group that are located in the same area grid and in the edge area of ​​the second image after the above division, obtain the fused second feature points as the fourth feature points, and obtain the fourth feature point group.

[0062] In practice, the execution entity may average at least one second feature point located in the same regional grid to obtain a fourth feature point.

[0063] In the second step, according to the relative position relationship of the target images in the above target image group, feature point matching is performed according to the third feature point group and the fourth feature point group corresponding to the target images in the above target image group to determine the splicing positions between the target images in the above target image group.

[0064] For example, taking the target image captured by the first camera and the target image captured by the second camera as examples, the above-mentioned execution subject can perform feature point matching on the third feature point group corresponding to the target image captured by the first camera and the third feature point group corresponding to the target image captured by the second camera. In this way, the number of feature point matching can be reduced by half. Specifically, feature point matching can determine whether the feature points match by similarity calculation. Determining the splicing position by feature point matching can minimize the appearance of duplicate images, reduce repeated unnecessary detection during subsequent defect detection, and improve the utilization efficiency of computing resources.

[0065] The third step is to stitch the target images in the target image group according to the stitching positions to obtain a stitched image corresponding to the target image group.

[0066] Step 105: for each stitched image in the stitched image set, perform defect detection on the stitched image by using a steel surface defect detection model to generate detection information.

[0067] In some embodiments, for each stitched image in the stitched image set, the execution entity may perform defect detection on the stitched image through a steel surface defect detection model to generate detection information. In practice, the steel surface defect detection model may be an EfficinetDet model. The detection information may characterize the defect location and defect type of the steel.

[0068] Optionally, the steel surface defect detection model includes: a first convolution layer, a second convolution layer, a first convolution module, a third convolution layer, a first variable convolution module, a fourth convolution layer, a second variable convolution module, a fifth convolution layer, a third variable convolution module, a spatial pyramid pooling module, a bidirectional routing attention mechanism module, a second convolution module, a first bidirectional feature pyramid network, a first upsampling layer, a third convolution module, a second bidirectional feature pyramid network, a second upsampling layer, a sixth convolution layer, a third bidirectional feature pyramid network, a fourth convolution module, a seventh convolution layer, a fourth bidirectional feature pyramid network and a fifth convolution module. Among them, the first convolution module, the second convolution module, the third convolution module, the fourth convolution module and the fifth convolution module are implemented by C2f module. The spatial pyramid pooling module is implemented by SPPF module. The first variable convolution module, the second variable convolution module and the third variable convolution module are implemented by C2f_DCNv2 module. The bidirectional routing attention mechanism module adopts BiFormer module. The first bidirectional feature pyramid network, the second bidirectional feature pyramid network, the third bidirectional feature pyramid network, and the fourth bidirectional feature pyramid network use the BiFPN_Concat2 module. In addition, the steel surface defect detection model also includes three detection heads, namely, detection head A, detection head B, and detection head C. The input size of detection head A is 80×80×256. The input size of detection head C is 40×40×512. The input size of detection head C is 20×20×512. The input size of the steel surface defect detection model is 640×640×3. By introducing the C2f_DCNv2 module on the basis of the C2f module, the edges and complex shapes of the target objects can be captured more effectively. In addition, by introducing the BiFPN_Concat2 module, compared with the traditional PAFPN module, without increasing the computational cost, it can more effectively integrate the features, especially the features between the levels. At the same time, BiFPN adopts the formaldehyde fusion method, and increases the model accuracy and generalization ability through the process of repeated weighted fusion. Furthermore, in order to improve the detection accuracy of small targets, the BiFormer module is introduced to achieve efficient small target feature extraction based on the attention mechanism. In addition, the steel surface defect detection model uses the Wise-IoUv3 (CIoU) loss function, which can effectively solve the overfitting problem of low-quality target bounding boxes.

[0069] Optionally, the detection information in the detection information set includes: first detection information, second detection information, and third detection information. The first detection information, the second detection information, and the third detection information correspond to detection results of different sizes. Specifically, the first detection information is generated by detection head A. The second detection information is generated by detection head B. The third detection information is generated by detection head C.

[0070] In some optional implementations of some embodiments, the execution subject performs defect detection on the stitched image through a steel surface defect detection model to generate detection information, which may include the following steps:

[0071] The first step is to input the above stitched image into the above first convolutional layer to obtain the first feature.

[0072] The second step is to input the first feature into the second convolutional layer to obtain the second feature.

[0073] The third step is to input the second feature into the first convolution module to obtain the third feature.

[0074] The fourth step is to input the third feature into the three convolutional layers to obtain the fourth feature.

[0075] The fifth step is to input the fourth feature into the first variable convolution module to obtain the fifth feature.

[0076] In the sixth step, the fifth feature is input into the fourth convolutional layer to obtain the sixth feature.

[0077] In the seventh step, the sixth feature is input into the second variable convolution module to obtain the seventh feature.

[0078] In the eighth step, the seventh feature is input into the fifth convolutional layer to obtain the eighth feature.

[0079] In the ninth step, the eighth feature is input into the third variable convolution module to obtain the ninth feature.

[0080] In the tenth step, the ninth feature is input into the spatial pyramid pooling module to obtain the tenth feature.

[0081] In the eleventh step, the tenth feature is input into the bidirectional routing attention mechanism module to obtain the eleventh feature.

[0082] In the twelfth step, the eleventh feature is upsampled through the second upsampling layer to obtain the twelfth feature.

[0083] In the thirteenth step, the seventh feature and the twelfth feature are input into the second bidirectional feature pyramid network to obtain the thirteenth feature.

[0084] In the fourteenth step, the thirteenth feature is input into the third convolution module to obtain the fourteenth feature.

[0085] In the fifteenth step, the fourteenth feature is upsampled through the first upsampling layer to obtain the fifteenth feature.

[0086] Step 16: Based on the fifteenth feature, generate the first detection information included in the detection information corresponding to the stitched image.

[0087] In the seventeenth step, the fifteenth feature is input into the sixth convolutional layer to obtain the seventeenth feature.

[0088] In the eighteenth step, the seventeenth feature, the fourteenth feature and the sixth feature are input into the third bidirectional feature pyramid network to obtain the eighteenth feature.

[0089] In the nineteenth step, the eighteenth feature is input into the fourth convolution module to obtain the nineteenth feature.

[0090] Step 20: Based on the above-mentioned nineteenth feature, generate the second detection information included in the detection information corresponding to the above-mentioned stitched image.

[0091] In the 21st step, the 19th feature is input into the 7th convolutional layer to obtain the 20th feature.

[0092] In the twenty-second step, the twentieth feature and the eleventh feature are input into the fourth bidirectional feature pyramid network to obtain the twenty-first feature.

[0093] In the twenty-third step, the twenty-first feature is input into the fifth convolution module to obtain the twenty-second feature.

[0094] Step 24: Generate the third detection information included in the detection information corresponding to the stitched image based on the 22nd feature.

[0095] The above-mentioned embodiments of the present disclosure have the following beneficial effects: through the steel material detection method of some embodiments of the present disclosure, high-accuracy steel material surface defect detection is achieved under the premise of low implementation cost. Specifically, the present disclosure realizes the movement and image acquisition of steel materials to be subjected to steel material surface defect detection through a steel material surface defect detection device. In practice, the weight of construction steel materials is large, and the handling cost is high if they are transported manually, and the danger is high during the image acquisition process. Therefore, the present disclosure realizes the movement and image acquisition of steel materials through a steel material surface defect detection device, and realizes automated movement and image acquisition. The specific analysis is as follows: First, in response to the above-mentioned first infrared sensor detecting infrared shielding and the above-mentioned main conveying track being in the steel material conveying state, the current ambient light state is determined through the above-mentioned ambient light sensor. Secondly, in response to the above-mentioned current ambient light state representing a weak ambient light state, the above-mentioned fill light is controlled to be turned on, and according to the above-mentioned current ambient light state, the brightness of the above-mentioned fill light is adjusted to collect the current ambient light state through the ambient light sensor, so as to determine whether to turn on the fill light for fill light, so as to avoid the problem of poor image quality obtained in a dark light environment. Then, in response to the adjustment being completed or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the first camera, the second camera and the third camera are controlled to perform image acquisition to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera and / or the fill light to turn off. Further, each target image contained in each target image group in the target image group set is image stitched to generate a stitched image to obtain a stitched image set. Through image stitching, the images collected by the first camera, the second camera and the third camera at different image acquisition positions at the same time are stitched. Finally, for each stitched image in the stitched image set, the steel surface defect detection model is used to perform defect detection on the stitched image to generate detection information. That is, defect detection is performed by image recognition. In summary, the present disclosure realizes high-accuracy steel surface defect detection by combining a steel surface defect detection device and image recognition at a low implementation cost.

[0096] Reference below Figure 2 , which shows a structural schematic diagram of an electronic device (eg, a computing device) 200 suitable for implementing some embodiments of the present disclosure. Figure 2 The electronic device shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present disclosure.

[0097] like Figure 2As shown, the electronic device 200 may include a processing device (e.g., a central processing unit, a graphics processing unit, etc.) 201, which can perform various appropriate actions and processes according to a program stored in a read-only memory 202 or a program loaded from a storage device 208 to a random access memory 203. Various programs and data required for the operation of the electronic device 200 are also stored in the random access memory 203. The processing device 201, the read-only memory 202, and the random access memory 203 are connected to each other via a bus 204. An input / output interface 205 is also connected to the bus 204.

[0098] Typically, the following devices may be connected to the I / O interface 205: an input device 206 including, for example, a touch screen, a touch pad, a keyboard, a mouse, a camera, a microphone, an accelerometer, a gyroscope, etc.; an output device 207 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 208 including, for example, a magnetic tape, a hard disk, etc.; and a communication device 209. The communication device 209 may allow the electronic device 200 to communicate with other devices wirelessly or by wire to exchange data. Although Figure 2 The electronic device 200 is shown with various devices, but it should be understood that it is not required to implement or possess all the devices shown. More or fewer devices may be implemented or possessed instead. Figure 2 Each block shown in the figure may represent one device, or may represent multiple devices as required.

[0099] In particular, according to some embodiments of the present disclosure, the process described above with reference to the flowchart can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In some such embodiments, the computer program can be downloaded and installed from the network through the communication device 209, or installed from the storage device 208, or installed from the read-only memory 202. When the computer program is executed by the processing device 201, the above-mentioned functions defined in the method of some embodiments of the present disclosure are executed.

[0100] It should be noted that the computer-readable medium recorded in some embodiments of the present disclosure may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In some embodiments of the present disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, device or device. In some embodiments of the present disclosure, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable program code. This propagated data signal may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. The computer readable signal medium may also be any computer readable medium other than a computer readable storage medium, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer readable medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0101] In some embodiments, the client and the server may communicate using any currently known or future developed network protocol such as HTTP (Hyper Text Transfer Protocol), and may be interconnected with any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network ("LAN"), a wide area network ("WAN"), an internet (e.g., the Internet), and a peer-to-peer network (e.g., an ad hoc peer-to-peer network), as well as any currently known or future developed network.

[0102] The computer-readable medium may be included in the electronic device; or it may exist independently without being assembled into the electronic device. The computer-readable medium carries one or more programs. When the one or more programs are executed by the electronic device, the electronic device: in response to the first infrared sensor detecting infrared blocking and the main conveying track being in a steel conveying state, determines the current ambient light state through the ambient light sensor; in response to the current ambient light state representing a weak ambient light state, controls the fill light to turn on, and adjusts the brightness of the fill light according to the current ambient light state; in response to the adjustment being completed or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared blocking, controls the first The first camera, the second camera and the third camera perform image acquisition to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera and / or the fill light to turn off; image stitching is performed on each target image group in the target image group set to generate a stitched image to obtain a stitched image set; for each stitched image in the stitched image set, defect detection is performed on the stitched image through a steel surface defect detection model to generate detection information.

[0103] Computer program code for performing the operations of some embodiments of the present disclosure may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present disclosure. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some implementations as replacements, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0105] The functions described above herein may be performed at least in part by one or more hardware logic components. For example, without limitation, exemplary types of hardware logic components that may be used include: field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on chip (SOCs), complex programmable logic devices (CPLDs), and the like.

[0106] The above descriptions are only some preferred embodiments of the present disclosure and an explanation of the technical principles used. Those skilled in the art should understand that the scope of the invention involved in the embodiments of the present disclosure is not limited to the technical solutions formed by a specific combination of the above-mentioned technical features, but should also cover other technical solutions formed by any combination of the above-mentioned technical features or their equivalent features without departing from the above-mentioned inventive concept. For example, the above-mentioned features are replaced with the technical features with similar functions disclosed in the embodiments of the present disclosure (but not limited to) and the technical solutions formed.

Claims

1. A steel material detection method, applied to a steel material surface defect detection device, wherein: The steel surface defect detection device comprises: an image acquisition device and a steel conveying device, the image acquisition device comprises: a first camera, a second camera, a third camera, a fill light and an ambient light sensor, the steel conveying device comprises: a main conveying track, a first infrared sensor, a second infrared sensor and a third infrared sensor, the second infrared sensor is arranged in front of the image acquisition device, and the method comprises: In response to the first infrared sensor detecting infrared shielding and the main conveying track being in a steel conveying state, determining a current ambient light state through the ambient light sensor; In response to the current ambient light state representing a weak ambient light state, controlling the fill light to turn on, and adjusting the brightness of the fill light according to the current ambient light state; In response to the adjustment being completed or the current ambient light state indicating a strong ambient light state, and the second infrared sensor detecting infrared occlusion, controlling the first camera, the second camera, and the third camera to acquire images to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the fill light to be turned off; Performing image stitching on each target image contained in each target image group in the target image group set to generate a stitched image, thereby obtaining a stitched image set; For each stitched image in the stitched image set, defect detection is performed on the stitched image using a steel surface defect detection model to generate detection information.

2. The method according to claim 1, wherein: The steel conveying device further includes: an auxiliary conveying track and a dragging device, wherein the track angle of the auxiliary conveying track is adjustable, the auxiliary conveying track is arranged before the main conveying track, the auxiliary conveying track is connected to the main conveying track, the dragging device includes: a magnetic suction cup, a dragging cable and an electric winch, the dragging device is arranged above the main conveying track, the dragging cable is respectively connected to the electric winch and the magnetic suction cup, the dragging cable is used to drag the steel to be inspected for surface defects of the steel, the dragging cable is retracted and released by the electric winch, and the first infrared sensor is arranged at the junction of the main conveying track and the auxiliary conveying track; and Before determining the current ambient light state by the ambient light sensor in response to the first infrared sensor detecting infrared shielding and the main conveying track being in the steel conveying state, the method further includes: Drive the auxiliary transmission track to change the track angle; In response to the auxiliary conveying track touching the ground or the track angle of the auxiliary conveying track being greater than or equal to a preset track angle, stopping the track angle change for the auxiliary conveying track; In response to the electric winch being turned on, the towing cable is reeled in by the electric winch; In response to the first infrared sensor detecting infrared blocking, starting the main conveying track; In response to the main transport track being started for a preset time, the magnetic suction cup is magnetically released, and the dragging cable is tightened by the electric winch.

3. The method according to claim 2, wherein: The main conveying track includes: an image acquisition area, the first camera is arranged directly below the image acquisition area, the second camera and the third camera are arranged above the image acquisition area, and the angles between the first camera, the second camera and the third camera are all 120 degrees; and The controlling the first camera, the second camera, and the third camera to collect images to obtain a target image group set includes: Determining a current track speed of the main transport track; Synchronizing the camera focal lengths of the first camera, the second camera, and the third camera; In response to the synchronization being successful, determining a current camera focal length; Determine an image acquisition time interval according to the current camera focal length and the current track speed; The first camera, the second camera and the third camera are controlled to perform synchronous image acquisition at the image acquisition time interval as the time interval to generate a target image group, thereby obtaining the target image group set.

4. The method according to claim 3, wherein: The performing image stitching on each target image contained in each target image group in the target image group set to generate a stitched image includes: For each of the target images, the following processing steps are performed: Extracting feature points from the first image edge region and the second image edge region respectively to generate a first feature point group and a second feature point group, wherein the first image edge region and the second image edge region are symmetrical image edge regions in the target image; Performing regional grid division on the first image edge region and the second image edge region respectively to obtain a divided first image edge region and a divided second image edge region; Performing feature point fusion on the first feature points in the first feature point group that are located in the same regional grid and in the edge region of the divided first image to obtain the fused first feature points as third feature points to obtain a third feature point group; Performing feature point fusion on the second feature points in the second feature point group that are located in the same regional grid and in the edge region of the divided second image to obtain fused second feature points as fourth feature points to obtain a fourth feature point group; According to the relative position relationship of the target images in the target image group, feature point matching is performed according to the third feature point group and the fourth feature point group corresponding to the target images in the target image group to determine the splicing positions between the target images in the target image group; According to the stitching positions, the target images in the target image group are stitched to obtain a stitched image corresponding to the target image group.

5. The method according to claim 4, wherein: The steel surface defect detection model includes: a first convolution layer, a second convolution layer, a first convolution module, a third convolution layer, a first variable convolution module, a fourth convolution layer, a second variable convolution module, a fifth convolution layer, a third variable convolution module, a spatial pyramid pooling module, a bidirectional routing attention mechanism module, a second convolution module, a first bidirectional feature pyramid network, a first upsampling layer, a third convolution module, a second bidirectional feature pyramid network, a second upsampling layer, a sixth convolution layer, a third bidirectional feature pyramid network, a fourth convolution module, a seventh convolution layer, a fourth bidirectional feature pyramid network and a fifth convolution module.

6. An electronic device comprising: one or more processors; a storage device having one or more programs stored thereon; When the one or more programs are executed by the one or more processors, the one or more processors implement the method according to any one of claims 1 to 5.

7. A computer readable medium having a computer program stored thereon, wherein: When the computer program is executed by a processor, the method according to any one of claims 1 to 5 is implemented.

Citation Information

Patent Citations

  • High-speed wide board detection method

    CN106706653A

  • Strip steel online surface defect detection system and detection method thereof

    CN109030503A

  • Surface defect detecting method, surface defect detecting device, method for manufacturing steel material, steel material quality control method, steel material manufacturing equipment, method for creating surface defect determination model, and surface defect determination model

    CN113196040A

  • Image light supplement control method and device, equipment and storage medium

    CN117857905A

  • Multi-mode multi-channel image acquisition and analysis system for scrap steel quality inspection and use method

    CN117870572A

Cited By

  • Protective glove safety detection method, computer equipment and readable medium

    CN119067941A

  • Security check article automatic sorting system and method

    CN120861460A