Steel testing methods, electronic equipment and computer-readable media

Abstract

This disclosure discloses a steel inspection method, electronic device, and computer-readable medium. One specific implementation of the method includes: in response to a first infrared sensor detecting infrared obstruction and the main conveyor track being in a steel conveying state, determining the current ambient light state using an ambient light sensor; in response to the current ambient light state indicating a weak ambient light state, controlling a supplementary light to turn on, and adjusting the brightness of the supplementary light according to the current ambient light state; in response to the adjustment being completed or the current ambient light state indicating a strong ambient light state, and a second infrared sensor detecting infrared obstruction, controlling a first camera, a second camera, and a third camera to acquire images; stitching together the individual target images contained in the target image group to generate a stitched image; and performing defect detection on the stitched image using a steel surface defect detection model. This implementation achieves high-accuracy steel surface defect detection.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more specifically to steel testing methods, electronic devices, and computer-readable media. Background Technology

[0002] Steel, as a crucial basic material, plays a vital role in national economic development. It is widely used in various fields, including but not limited to construction, manufacturing, transportation, and energy. Especially in construction, the demand for steel increases significantly with building height. Surface defects generated during steel production, such as inclusions, spots, and pitting, affect steel quality and consequently, building quality. Currently, surface defect detection in steel typically employs methods such as manual inspection and laser scanning. However, manual inspection is costly and susceptible to subjective judgment, leading to low accuracy and efficiency. Laser scanning, on the other hand, is also costly and therefore difficult to widely adopt. Summary of the Invention

[0003] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0004] Some embodiments of this disclosure provide steel testing methods, electronic devices, and computer-readable media to address one or more of the technical problems mentioned in the background section above.

[0005] In a first aspect, some embodiments of this disclosure provide a steel inspection method applied to a steel surface defect detection device, wherein the steel surface defect detection device includes: an image acquisition device and a steel conveying device; the image acquisition device includes: a first camera, a second camera, a third camera, a supplementary light, and an ambient light sensor; the steel conveying device includes: a main conveying track, a first infrared sensor, a second infrared sensor, and a third infrared sensor; the second infrared sensor is disposed in front of the image acquisition device; the method includes: in response to the first infrared sensor detecting infrared occlusion and the main conveying track being in a steel conveying state, determining the current ambient light state through the ambient light sensor; in response to the current ambient light state representing a weak ambient light state, controlling the supplementary light to turn on; and based on the current ambient light state... In response to the ambient light condition, the brightness of the supplementary light is adjusted; upon completion of the adjustment or when the current ambient light condition indicates a strong ambient light condition and the second infrared sensor detects infrared obstruction, the first camera, the second camera, and the third camera are controlled to acquire images to obtain a set of target images, wherein each target image in the target image set corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the supplementary light to turn off; the target images contained in each target image set in the target image set are stitched together to generate a stitched image set; for each stitched image in the stitched image set, a defect detection is performed on the stitched image using a steel surface defect detection model to generate detection information.

[0006] Secondly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, wherein when the one or more programs are executed by the one or more processors, the one or more processors implement the method described in any implementation of the first aspect above.

[0007] Thirdly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method described in any of the implementations of the first aspect above.

[0008] The various embodiments of this disclosure have the following beneficial effects: Through the steel inspection methods of some embodiments of this disclosure, high-accuracy steel surface defect detection is achieved with low implementation cost. Specifically, this disclosure uses a steel surface defect detection device to realize the movement of the steel to be inspected and image acquisition. In practice, construction steel is heavy, and manual handling is costly and dangerous during image acquisition. Therefore, this disclosure uses a steel surface defect detection device to realize the movement of the steel and image acquisition, achieving automated movement and image acquisition. The specific analysis is as follows: First, in response to the first infrared sensor detecting infrared obstruction and the main conveyor track being in a steel conveying state, the ambient light sensor determines the current ambient light state. Second, in response to the current ambient light state indicating a weak ambient light state, the supplementary light is turned on, and the brightness of the supplementary light is adjusted according to the current ambient light state. The ambient light sensor collects the current ambient light state to determine whether to turn on the supplementary light for supplementary lighting, thus avoiding the problem of poor image quality acquired in low-light environments. Next, in response to the completion of adjustment or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the first camera, the second camera, and the third camera are controlled to acquire images, obtaining a set of target image groups. Each target image in the target image group corresponds to the same central axis. The third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the supplementary lighting to turn off. Further, the target images contained in each target image group in the target image group set are stitched together to generate a stitched image set. Image stitching is used to stitch together images acquired simultaneously by the first camera, the second camera, and the third camera at different image acquisition positions. Finally, for each stitched image in the stitched image set, a steel surface defect detection model is used to perform defect detection on the stitched image to generate detection information. That is, defect detection is performed through image recognition. In summary, this disclosure, by combining a steel surface defect detection device and using image recognition, achieves high-accuracy steel surface defect detection with low implementation cost. Attached Figure Description

[0009] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0010] Figure 1 This is a flowchart of some embodiments of the steel testing method according to this disclosure;

[0011] Figure 2 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0012] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0013] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0014] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0015] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0017] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0018] refer to Figure 1 The diagram illustrates a flow 100 of some embodiments of a steel testing method according to the present disclosure. This steel testing method includes the following steps:

[0019] Step 101: In response to the first infrared sensor detecting infrared occlusion and the main conveyor track being in the steel conveying state, the current ambient light state is determined by the ambient light sensor.

[0020] In some embodiments, the execution entity of the steel inspection method (e.g., a computing device) can determine the current ambient light state via an ambient light sensor in response to a first infrared sensor detecting infrared occlusion and the main conveyor track being in a steel conveying state. The steel inspection method is applied to a steel surface defect detection device. This device includes an image acquisition device and a steel conveying device. The image acquisition device includes a first camera, a second camera, a third camera, a supplementary light, and an ambient light sensor. The steel conveying device includes a main conveyor track, a first infrared sensor, a second infrared sensor, and a third infrared sensor. The second infrared sensor is positioned before the image acquisition device. The third infrared sensor is positioned at the end of the main conveyor track. The first infrared sensor is positioned before the second infrared sensor, at the front end of the main conveyor track. The main conveyor track may include multiple conveyor track shafts; when the main conveyor track is open, the conveyor track shafts rotate, thereby moving the items on the main conveyor track. The ambient light sensors can be positioned on both sides of the main conveyor track, facing inwards, to detect the ambient light state inside the main conveyor track. Specifically, the aforementioned executing entity can map the ambient light state based on the signal value detected by the ambient light sensor, and use this as the current ambient light state. Since the current ambient light state is used to determine whether to turn on the brightness of the supplementary lighting, the ambient light state can include: weak ambient light state and strong ambient light state.

[0021] It should be noted that the aforementioned computing device 101 can be either hardware or software. When the computing device is hardware, it can be implemented as a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. Specifically, the aforementioned steel surface defect detection device may also include a microcomputer. This microcomputer can be used to schedule and control the steel conveying device, as well as to perform image stitching and defect detection. Therefore, the executing entity can be the aforementioned microcomputer.

[0022] Optionally, the steel conveying device further includes: a secondary conveying track and a dragging device. The angle of the secondary conveying track is adjustable. The secondary conveying track is positioned before the main conveying track and is connected to it. The dragging device includes: a magnetic chuck, a dragging cable, and an electric winch. The dragging device is positioned above the main conveying track. The dragging cable is connected to both the electric winch and the magnetic chuck. The dragging cable is used to drag the steel to be inspected for surface defects. The dragging cable is wound up and down by the electric winch. The first infrared sensor is located at the junction of the main conveying track and the secondary conveying track. Specifically, the secondary conveying track can rotate counterclockwise around its junction with the main conveying track to adjust the track angle. The magnetic chuck may contain a coil that generates magnetic force when energized, thereby controlling the magnetic chuck to adhere to the steel surface. The first infrared sensor can detect whether steel has moved into the main conveying track.

[0023] In some optional implementations of certain embodiments, before determining the current ambient light state via the ambient light sensor in response to the first infrared sensor detecting infrared occlusion and the main conveying track being in a steel conveying state, the method further includes:

[0024] The first step is to drive the secondary conveyor track to change its track angle.

[0025] In practice, when the steel surface defect detection device is turned on, the aforementioned execution body can control the motor to drive the auxiliary conveyor track to rotate counterclockwise around the junction with the main conveyor track, thereby adjusting the track angle.

[0026] The second step is to stop changing the track angle of the secondary transport track in response to the secondary transport track touching the ground or the track angle of the secondary transport track being greater than or equal to a preset track angle.

[0027] In practice, the aforementioned actuator can detect the motor torque to determine whether the secondary conveyor track is in contact with the ground. However, there may be uneven ground. In such cases, relying solely on whether the secondary conveyor track is in contact with the ground to determine whether to stop changing the track angle could lead to excessively large changes in the track angle. Therefore, a limitation on using a preset track angle for track angle changes has been added.

[0028] The third step is to respond to the opening of the electric winch and to wind up the towing cable using the electric winch.

[0029] In practice, when the magnetic chuck is energized and the electric winch is turned on, it indicates that the steel has been fixed. Therefore, the aforementioned actuator can control the electric winch to wind up the drag cable to achieve the purpose of dragging the steel onto the auxiliary conveyor track.

[0030] Fourth step: In response to the infrared obstruction detected by the first infrared sensor, the main transport track is activated.

[0031] In practice, when the first infrared sensor detects infrared obstruction, it indicates that the steel has been transported to the junction of the main and auxiliary conveyor tracks, at which point the main conveyor track can be activated. Controlling the activation of the main conveyor track via the first infrared sensor avoids idling and unnecessary power consumption.

[0032] Fifth, in response to the preset start time of the main conveyor track, the magnetic chuck is magnetically released, and the towing cable is tightened by the electric winch.

[0033] In practice, because the towing device is positioned above the main conveyor track, the towing cable actually restricts the movement of the steel when it passes the towing device. Furthermore, if the magnetic chuck is immediately released when the first infrared sensor detects infrared obstruction, the steel may become stuck on the secondary conveyor track, preventing it from entering the main conveyor track normally. Therefore, this disclosure sets a preset time to ensure the steel enters the main conveyor track normally. Specifically, since the distance between the junction of the main and secondary conveyor tracks and the towing device is fixed, the preset time can also change when the conveying speed of the main conveyor track changes. For example, when the conveying speed increases, the preset time decreases; when the conveying speed decreases, the preset time increases. In addition, the towing cable is tightened by the electric winch to prevent wear on the main conveyor track caused by the towing cable and the magnetic chuck.

[0034] Step 102: In response to the current ambient light state representing a weak ambient light state, control the fill light to turn on, and adjust the brightness of the fill light according to the current ambient light state.

[0035] In some embodiments, the aforementioned execution entity can control the activation of supplementary lighting in response to the current ambient light state indicating a weak ambient light state, and adjust the brightness of the supplementary lighting according to the current ambient light state. In practice, the aforementioned execution entity can map the current ambient light state to a supplementary lighting level. For example, a supplementary lighting level of 1-5 corresponds to a strong ambient light state, while a supplementary lighting level of 6-10 corresponds to a weak ambient light state. Therefore, when the current ambient light state indicates a weak ambient light state, the aforementioned execution entity can adjust the brightness of the supplementary lighting to match the supplementary lighting level corresponding to the current ambient light state. In practice, for example, supplementary lighting can be respectively set on the lower side and the upper side of the main conveyor track to provide supplementary lighting to the upper and lower sides of the steel being conveyed on the main conveyor track. Furthermore, to achieve complete supplementary lighting to the surface of the steel, the image acquisition device can include three supplementary lighting units, such as supplementary lighting unit A, supplementary lighting unit B, and supplementary lighting unit C. Supplementary lighting unit A is located directly below the main conveyor track. Supplemental lights B and C are positioned on the upper side of the main conveyor track. Supplemental lights A, B, and C form a 120-degree angle with each other. The positions of supplemental lights A, B, and C are coplanar, and the normal vector of the plane containing supplemental lights A, B, and C is parallel to the running direction of the main conveyor track.

[0036] Step 103: In response to the completion of adjustment or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, control the first camera, the second camera, and the third camera to acquire images and obtain a set of target images.

[0037] In some embodiments, in response to the adjustment being completed or the current ambient light state indicating a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the aforementioned executing entity can control the first camera, second camera, and third camera to acquire images, obtaining a target image set. Each target image in the target image set corresponds to the same central axis. The third infrared sensor is used to control the first camera, second camera, third camera, and / or the supplementary lighting to turn off. Specifically, since the third infrared sensor is located at the end of the main conveyor track, when the third infrared sensor detects infrared occlusion, it switches to not detecting infrared occlusion, indicating that there is no steel on the main conveyor track. Therefore, the first camera, second camera, and third camera can be controlled to turn off. By controlling the cameras to turn off, firstly, unnecessary power consumption can be reduced. Secondly, it can avoid the cameras acquiring images that do not include the steel surface, thus eliminating the need for image filtering, reducing processing steps, and lowering the computational burden on the microcomputer.

[0038] In practice, the first, second, and third cameras are all oriented towards the center of their respective planes. Specifically, the executing entity can synchronize the acquisition timing of the first, second, and third cameras using a clock synchronizer.

[0039] Optionally, the main transport track includes an image acquisition area. Specifically, the image acquisition area can be the area in the middle of the main transport track that does not include the transport track axis. By setting the image acquisition area, the obstruction of the camera by the transport track axis is avoided, thereby improving image quality. The first camera is positioned directly below the image acquisition area. The second and third cameras are positioned above the image acquisition area. The included angles between the first, second, and third cameras are all 120 degrees.

[0040] In some optional implementations of certain embodiments, the execution entity controls the first camera, the second camera, and the third camera to acquire images and obtain a target image set, which may include the following steps:

[0041] The first step is to determine the current track speed of the main transport track.

[0042] In practice, for example, since the conveyor track shafts in the main conveyor track are driven by motors, the current track speed can be obtained by mapping the real-time rotational speed of the motor. Alternatively, the rotational speed of the conveyor track shafts can be detected using speedometers as the current track speed. Furthermore, to improve the accuracy of the obtained current track speed, multiple speedometers can be used to detect the rotational speeds of different conveyor track shafts in the main conveyor track, and the average value can be taken as the current track speed.

[0043] The second step is to synchronize the focal lengths of the first, second, and third cameras mentioned above.

[0044] In practice, firstly, the aforementioned executing entity can broadcast a current focus confirmation request to the first camera, the second camera, and the third camera. Then, when the current focus of the first camera, the second camera, and the third camera are inconsistent, a camera focus adjustment request is sent to the second camera and the third camera, using the current focus of the first camera as the reference.

[0045] The third step, in response to successful synchronization, is to determine the current camera focal length.

[0046] The current camera focal length can be the current focal length of the first camera.

[0047] The fourth step is to determine the image acquisition time interval based on the current camera focal length and the current track speed.

[0048] In practice, due to the size limitations of the camera's image sensor, the size of the images captured by the camera is limited. Lower image acquisition density may lead to missed images, while higher image acquisition density may result in greater overlap between images. Specifically, this can be simplified to assume a fixed image size. Therefore, under pre-calibration, there is a mapping relationship between the image coordinate system where the camera captures the images and the geodetic coordinate system where the steel surface defect detection device is located. Thus, the region corresponding to the image in the geodetic coordinate system can be determined. When the current track speed is known, the shooting interval between every two images can be determined using the speed formula, serving as the acquisition time interval.

[0049] Fifth, using the aforementioned image acquisition time interval as the time interval, control the first camera, the second camera, and the third camera to perform synchronous image acquisition to generate a target image group, thus obtaining the aforementioned target image group set.

[0050] In practice, for example, the aforementioned executing entity can send time intervals to the first, second, and third cameras, enabling them to automatically acquire images. Since the time intervals and acquisition start times are the same, synchronous image acquisition can be achieved—that is, synchronous image acquisition with a single control. Alternatively, the executing entity can use the image acquisition time interval as the time interval and control the first, second, and third cameras to acquire images synchronously via timed broadcasting—that is, synchronous image acquisition through multiple controls, which offers higher control precision compared to a single control method.

[0051] Step 104: Perform image stitching on each target image contained in each target image group in the target image group set to generate a stitched image, thus obtaining a stitched image set.

[0052] In some embodiments, the execution entity can stitch together the target images contained in each target image group in the target image group set to generate a stitched image, thus obtaining a stitched image set. In practice, since the target images in the target image group are images of the same steel area captured by different cameras from different perspectives, the execution entity can stitch the images sequentially in a clockwise order to obtain a stitched image.

[0053] In some optional implementations of certain embodiments, the execution entity performs image stitching on each target image contained in each target image group in the target image group set to generate a stitched image, which may include the following steps:

[0054] The first step is to perform the following processing steps on each of the target images mentioned above:

[0055] The first sub-step involves extracting feature points from the edge regions of the first and second images respectively to generate a first feature point group and a second feature point group.

[0056] The first image edge region and the second image edge region are symmetrical image edge regions in the target image. In practice, the four sides of the target image are defined clockwise: the left side can be defined as side A, the top side as side B, the right side as side C, and the bottom side as side D. Since image stitching only involves two sides, specifically, for the target image captured by the first camera, sides A and C are involved, with side A being the first image edge region and side C being the second image edge region. For the target image captured by the second camera, sides B and D are involved, with side B being the first image edge and side D being the second image edge. In practice, the aforementioned execution entity can extract feature points using the SIFT algorithm. Specifically, at the same time, there are overlapping areas between images captured by different cameras. Simply stitching images based on their edges will lead to repeated detection of duplicate image content. Furthermore, image stitching only involves two sides of the image; the other two sides are not involved in the stitching. Therefore, extracting feature points from the entire image would waste computational resources.

[0057] The second sub-step involves dividing the edge regions of the first and second images into regional grids to obtain the divided edge regions of the first and second images.

[0058] In practice, the principle of voxelization can be used as a reference to divide the edge regions of the first image and the edge regions of the second image in a plane into regional grids.

[0059] The third sub-step involves fusing the first feature points within the first feature point group that are located in the same grid area within the edge region of the first image after the above division, to obtain the fused first feature point, which is then used as the third feature point to obtain the third feature point group.

[0060] In practice, the aforementioned executing entity can average at least one first feature point located within the same grid area to obtain a third feature point.

[0061] The fourth sub-step involves fusing the second feature points within the second feature point group that are located in the same grid area within the edge region of the second image after the above division, to obtain the fused second feature points, which are then used as the fourth feature points to obtain the fourth feature point group.

[0062] In practice, the aforementioned executing entity can average at least one second feature point located within the same grid area to obtain a fourth feature point.

[0063] The second step involves matching feature points based on the relative positions of the target images in the target image group and the third and fourth feature point groups corresponding to the target images in the target image group, thereby determining the stitching positions between the target images in the target image group.

[0064] For example, taking the target image captured by the first camera and the target image captured by the second camera as examples, the aforementioned execution entity can perform feature point matching between the third feature point group corresponding to the target image captured by the first camera and the third feature point group corresponding to the target image captured by the second camera. This method can halve the number of feature point matches. Specifically, feature point matching can determine whether feature points match through similarity calculation. Determining the stitching position through feature point matching can minimize the occurrence of duplicate images, reduce unnecessary redundant detections during subsequent defect detection, and improve the efficiency of computational resource utilization.

[0065] The third step is to stitch the target images in the target image group according to the stitching position to obtain the stitched image corresponding to the target image group.

[0066] Step 105: For each stitched image in the stitched image set, perform defect detection on the stitched image using the steel surface defect detection model to generate detection information.

[0067] In some embodiments, for each stitched image in the stitched image set, the aforementioned execution entity can perform defect detection on the stitched image using a steel surface defect detection model to generate detection information. In practice, the aforementioned steel surface defect detection model can be the EfficinetDet model. The detection information can characterize the location and type of defects in the steel.

[0068] Optionally, the steel surface defect detection model includes: a first convolutional layer, a second convolutional layer, a first convolutional module, a third convolutional layer, a first variable convolutional module, a fourth convolutional layer, a second variable convolutional module, a fifth convolutional layer, a third variable convolutional module, a spatial pyramid pooling module, a bidirectional routing attention mechanism module, a second convolutional module, a first bidirectional feature pyramid network, a first upsampling layer, a third convolutional module, a second bidirectional feature pyramid network, a second upsampling layer, a sixth convolutional layer, a third bidirectional feature pyramid network, a fourth convolutional module, a seventh convolutional layer, a fourth bidirectional feature pyramid network, and a fifth convolutional module. The first, second, third, fourth, and fifth convolutional modules are implemented using C2f modules. The spatial pyramid pooling module is implemented using SPPF modules. The first, second, and third variable convolutional modules are implemented using C2f_DCNv2 modules. The bidirectional routing attention mechanism module is implemented using BiFormer modules. The first, second, third, and fourth bidirectional feature pyramid networks employ the BiFPN_Concat2 module. Furthermore, the steel surface defect detection model includes three detection heads: Detection Head A, Detection Head B, and Detection Head C. Detection Head A has an input size of 80×80×256. Detection Head C has an input size of 40×40×512. Detection Head B has an input size of 20×20×512. The total input size of the steel surface defect detection model is 640×640×3. By introducing the C2f_DCNv2 module on top of the C2f module, the edges and complex shapes of the target object are captured more effectively. Moreover, by introducing the BiFPN_Concat2 module, compared to the traditional PAFPN module, it can more effectively combine features, especially between different levels, without increasing computational cost. Simultaneously, BiFPN employs a formaldehyde fusion method, increasing model accuracy and generalization ability through repeated weighted fusion processes. Furthermore, to improve the detection accuracy of small targets, a BiFormer module was introduced to achieve efficient small target feature extraction based on an attention mechanism. In addition, the steel surface defect detection model employs the Wise-IoUv3 (CIoU) loss function, which can effectively solve the overfitting problem of low-quality target bounding boxes.

[0069] Optionally, the detection information in the detection information set includes: first detection information, second detection information, and third detection information. The first, second, and third detection information correspond to detection results of different sizes. Specifically, the first detection information is generated by detection head A. The second detection information is generated by detection head B. The third detection information is generated by detection head C.

[0070] In some optional implementations of certain embodiments, the execution entity performs defect detection on the stitched image using a steel surface defect detection model to generate detection information, which may include the following steps:

[0071] The first step is to input the stitched image into the first convolutional layer to obtain the first feature.

[0072] The second step is to input the first feature into the second convolutional layer to obtain the second feature.

[0073] The third step is to input the second feature into the first convolution module to obtain the third feature.

[0074] The fourth step is to input the third feature into the three convolutional layers to obtain the fourth feature.

[0075] Fifth step: Input the fourth feature into the first variable convolution module to obtain the fifth feature.

[0076] The sixth step is to input the fifth feature into the fourth convolutional layer to obtain the sixth feature.

[0077] Step 7: Input the sixth feature into the second variable convolution module to obtain the seventh feature.

[0078] Step 8: Input the seventh feature into the fifth convolutional layer to obtain the eighth feature.

[0079] The ninth step is to input the eighth feature into the third variable convolution module to obtain the ninth feature.

[0080] Step 10: Input the ninth feature into the spatial pyramid pooling module to obtain the tenth feature.

[0081] In the eleventh step, the tenth feature is input into the bidirectional routing attention mechanism module to obtain the eleventh feature.

[0082] In the twelfth step, the eleventh feature is upsampled through the second upsampling layer to obtain the twelfth feature.

[0083] Step 13: Input the seventh and twelfth features into the second bidirectional feature pyramid network to obtain the thirteenth feature.

[0084] Step fourteen: Input the thirteenth feature into the third convolution module to obtain the fourteenth feature.

[0085] Step 15: Through the first upsampling layer described above, the fourteenth feature is upsampled to obtain the fifteenth feature.

[0086] Step 16: Based on the fifteenth feature mentioned above, generate the first detection information included in the detection information corresponding to the stitched image.

[0087] Step 17: Input the aforementioned 15th feature into the aforementioned 6th convolutional layer to obtain the 17th feature.

[0088] Step 18: Input the seventeenth feature, the fourteenth feature, and the sixth feature into the third bidirectional feature pyramid network to obtain the eighteenth feature.

[0089] Step 19: Input the eighteenth feature into the fourth convolution module to obtain the nineteenth feature.

[0090] Step 20: Based on the nineteenth feature mentioned above, generate the second detection information included in the detection information corresponding to the stitched image.

[0091] Step 21: Input the nineteenth feature mentioned above into the seventh convolutional layer to obtain the twentieth feature.

[0092] Step 22: Input the 20th and 11th features into the fourth bidirectional feature pyramid network to obtain the 21st feature.

[0093] Step 23: Input the aforementioned twenty-first feature into the aforementioned fifth convolution module to obtain the twenty-second feature.

[0094] Step 24: Based on the aforementioned feature 22, generate the third detection information included in the detection information corresponding to the stitched image.

[0095] The various embodiments of this disclosure have the following beneficial effects: Through the steel inspection methods of some embodiments of this disclosure, high-accuracy steel surface defect detection is achieved with low implementation cost. Specifically, this disclosure uses a steel surface defect detection device to realize the movement of the steel to be inspected and image acquisition. In practice, construction steel is heavy, and manual handling is costly and dangerous during image acquisition. Therefore, this disclosure uses a steel surface defect detection device to realize the movement of the steel and image acquisition, achieving automated movement and image acquisition. The specific analysis is as follows: First, in response to the first infrared sensor detecting infrared obstruction and the main conveyor track being in a steel conveying state, the ambient light sensor determines the current ambient light state. Second, in response to the current ambient light state indicating a weak ambient light state, the supplementary light is turned on, and the brightness of the supplementary light is adjusted according to the current ambient light state. The ambient light sensor collects the current ambient light state to determine whether to turn on the supplementary light for supplementary lighting, thus avoiding the problem of poor image quality acquired in low-light environments. Next, in response to the completion of adjustment or the current ambient light state representing a strong ambient light state, and the second infrared sensor detecting infrared occlusion, the first camera, the second camera, and the third camera are controlled to acquire images, obtaining a set of target image groups. Each target image in the target image group corresponds to the same central axis. The third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the supplementary lighting to turn off. Further, the target images contained in each target image group in the target image group set are stitched together to generate a stitched image set. Image stitching is used to stitch together images acquired simultaneously by the first camera, the second camera, and the third camera at different image acquisition positions. Finally, for each stitched image in the stitched image set, a steel surface defect detection model is used to perform defect detection on the stitched image to generate detection information. That is, defect detection is performed through image recognition. In summary, this disclosure, by combining a steel surface defect detection device and using image recognition, achieves high-accuracy steel surface defect detection with low implementation cost.

[0096] The following is for reference. Figure 2 It shows a schematic diagram of the structure of an electronic device (e.g., a computing device) 200 suitable for implementing some embodiments of the present disclosure. Figure 2 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0097] like Figure 2As shown, the electronic device 200 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 201, which can perform various appropriate actions and processes according to a program stored in the read-only memory 202 or a program loaded from the storage device 208 into the random access memory 203. The random access memory 203 also stores various programs and data required for the operation of the electronic device 200. The processing unit 201, the read-only memory 202, and the random access memory 203 are interconnected via a bus 204. An input / output interface 205 is also connected to the bus 204.

[0098] Typically, the following devices can be connected to I / O interface 205: input devices 206 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 207 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 208 including, for example, magnetic tapes, hard disks, etc.; and communication devices 209. Communication device 209 allows electronic device 200 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 2 An electronic device 200 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 2 Each box shown can represent a device or multiple devices as needed.

[0099] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 209, or installed from storage device 208, or installed from read-only memory 202. When the computer program is executed by processing device 201, it performs the functions defined above in the methods of some embodiments of this disclosure.

[0100] It should be noted that, in some embodiments of this disclosure, the computer-readable medium may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example,—but not limited to—an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0101] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and end-to-end networks (e.g., ad hoc end-to-end networks), as well as any currently known or future-developed networks.

[0102] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs that, when executed by the electronic device, cause the electronic device to: in response to the aforementioned first infrared sensor detecting infrared obstruction and the aforementioned main conveying track being in a steel conveying state, determine the current ambient light state via the aforementioned ambient light sensor; in response to the aforementioned current ambient light state indicating a weak ambient light state, control the aforementioned supplementary light to turn on, and adjust the brightness of the aforementioned supplementary light according to the aforementioned current ambient light state; in response to the adjustment being completed or the aforementioned current ambient light state indicating a strong ambient light state, and the aforementioned second infrared sensor detecting infrared obstruction, control the aforementioned first... The camera, the second camera, and the third camera acquire images to obtain a set of target images, wherein each target image in the target image set corresponds to the same central axis. The third infrared sensor is used to control the first camera, the second camera, the third camera, and / or the supplementary light to turn off. The target images contained in each target image set in the target image set are stitched together to generate a stitched image set. For each stitched image in the stitched image set, a steel surface defect detection model is used to perform defect detection on the stitched image to generate detection information.

[0103] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0104] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0105] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0106] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A steel material detection method applied to a steel material surface defect detection device, wherein, The steel surface defect detection device comprises an image acquisition device and a steel conveying device, the image acquisition device comprises a first camera, a second camera, a third camera, a light supplement lamp and an ambient light sensor, the steel conveying device comprises a main conveying track, a first infrared sensor, a second infrared sensor and a third infrared sensor, the second infrared sensor is arranged before the image acquisition device, the main conveying track comprises an image acquisition area, the first camera arranged directly below the image acquisition area, and the included angle between the second camera and the third camera arranged above the image acquisition area is 120 degrees, and the method comprises: in response to the first infrared sensor detecting infrared obstruction and the main conveying track being in a steel conveying state, determining the current ambient light state through the ambient light sensor; in response to the current ambient light state representing a weak ambient light state, controlling the light supplement lamp to be turned on, and adjusting the light supplement lamp brightness according to the current ambient light state; in response to the adjustment being completed or the current ambient light state representing a strong ambient light state and the second infrared sensor detecting infrared obstruction, controlling the first camera, the second camera and the third camera to perform synchronous image acquisition to obtain a target image group set, wherein each target image in the target image group corresponds to the same central axis, and the third infrared sensor is used to control the first camera, the second camera, the third camera and / or the light supplement lamp to be turned off; performing image stitching on each target image contained in each target image group in the target image group set to generate a stitched image, thereby obtaining a stitched image set. For each of the set of spliced images, the spliced image is subjected to defect detection by a steel surface defect detection model to generate detection information, wherein the steel surface defect detection model comprises: a first convolutional layer, a second convolutional layer, a first convolutional module, a third convolutional layer, a first variable convolutional module, a fourth convolutional layer, a second variable convolutional module, a fifth convolutional layer, a third variable convolutional module, a spatial pyramid pooling module, a bidirectional routing attention mechanism module, a second convolutional module, a first bidirectional feature pyramid network, a first upsampling layer, a third convolutional module, a second bidirectional feature pyramid network, a second upsampling layer, a sixth convolutional layer, a third bidirectional feature pyramid network, a fourth convolutional module, a seventh convolutional layer, a fourth bidirectional feature pyramid network, a fifth convolutional module, a detection head A, a detection head B, and a detection head C, wherein the first convolutional module, the second convolutional module, the third convolutional module, the fourth convolutional module, and the fifth convolutional module are implemented using a C2f module, the spatial pyramid pooling module is implemented using a SPPF module, the first variable convolutional module, the second variable convolutional module, and the third variable convolutional module are implemented using a C2f_DCNv2 module, the bidirectional routing attention mechanism module is implemented using a BiFormer module, the first bidirectional feature pyramid network, the second bidirectional feature pyramid network, the third bidirectional feature pyramid network, and the fourth bidirectional feature pyramid network are implemented using a BiFPN_Concat2 module, the first variable convolutional module is arranged between the third convolutional layer and the fourth convolutional layer, the second variable convolutional module is arranged between the fourth convolutional layer and the fifth convolutional layer, the third variable convolutional module is arranged between the fifth convolutional layer and the spatial pyramid pooling module, the bidirectional routing attention mechanism module is arranged after the spatial pyramid pooling module, the second bidirectional feature pyramid network is connected with the second variable convolutional module and the second upsampling layer, the third bidirectional feature pyramid network is connected with the sixth convolutional layer, the third convolutional module, and the fourth convolutional layer respectively, and the fourth bidirectional feature pyramid network is connected with the seventh convolutional layer and the bidirectional routing attention mechanism module respectively.

2. The method of claim 1, wherein, The steel conveying device further comprises a secondary conveying track and a dragging device, wherein the track angle of the secondary conveying track is adjustable, the secondary conveying track is arranged before the main conveying track, the secondary conveying track is connected with the main conveying track, the dragging device comprises a magnetic suction disc, a dragging cable, and an electric winch, the dragging device is arranged above the main conveying track, the dragging cable is connected with the electric winch and the magnetic suction disc respectively, the dragging cable is used to drag the steel to be subjected to steel surface defect detection, the dragging cable is retracted and released by the electric winch, and the first infrared sensor is arranged at the joint of the main conveying track and the secondary conveying track; and Before the current ambient light state is determined by the ambient light sensor in response to the detection of the infrared obstruction by the first infrared sensor and the main conveying track being in a steel conveying state, the method further comprises: driving the secondary conveying track to change the track angle; stop the track angle transformation for the secondary conveying track in response to the secondary conveying track touching the ground or the track angle of the secondary conveying track being greater than or equal to a preset track angle; wind the tow cable by the electric winch in response to the electric winch being turned on; start the main conveying track in response to the first infrared sensor detecting an infrared obstruction; release the magnetic force of the magnetic suction disc and tighten the tow cable by the electric winch in response to the main conveying track being started for a preset time length.

3. The method of claim 2, wherein, the control of the first camera, the second camera and the third camera to capture images to obtain a target image group set comprises: determining a current track moving speed of the main conveying track; synchronizing the camera focal lengths of the first camera, the second camera and the third camera; determining a current camera focal length in response to successful synchronization; determining an image capture time interval according to the current camera focal length and the current track moving speed; controlling the first camera, the second camera and the third camera to capture images synchronously with the image capture time interval as the time interval to generate a target image group and obtain the target image group set.

4. The method of claim 3, wherein, the image stitching of each target image contained in the target image group set to generate a stitched image comprises: performing the following processing steps on each target image in the target image: extract feature points from the first image edge region and the second image edge region respectively to generate a first feature point group and a second feature point group, wherein the first image edge region and the second image edge region are symmetrical image edge regions in the target image; perform region grid division on the first image edge region and the second image edge region respectively to obtain a divided first image edge region and a divided second image edge region; fuse first feature points in the first feature point group located in the same region grid in the divided first image edge region to obtain a fused first feature point as a third feature point to obtain a third feature point group; fuse second feature points in the second feature point group located in the same region grid in the divided second image edge region to obtain a fused second feature point as a fourth feature point to obtain a fourth feature point group; perform feature point matching according to the third feature point group and the fourth feature point group corresponding to the target images in the target image group according to the relative positional relationship of the target images in the target image group to determine the stitching positions between the target images in the target image group; perform image stitching on the target images in the target image group according to the stitching positions to obtain a stitched image corresponding to the target image group. 5.An electronic device, comprising: one or more processors; a storage having one or more programs stored thereon; when the one or more programs are executed by the one or more processors, the one or more processors implement the method of any one of claims 1 to 4.

6. A computer readable medium having stored thereon a computer program, wherein, The computer program, which is executed by a processor, implements the method as claimed in any one of claims 1 to 4.

Images