Steel bar bundling quality detection method, device, storage medium and processor

By conducting convolution and feature information fusion detection on the reinforcement video data, the problem of low quality detection efficiency of steel bar bundling is solved, and the rapid accuracy of global visual scanning and quality evaluation of on-site images is achieved.

CN118333964BActive Publication Date: 2024-11-12GUANGDONG POWER GRID CO LTD +1
View PDF 4 Cites 0 Cited by

Patent Information

Application Number
CN202410436655.7
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-04-11
Publication Date
2024-11-12
Estimated Expiration
2044-04-11

AI Technical Summary

Technical Problem

The quality detection efficiency of steel bar bundling is low, making it difficult to perform global visual scanning of on-site images, and it is impossible to quickly distinguish between important and non-important information.

Method used

By obtaining the video data of the steel bars, convolution is performed using a quality detection model trained based on image data samples, initial and final feature information is extracted, and fusion detection is performed to obtain the quality score of the steel bar bundling.

Benefits of technology

The global visual scanning of on-site images is realized, the efficiency of steel bar bundling quality detection is improved, and the quality of steel bar bundling can be quickly and accurately evaluated.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118333964B_ABST
    Figure CN118333964B_ABST
Patent Text Reader

Abstract

The present invention discloses a method, device, storage medium and processor for quality inspection of steel bar bundling. The method comprises: obtaining video data of steel bars in a bundling state; determining at least one video frame data to be tested in the video data; inputting the video frame data to be tested into a quality inspection model for convolution to obtain feature information of the steel bars; fusing the first feature information and the second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent the feature information output from the initial extraction layer in the quality inspection model, and the second feature information is used to represent the feature information output from the final extraction layer in the quality inspection model; detecting the target feature information to obtain a detection result, wherein the detection result is used to represent the quality of the steel bar bundling. The present invention solves the technical problem of low efficiency of quality inspection of steel bar bundling.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the electric power field, and in particular to a quality detection method, device, storage medium and processor for steel bar bundling. Background Art

[0002] At present, in the quality inspection of steel bar bundling, it is often difficult to perform a global visual scan of the on-site image, and it is also difficult to quickly notice which information is important and which is unimportant in the scanned image, resulting in the technical problem of low efficiency of the quality inspection of steel bar bundling.

[0003] With respect to the technical problem of low efficiency in quality inspection of the above-mentioned steel bar bundling, no effective solution has been proposed yet. Summary of the invention

[0004] Embodiments of the present invention provide a method, device, storage medium and processor for detecting the quality of steel bar bundling, so as to at least solve the technical problem of low efficiency of the quality detection of steel bar bundling.

[0005] According to one aspect of an embodiment of the present invention, a quality inspection method for steel bar bundling is provided. The method comprises: obtaining video data of steel bars in a bundling state; determining at least one video frame data to be tested in the video data; inputting the video frame data to be tested into a quality inspection model for convolution to obtain feature information of the steel bars, wherein the quality inspection model is obtained by classification training based on at least image data samples, and the image data samples are obtained by dividing the video data samples; fusing the first feature information and the second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent the feature information output from the initial extraction layer in the quality inspection model, and the second feature information is used to represent the feature information output from the final extraction layer in the quality inspection model; detecting the target feature information to obtain a detection result, wherein the detection result is used to represent the quality of the steel bar bundling.

[0006] Optionally, determining at least one video frame data to be tested in the video data includes: traversing at least one video frame data in the video data to obtain at least one initial video frame data; and determining the video frame data to be tested based on the initial video frame data.

[0007] Optionally, based on the initial video frame data, determining the video frame data to be tested includes: marking the initial video frame data according to a target boundary to obtain the video frame data to be tested, wherein the target boundary is used to display the steel bars in a bundled state.

[0008] Optionally, the video frame data to be tested is input into the quality inspection model for convolution to obtain characteristic information of the steel bars, including: inputting the video frame data to be tested into the preprocessing layer in the quality inspection model for conversion to obtain converted video frame data to be tested; inputting the converted video frame data to be tested into the initial extraction layer for convolution to obtain first characteristic information; determining the second characteristic information based on the first characteristic information; and determining the first characteristic information and the second characteristic information as characteristic information.

[0009] Optionally, the video frame data to be tested is input into a preprocessing layer in a quality detection model for conversion to obtain converted video frame data to be tested, including: inputting the video frame data to be tested into a preprocessing layer for convolution to obtain convolved video frame data to be tested; in the preprocessing layer, performing interval conversion on the convolved video frame data to be tested to obtain converted video frame data to be tested.

[0010] Optionally, based on the first feature information, determining the second feature information includes: inputting the first feature information into a next extraction layer adjacent to the initial extraction layer for convolution to obtain the first feature information after convolution, wherein the next extraction layer includes at least one extraction layer; inputting the first feature information after convolution into a final extraction layer for convolution to obtain the second feature information, wherein the final extraction layer is adjacent to the next extraction layer.

[0011] Optionally, the target feature information is detected to obtain a detection result, including: using a detection sub-model to detect the target feature information to obtain a detection result, wherein the detection sub-model is set in the quality detection model, and the detection sub-model is a classification model set based on a regression algorithm.

[0012] Optionally, the method also includes: uploading the detection results to the cloud.

[0013] Optionally, obtaining video data of steel bars in a bundled state includes: monitoring whether the steel bars are in a bundled state to obtain a monitoring result; and obtaining video data in response to the monitoring result that the steel bars are in a bundled state.

[0014] According to one aspect of an embodiment of the present invention, a quality inspection device for steel bar bundling is provided. The device may include: an acquisition unit for acquiring video data of steel bars in a bundling state; a determination unit for determining at least one video frame data to be tested in the video data; a convolution unit for inputting the video frame data to be tested into a quality inspection model for convolution to obtain feature information of the steel bars, wherein the quality inspection model is obtained by classification training based at least on image data samples, and the image data samples are obtained by dividing the video data samples; a fusion unit for fusing the first feature information and the second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent the feature information output from the initial extraction layer in the quality inspection model, and the second feature information is used to represent the feature information output from the final extraction layer in the quality inspection model; a detection unit for detecting the target feature information to obtain a detection result, wherein the detection result is used to represent the quality of the steel bar bundling.

[0015] According to another aspect of the present invention, a computer-readable storage medium is provided, wherein the computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the steel bar bundling quality detection method of the present invention.

[0016] According to another aspect of an embodiment of the present invention, a processor is further provided, wherein the processor is used to run a program, wherein when the program is run by the processor, the quality detection method for steel bar bundling according to an embodiment of the present invention is executed.

[0017] According to another aspect of an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program. When the computer program is executed by a processor, the quality detection method for steel bar bundling according to an embodiment of the present invention is implemented.

[0018] In an embodiment of the present invention, after determining that the steel bars are in a bundled state, video data of the steel bars in the bundled state is obtained, and then at least one video frame data to be tested in the video data is determined, and then the determined at least one video frame data to be tested is input into a quality detection model for convolution, so that feature information of the steel bars can be obtained, and by fusing the first feature information and the second feature information in the feature information, target feature information can be obtained, and finally, by detecting the target feature information, a detection result can be obtained, that is, the quality of the steel bar bundling can be obtained, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0020] Figure 1 is a flow chart of a method for detecting quality of steel bar bundling according to an embodiment of the present invention;

[0021] Figure 2 is a flow chart of a method for detecting steel bar bundling data in a video according to an embodiment of the present invention;

[0022] Figure 3 is a flow chart of a classification method based on the CBAMy-RFB model according to an embodiment of the present invention;

[0023] FIG4( a ) is a schematic diagram of the structure of a CBAMy-RFB model according to an embodiment of the present invention;

[0024] FIG4( b ) is a schematic diagram of a backbone network RESInet incorporating the residual idea according to an embodiment of the present invention;

[0025] FIG4( c ) is a schematic diagram of a backbone network RESInet with a CBAM module added according to an embodiment of the present invention;

[0026] FIG4( d ) is a schematic diagram of an image feature enhancement module based on a deep network according to an embodiment of the present invention;

[0027] Figure 5 Schematic diagram of a quality detection device for steel bar bundling according to an embodiment of the present invention. DETAILED DESCRIPTION

[0028] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only embodiments of a part of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0029] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0030] Example 1

[0031] According to an embodiment of the present invention, a method for inspecting the quality of steel bar bundling is provided. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0032] Figure 1 is a flow chart of a method for quality inspection of steel bar bundling according to an embodiment of the present invention, the method may include the following steps:

[0033] Step S101, obtaining video data of steel bars in a bundled state.

[0034] In the technical solution provided in the above step S101 of the present invention, the above-mentioned steel bars can be set at least in the power pipeline corridor, and the above-mentioned video data can be obtained by a collection device by shooting the steel bars in a bundled state. For example, the collection device can include at least one of the following devices: a drone, a camera, a video camera, etc., which are only used as examples here and are not specifically limited.

[0035] In this embodiment, video data of the steel bars in a bundled state is obtained. For example, after determining that the steel bars are in a bundled state, video data of the steel bars in a bundled state can be collected by controlling the acquisition equipment. For example, by controlling the flight of a drone, video data of the steel bars in a bundled state can be captured, thereby obtaining video data of the steel bars in a bundled state. This is only an example and is not specifically limited here.

[0036] Step S102, determining at least one video frame data to be tested in the video data.

[0037] In the technical solution provided in the above step S102 of the present invention, the above-mentioned video frame data to be tested may be red, green and blue (RGB) three-channel data. For example, the above-mentioned video frame data to be tested may be 416×416 pixels. This is only used as an example and is not specifically limited.

[0038] In this embodiment, after obtaining the video data of the steel bars in a bundled state, at least one video frame data to be tested in the video data is determined. For example, by traversing at least one video frame data in the video data, at least one video frame data to be tested can be obtained, wherein the above traversal can be sequential traversal, reverse order traversal, in-order traversal, depth-first traversal (DFS), breadth-first traversal (BFS), etc., which is only used as an example here and is not specifically limited.

[0039] Step S103: input the video frame data to be tested into the quality inspection model for convolution to obtain the characteristic information of the steel bars.

[0040] In the technical solution provided in the above step S103 of the present invention, the above-mentioned quality detection model can be obtained by classification training based on at least image data samples and deep neural networks, and the image data samples can be obtained by dividing video data samples. For example, the image data samples can at least include: image training samples and image verification samples. The video data samples can be obtained by photographing steel bars in a bundled state by an acquisition device. The division can be a division according to a preset ratio. The preset ratio can be changed according to changes in the scene. If the placement position of the steel bars in the power corridor scene changes, the preset ratio will also change. This is only an example and is not specifically limited.

[0041] In this embodiment, after determining at least one video frame data to be tested in the video data, the video frame data to be tested is input into the quality detection model for convolution to obtain characteristic information of the steel bars. For example, by inputting the determined at least one video frame data to be tested into the quality detection model for conversion, the converted video frame data to be tested can be obtained, and then the converted video frame data to be tested is convolved to obtain characteristic information of the steel bars in a bundled state. This is only an example and is not specifically limited.

[0042] It should be noted that the above preset ratios may be as follows: 70% for image training samples, 15% for image verification samples, and 15% for image test samples. The preset ratios here are only used as examples and are not specifically limited.

[0043] Step S104: merge the first feature information and the second feature information in the feature information to obtain target feature information.

[0044] In the technical solution provided in the above step S104 of the present invention, the above first characteristic information can be used to represent the characteristic information output from the initial extraction layer in the quality detection model, and the above second characteristic information can be used to represent the characteristic information output from the final extraction layer in the quality detection model, wherein the initial extraction layer can be separated from the final extraction layer by at least one intermediate extraction layer, which is only used as an example here and is not specifically limited.

[0045] In this embodiment, after the video frame data to be tested is input into the quality inspection model for convolution to obtain the feature information of the steel bars, the first feature information and the second feature information in the feature information are fused to obtain the target feature information. For example, the first feature information is output from the initial extraction layer in the quality inspection model, and the second feature information is output from the final extraction layer in the quality inspection model. Then, the target feature information is obtained by fusing the output first feature information with the output second feature information. The above fusion can be a merging operation performed on the feature information, which is only used as an example here without specific limitation.

[0046] Step S105, detecting the target feature information to obtain a detection result.

[0047] In the technical solution provided in the above step S105 of the present invention, the above detection result can be used to indicate the quality of the steel bar bundling. For example, the above detection result can be that the quality of the steel bar bundling is good (for example, the quality score of the steel bar bundling is in the range of 85 points to 100 points), or it can be that the quality of the steel bar bundling is average (for example, the quality score of the steel bar bundling is in the range of 61 points to 84 points), or it can be that the quality of the steel bar bundling is poor (for example, the quality score of the steel bar bundling is in the range of 0 points to 60 points), etc., which is only used as an example here and is not specifically limited.

[0048] In this embodiment, after the first feature information and the second feature information in the feature information are fused to obtain the target feature information, the target feature information is detected to obtain the detection result. For example, in the quality detection model, the quality of the steel bar bundling can be obtained by performing quality detection on the target feature information, thereby determining whether the steel bars need to be bundled again to ensure that the quality score of the steel bar bundling is within the preferred range, for example, to ensure that the quality score of the steel bar bundling is within the range of 85 to 100 points. This is only an example and is not specifically limited.

[0049] In the above steps S101 to S105 of the present application, when it is determined that the steel bars are in a bundled state, video data of the steel bars in the bundled state is obtained, and then at least one video frame data to be tested in the video data is determined, and then the feature information of the steel bars can be obtained by inputting the determined at least one video frame data to be tested into the quality detection model for convolution, and the target feature information can be obtained by fusing the first feature information and the second feature information in the feature information. Finally, the detection result can be obtained by detecting the target feature information, that is, the quality of the steel bar bundling can be obtained, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling.

[0050] The above method of this embodiment is further introduced below.

[0051] As an optional implementation manner, step S102, determining at least one video frame data to be tested in the video data, includes: traversing at least one video frame data in the video data to obtain at least one initial video frame data; and determining the video frame data to be tested based on the initial video frame data.

[0052] In this embodiment, after obtaining the video data of the steel bars in a bundled state, at least one video frame data in the video data is traversed to obtain at least one initial video frame data; based on the initial video frame data, the video frame data to be tested is determined. For example, by traversing at least one video frame data in the video data, at least one initial video frame data can be obtained, and then by marking the obtained initial video frame data, the video frame data to be tested can be obtained. This is only an example and is not specifically limited.

[0053] Optionally, by performing a pre-order traversal on at least one video frame in the video data, at least one initial video frame can be obtained, and then on a visual operation interface, by marking the obtained initial video frame data, the video frame data to be tested can be obtained. This is only an example and is not specifically limited.

[0054] As an optional implementation manner, based on the initial video frame data, determining the video frame data to be tested includes: marking the initial video frame data according to the target boundary to obtain the video frame data to be tested.

[0055] In this embodiment, the target boundary can be used to display the steel bars in a bundled state to highlight the bundled state of the steel bars. For example, the target boundary can be represented by a picture frame, which is only used as an example and is not specifically limited.

[0056] In this embodiment, after traversing at least one video frame data in the video data to obtain at least one initial video frame data, the initial video frame data is annotated according to the target boundary to obtain the video frame data to be tested. For example, according to the bundling state of the steel bars, the shape and size of the target boundary to be generated can be determined. On the basis of determining the shape and size of the target boundary to be generated, a target boundary corresponding to the bundling state can be generated. Then, according to the generated target boundary, the initial video frame data is frame-annotated, and the annotated initial video frame data is determined as the video frame data to be tested, thereby obtaining the video frame data to be tested.

[0057] It should be noted that the shape and size of the target boundary can be adjusted according to the bundling state of the steel bars. That is, if the bundling state of the steel bars is different, the shape and size of the target boundary will also be different.

[0058] As an optional implementation method, step S103, inputs the video frame data to be tested into the quality inspection model for convolution to obtain feature information of the steel bar, including: inputting the video frame data to be tested into the preprocessing layer in the quality inspection model for conversion to obtain converted video frame data to be tested; inputting the converted video frame data to be tested into the initial extraction layer for convolution to obtain first feature information; based on the first feature information, determining the second feature information; determining the first feature information and the second feature information as feature information.

[0059] In this embodiment, the above-mentioned preprocessing layer can be used to represent a convolutional layer adjacent to an input layer in a quality detection model.

[0060] In this embodiment, after determining at least one video frame data to be tested in the video data, by inputting the determined at least one video frame data to be tested into the preprocessing layer in the quality inspection model for conversion, the converted video frame data to be tested can be obtained, and then the converted video frame data to be tested is input into the initial extraction layer for convolution, so as to obtain the first feature information of the steel bars in a bundled state, and then by inputting the obtained first feature information of the steel bars into at least one next extraction layer adjacent to the initial extraction layer for convolution, the second feature information of the steel bars can be obtained, and the first feature information of the steel bars and the second feature information of the steel bars are determined as the feature information of the steel bars, which is only used as an example here and is not specifically limited.

[0061] As an optional implementation method, the video frame data to be tested is input into the preprocessing layer in the quality detection model for conversion to obtain the converted video frame data to be tested, including: inputting the video frame data to be tested into the preprocessing layer for convolution to obtain the convolved video frame data to be tested; in the preprocessing layer, performing interval conversion on the convolved video frame data to be tested to obtain the converted video frame data to be tested.

[0062] In this embodiment, after determining at least one video frame data to be tested in the video data, the video frame data to be tested is input into a preprocessing layer for convolution to obtain the convolved video frame data to be tested; in the preprocessing layer, the convolved video frame data to be tested is interval-converted to obtain the converted video frame data to be tested. For example, by inputting at least one video frame data to be tested into a preprocessing layer in a quality detection model for convolution, the convolved video frame data to be tested can be obtained, and then in the preprocessing layer, by performing interval conversion on the convolved video frame data to be tested, the converted video frame data to be tested can be obtained. This is only an example and is not specifically limited.

[0063] Optionally, in the preprocessing layer, the convolved video frame data to be tested can be mapped to a preset range by standardizing the convolved video frame data to be tested. The video frame data to be tested mapped to the preset range is the converted video frame data to be tested, thereby obtaining the video frame data to be tested, wherein the preset range can change according to the change of the scene. For example, if the number of steel bars in the power corridor scene changes, the preset range also changes. This is only an example for illustration and is not specifically limited.

[0064] As an optional implementation method, the second feature information is determined based on the first feature information, including: inputting the first feature information into the next extraction layer adjacent to the initial extraction layer for convolution to obtain the first feature information after convolution; inputting the first feature information after convolution into the final extraction layer for convolution to obtain the second feature information.

[0065] In this embodiment, the next extraction layer may include at least one extraction layer, and the final extraction layer may be adjacent to the next extraction layer.

[0066] In this embodiment, after the converted video frame data to be tested is input into the initial extraction layer for convolution to obtain the first feature information, the first feature information is input into the next extraction layer adjacent to the initial extraction layer for convolution to obtain the first feature information after convolution; the first feature information after convolution is input into the final extraction layer for convolution to obtain the second feature information. For example, the first feature information of the steel bar is input into at least one next extraction layer adjacent to the initial extraction layer for convolution to obtain the first feature information after convolution, and then the second feature information of the steel bar is obtained by inputting the first feature information of the steel bar into the final extraction layer for convolution.

[0067] As an optional implementation manner, step S105, detecting the target feature information to obtain a detection result, includes: using a detection sub-model to detect the target feature information to obtain a detection result.

[0068] In this embodiment, the above detection sub-model may be set in the quality detection model, and the above detection sub-model may be a classification model set based on a regression algorithm.

[0069] In this embodiment, after the first feature information and the second feature information in the feature information are fused to obtain the target feature information, the target feature information is detected using the detection sub-model to obtain the detection result. For example, by inputting the target feature information into the detection sub-model for detection, the detection result can be obtained, that is, the quality of the steel bar bundling can be obtained, thereby determining whether the steel bars need to be bundled again to ensure that the quality score of the steel bar bundling is within the preferred range, for example, to ensure that the quality score of the steel bar bundling is within the range of 85 to 100 points. This is only an example and is not specifically limited.

[0070] As an optional implementation manner, the method also includes: uploading the detection results to the cloud.

[0071] In this embodiment, the cloud may be at least one server, which is merely an example and not specifically limited.

[0072] In this embodiment, the obtained test results are uploaded to the cloud so that the test results can be downloaded in real time and verified to be accurate.

[0073] As an optional implementation manner, step S101, obtaining video data of steel bars in a bundled state, includes: monitoring whether the steel bars are in a bundled state to obtain a monitoring result; in response to the monitoring result that the steel bars are in a bundled state, obtaining video data.

[0074] In this embodiment, the above monitoring result may be that the steel bars are in a bundled state, or may be that the steel bars are not in a bundled state, which is merely an example and not specifically limited here.

[0075] In this embodiment, whether the steel bars are in a bundled state is monitored to obtain a monitoring result; in response to the monitoring result being that the steel bars are in a bundled state, video data is acquired. For example, by real-time monitoring of whether the steel bars are in a bundled state, the monitoring result can be obtained, and then the obtained monitoring result is analyzed. If the monitoring result is analyzed to be that the steel bars are in a bundled state, the acquisition device is controlled to collect the video data of the steel bars in the bundled state. If the monitoring result is analyzed to be that the steel bars are not in a bundled state, there is no need to control the acquisition device to collect the video data of the steel bars in the bundled state, and only real-time monitoring of whether the steel bars are in a bundled state needs to be continued.

[0076] In this embodiment, when it is determined that the steel bars are in a bundled state, video data of the steel bars in the bundled state is obtained, and then at least one video frame data to be tested in the video data is determined, and then the determined at least one video frame data to be tested is input into the quality detection model for convolution, so that the feature information of the steel bars can be obtained, and by fusing the first feature information and the second feature information in the feature information, the target feature information can be obtained, and finally, by detecting the target feature information, the detection result can be obtained, that is, the quality of the steel bar bundling can be obtained, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling.

[0077] Example 2

[0078] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0079] In the quality inspection of steel bar bundling, it is often difficult to perform a global visual scan of the on-site image, and it is also difficult to quickly notice which information is important and which is unimportant in the scanned image, resulting in the technical problem of low efficiency of the quality inspection of steel bar bundling.

[0080] However, an embodiment of the present invention proposes a method for detecting the quality of steel bar bundling. By enhancing the feature information extracted by the deep network, the steel bar bundling situation can be quickly detected and the corresponding detection results can be obtained, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling.

[0081] Figure 2is a flow chart of a method for detecting steel bar bundling data in a video according to an embodiment of the present invention. Figure 2 As shown, the method may include the following steps:

[0082] Step S201, collecting and preprocessing video data of steel bars in a bundled state.

[0083] In the technical solution provided in the above step S201 of the present invention, by controlling the drone, video data of the steel bar bundling site can be obtained, and each frame of the steel bar bundling image in the video data is preprocessed. All preprocessed steel bar bundling images are 416×416 pixels and have RGB three channels.

[0084] Step S202, extracting image features of the steel bars in a bundled state.

[0085] In the technical solution provided in the above step S202 of the present invention, after the video data of the steel bars in the bundled state are collected and preprocessed, the image features of the steel bars in the bundled state are extracted.

[0086] In this embodiment, the 416×416×3 steel bar bundling image obtained by step S201 is input into the backbone network (RESInet) of the lightweight attention module (CBAMy)-receptive field module (RFB) algorithm, and the image features of the steel bars in the bundled state can be extracted. That is, by inputting the 416×416×3 steel bar bundling image into the R2 layer of the backbone network, and performing convolution and normalization processing, a 416×416×32 activation map can be obtained, and then by inputting the activation map into the R3 layer, a 104×104×64 feature map can be output, and the feature map output by the R3 layer is copied, one is input into the R4 layer, and the other is processed and fused with the output of the last layer R7 of RESInet, and then enters the processing of enhancing the image features of the steel bar bundling area, thereby obtaining the final result of RESInet, wherein the 104×104×64 feature map can include shallow features of the image.

[0087] Step S203, obtaining an image of the steel bar bundling area.

[0088] In the technical solution provided in the above step S203 of the present invention, after the image features of the steel bars in the bundled state are extracted, an image of the steel bar bundling area is obtained.

[0089] In this embodiment, the shallow feature image of the steel bar bundling outputted from the R3 layer is inputted into the R4 layer, and a 104×104×64 image of the steel bar bundling area can be outputted from the R4 layer.

[0090] Step S204, enhancing the image features of the steel bar bundling area.

[0091] In the technical solution provided in the above step S204 of the present invention, after the image of the steel bar bundling area is obtained, the image features of the steel bar bundling area are enhanced.

[0092] In this embodiment, the image of the steel bar bundling area is input into the R5 layer, R6 layer and R7 layer of RESInet, wherein the outputs of the R5 layer and the R6 layer are not only input into the R6 layer and the R7 layer respectively, but also processed together with the processing result of the R7 layer to be the final output of the R7 layer, that is, the image feature enhanced map of the steel bar bundling area.

[0093] Step S205, inputting the enhanced image features into the steel bar binding point classifier to perform three-classification prediction.

[0094] In the technical solution provided in the above step S205 of the present invention, after the image features of the steel bar bundling area are enhanced, the enhanced image features are input into the steel bar bundling point classifier for three-classification prediction.

[0095] In this embodiment, the results obtained by the above three-category prediction may be three result types: no steel bar bundling, poor steel bar bundling quality, and good steel bar bundling quality.

[0096] Step S206, dividing the steel bar bundling data set.

[0097] In the technical solution provided in the above step S206 of the present invention, after the enhanced image features are input into the steel bar bundling point classifier and three-classification prediction is performed, the steel bar bundling data set is divided.

[0098] In this embodiment, by dividing the steel bar bundling data set in proportion, a training set, a validation set and a test set can be obtained, wherein the training set, the validation set and the test set are mutually exclusive data, the training set can account for 70% of the total data set, and the validation set and the test set can each account for 15% of the total data set. This is only an example and is not specifically limited.

[0099] Step S207, training and verifying the CBAMy-RFB model.

[0100] In the technical solution provided in the above step S207 of the present invention, after the steel bar bundling data set is divided, the CBAMy-RFB model is trained and verified.

[0101] In this embodiment, the training parameters in the deep learning-based steel bar bundling quality detection model are first initialized, and then the training set and the verification set are respectively loaded into the deep learning-based steel bar bundling quality detection model to perform training and update the network parameters, and then the test set is input into the trained deep learning-based steel bar bundling quality detection model to verify the model.

[0102] Step S208, applying the CBAMy-RFB model to automatically detect the steel bar bundling data in the video data.

[0103] In the technical solution provided in the above step S208 of the present invention, after the CBAMy-RFB model is trained and verified, the CBAMy-RFB model is applied to automatically detect the steel bar bundling data in the video data.

[0104] In this embodiment, the above-mentioned steel bar bundling data can be used to represent the steel bar bundling image data extracted frame by frame from the video data.

[0105] In this embodiment, the steel bar bundling image data extracted frame by frame is input into the CBAMy-RFB model for detection, so that the steel bar bundling detection result can be obtained, that is, the quality of the steel bar bundling can be obtained.

[0106] Figure 3 is a flow chart of a classification method based on the CBAMy-RFB model according to an embodiment of the present invention, such as Figure 3 As shown, the method may include the following steps:

[0107] Step S301, input the preprocessed steel bar bundling image into the CBAMy-RFB model.

[0108] Step S302, extracting the steel bar bundling image features.

[0109] In the technical solution provided in the above step S302 of the present invention, after the pre-processed steel bar bundling image is input into the CBAMy-RFB model, the steel bar bundling image features are extracted.

[0110] Step S303, obtaining an image of the steel bar bundling area.

[0111] In the technical solution provided in the above step S303 of the present invention, after extracting the steel bar bundling image features, an image of the steel bar bundling area is obtained.

[0112] Step S304, enhancing the image features of the steel bar bundling area.

[0113] In the technical solution provided in the above step S304 of the present invention, after the image of the steel bar bundling area is obtained, the image features of the steel bar bundling area are enhanced.

[0114] Step S305: input the enhanced image features into a classifier for classification.

[0115] In the technical solution provided in the above step S305 of the present invention, after the image features of the steel bar bundling area are enhanced, the enhanced image features are input into a classifier for classification.

[0116] Step S306: performing non-maximum suppression (NMS) on the classification results.

[0117] In the technical solution provided in the above step S306 of the present invention, after the enhanced image features are input into the classifier for classification, non-maximum suppression (NMS) is performed on the classification results.

[0118] Step S307, outputting the suppressed classification result.

[0119] In the technical solution provided in the above step S307 of the present invention, after non-maximum suppression is performed on the classification result, the suppressed classification result is output.

[0120] Figure 4(a) is a schematic diagram of the structure of a CBAMy-RFB model according to an embodiment of the present invention. As shown in Figure 4(a), the structure may include: a CBAMy module 401, a RESInet network 402 and an output layer 403, wherein the R4 layer may include: a convolution (CBM) layer 4011, a merging (Concat) layer 4012, convolution layers 4013 to 4016, an adding layer (Add) 4017 and a compression (CBAM) layer 4018, and the RESInet network 402 may include: an R1 layer, an R2 layer, an R3 layer, an R4 layer, an R5 layer, an R6 layer and an R7 layer.

[0121] FIG4( b ) is a schematic diagram of a backbone network RESInet incorporating the residual idea according to an embodiment of the present invention. As shown in FIG4( b ), the RESInet network may include: an R1 layer, an R2 layer, an R3 layer, an R4 layer, an R5 layer, an R6 layer, and an R7 layer.

[0122] In this embodiment, by incorporating the residual idea into RESInet, the preprocessed steel bar bundling image can be received as input, and feature extraction can be performed on the preprocessed steel bar bundling image. Incorporating the residual idea into RESInet means copying the output of the R3 layer including the shallow features of the image, one copy enters the R4 layer, and the other copy is processed and fused with the output of the last layer R7 of RESInet, and then enters the process of enhancing the image features of the steel bar bundling area, thereby obtaining the final result of RESInet.

[0123] In this embodiment, the process of constructing a backbone network incorporating the residual idea may include the following steps:

[0124] Step A: Input the steel bar bundling image into RESInet.

[0125] In the technical solution provided in the above step A of the present invention, the convolutional layer that the above-mentioned steel bar bundling image first enters may be the R1 layer, that is, the input layer, and the steel bar bundling image may be a 416×416 pixel, RGB three-channel image.

[0126] In step B, after convolution and standardization operations of the R2 layer, the steel bar bundling image is converted into an activation map of 416×416×32.

[0127] In the technical solution provided in the above step B of the present invention, after the steel bar bundling image is input into RESInet, the steel bar bundling image is converted into an activation map of 416×416×32 after convolution and standardization operations of the R2 layer.

[0128] In step C, the activation map is input into the R3 layer, and the shallow feature image of steel bar bundling output by the R3 layer is directly fused with the deep feature image of steel bar bundling output by the last layer R7 of RESInet. That is to say, both the shallow feature image of steel bar bundling and the deep feature image of steel bar bundling are directly input into the feature fusion module (MSPP).

[0129] In the technical solution provided in the above step C of the present invention, after the steel bar bundling image is converted into an activation map of 416×416×32, the activation map is input into the R3 layer, and the shallow feature image of the steel bar bundling output by the R3 layer is directly fused with the deep feature image of the steel bar bundling output by the last layer R7 of RESInet.

[0130] In this embodiment, the input of the above-mentioned MSPP can be the shallow features of the steel bar bundling point image output by the R3 layer and the deep features of the steel bar bundling point image output by the R7 layer. The above-mentioned MSPP can enable more shallow features of the steel bar bundling image to be integrated into the deep features of the steel bar bundling image.

[0131] In step D, the front and back convolutional layers of MSPP are increased to 7 layers, and the depth of the network is increased to obtain more target information.

[0132] In the technical solution provided in the above step D of the present invention, after the shallow feature image of steel bar bundling output by the R3 layer is directly fused with the deep feature image of steel bar bundling output by the last layer R7 of RESInet, the front and rear convolutional layers of MSPP are increased to 7 layers, and the depth of the network is increased to obtain more target information.

[0133] In this embodiment, in the MSPP module, 7 layers of convolution are performed on the shallow features of the steel bar bundling point image output by the R3 layer and the deep features of the steel bar bundling point image output by the R7 layer (corresponding to Conv*7 as shown in FIG. 4( b)). Then, by dividing the input feature map into different scales, four sub-regions can be obtained, and their interval sizes can be respectively: 5×5, 9×9, 13×13, and 17×17. This is only used as an example and is not specifically limited.

[0134] In step E, average pooling is performed on each sub-region to obtain a vector of fixed length.

[0135] In the technical solution provided in the above step E of the present invention, after increasing the front and rear convolutional layers of MSPP to 7 layers and increasing the depth of the network to obtain more target information, average pooling is performed on each sub-region to obtain a vector of fixed length.

[0136] In step F, the vectors corresponding to all sub-regions are merged and convolved with 7 layers to obtain the feature map of the steel bar bundling points, which is used as the input for the image operation of the enhanced steel bar bundling area.

[0137] In the technical solution provided in the above step F of the present invention, after average pooling is performed on each sub-region to obtain a vector of a fixed length, the vectors corresponding to all sub-regions are merged and subjected to 7-layer convolution to obtain a feature map of the steel bar bundling points as input for enhancing the image operation of the steel bar bundling area.

[0138] Figure 4(c) is a schematic diagram of a backbone network RESInet with a CBAM module added according to an embodiment of the present invention. As shown in Figure 4(c), the RESInet network may include: R1 layer, R2 layer, R3 layer, R4 layer, R5 layer, R6 layer and R7 layer, and the R4 layer may include: CBM block 4011, Concat block 4012, CBM blocks 4013 to 4016, add block 4017 and CBAM block 4018.

[0139] In this embodiment, by adding CBAM to RESInet, it is possible to receive the steel bar bundling point feature map as input, identify each steel bar bundling point area in the steel bar bundling image, and independently construct a steel bar bundling area image, wherein the steel bar bundling image can be converted into a 208×208 pixel, 64-channel steel bar bundling shallow feature image processed by the R3 layer, and after being processed by the steel bar bundling point information integration method, it can be converted into a 104×104 pixel, 128-channel steel bar bundling area image.

[0140] In this embodiment, obtaining the steel bar bundling area image may include the following steps:

[0141] In the first step, the shallow feature image of steel bar bundling with 208×208 pixels and 64 channels obtained by the R3 layer is input into the R4 layer.

[0142] In the second step, the CBM block processes the input shallow feature image of the steel bar bundling through the convolution layer (Conv) for feature extraction, and the convolution output is normalized through the batch normalization layer (BatchNorm), and the normalized output is then passed through the activation function (Mish).

[0143] In the technical solution provided in the second step of the present invention, after the shallow feature image of steel bar bundling with 208×208 pixels and 64 channels obtained by the R3 layer is input into the R4 layer, it is processed by the CBM block, and the shallow feature image of steel bar bundling is input through the convolution layer (Conv) for feature extraction, and the convolution output is normalized by the batch normalization layer (BatchNorm), and the normalized output is then passed through the activation function (Mish).

[0144] In this embodiment, the processing result can be a feature map after convolution, batch normalization and activation function processing, and the feature map is copied, one copy enters the a direction processing process for subsequent processing, and the other copy is processed by the b direction CBM block.

[0145] In the third step, the CBAM module is embedded into the next step of the adding layer processing in the R4 layer of RESInet, that is, after the adding layer performs feature fusion on the rebar bundling image.

[0146] In the technical solution provided in the third step of the present invention, after being processed by the CBM block, the input shallow feature image of the steel bar bundling is subjected to feature extraction through the convolution layer, and the convolution output is normalized through the batch normalization layer. After the normalized output is subjected to the activation function, the CBAM module is embedded in the next step of adding the layer in the R4 layer of RESInet.

[0147] In the fourth step, along the processing direction of a, it is convolved by two CBM layers and normalized by an Add layer before being input into CBAM.

[0148] In the technical solution provided in the fourth step of the present invention, after the CBAM module is embedded in the next step of the Add layer processing in the R4 layer of RESInet, it is input into the CBAM after being convolutionally processed by two CBM layers and normalized by an Add layer along the processing flow direction a.

[0149] In the fifth step, further feature extraction is performed on the rebar bundling image sequence based on the two effective preliminary features of channel attention and spatial attention of CBAM, and then the spatial information is aggregated to generate a channel attention map, while the size of the feature map is halved until a 1024-channel feature map of the rebar bundling image is obtained.

[0150] In the technical solution provided in the fifth step of the present invention, after being convolved by two CBM layers and normalized by an Add layer along the processing flow direction a and input into CBAM, the rebar bundling image is sequentially subjected to further feature extraction using two effective preliminary features of CBAM, namely, channel attention and spatial attention. The spatial information is then aggregated to generate a channel attention map, and the size of the feature map is halved until a 1024-channel rebar bundling image feature map is obtained.

[0151] In the sixth step, the final feature map is passed through a global average pooling layer and a fully connected layer, and then the channel attention map is weighted with the input feature map to obtain the input feature map of the spatial attention module, the channel dimension of the input feature map is compressed, and the input feature map of the spatial attention module is spliced ​​into a feature descriptor. The spliced ​​feature descriptor is passed through a convolution layer and an activation function (Sigmoid) to obtain a spatial attention feature map to highlight the key areas, and finally the obtained spatial attention feature map is bitwise multiplied.

[0152] In the technical solution provided in the sixth step of the present invention, after obtaining a 1024-channel steel bar bundling image feature map, the vectors corresponding to all sub-regions are merged and subjected to 7-layer convolution to obtain a steel bar bundling point feature map as input for enhancing the steel bar bundling region image operation.

[0153] In this embodiment, the compression operation considers both the maximum pooling operation and the average pooling operation.

[0154] In the seventh step, after being processed by a CBM block, the final result of the processing flow in direction a is obtained as shown in Figure 4(c), which is then merged with the result in direction b to obtain a 104×104 pixel, 128-channel steel bar bundling area image. The obtained 104×104 pixel, 128-channel steel bar bundling area image is then input into the R5 layer of RESInet for convolution.

[0155] In the technical solution provided in the seventh step of the present invention, after the obtained spatial attention feature map is bitwise multiplied and processed by a CBM block, the final result of the processing flow in the a direction as shown in FIG4(c) is obtained, and then merged with the result in the b direction to obtain a 104×104 pixel, 128-channel steel bar bundling area image, and then the obtained 104×104 pixel, 128-channel steel bar bundling area image is input into the R5 layer of RESInet for convolution, thereby preparing for the enhancement of the feature extraction performance of the subsequent deep network.

[0156] Figure 4(d) is a schematic diagram of an image feature enhancement module based on a deep network according to an embodiment of the present invention. As shown in Figure 4(d), the RESInet network may include: R1 layer, R2 layer, R3 layer, R4 layer, R5 layer, R6 layer and R7 layer.

[0157] In this embodiment, a deep network feature extraction performance enhancement module is constructed, and the steel bar bundling area image processed by the R5 layer and the R6 layer convolution and the obtained steel bar bundling point feature map are used as input. The deep network feature extraction performance enhancement module can effectively improve the robustness of the network model feature extraction, thereby achieving the effect of enhancing the feature extraction performance of the deep network. The deep network feature extraction performance enhancement module can be specifically shown in Figure 4(d).

[0158] In this embodiment, enhancing the image features of the steel bar bundling area may include the following steps:

[0159] Step 1: Integrate the RFB module in RESInet.

[0160] In the technical solution provided in the above step 1 of the present invention, the RFB module is used after the first upsampling feature fusion, the second upsampling feature fusion and the final feature fusion of the CBAMy-RFB algorithm to enhance the three size detection heads of 52×52, 26×26 and 13×13 and input feature information.

[0161] Step 2: Add the generated steel bar bundling point feature map to the enhanced module of deep network feature extraction performance, and then copy the generated steel bar bundling point feature map.

[0162] In the technical solution provided in the above step 2 of the present invention, after combining the RFB module in RESInet, the generated steel bar bundling point feature map is added to the enhancement module of deep network feature extraction performance, and then the generated steel bar bundling point feature map is copied.

[0163] In this embodiment, the above copy operation can be specifically as follows: one copy enters the processing flow indicated by direction a as shown in FIG. 4( d ), and the other copy enters the processing flow indicated by direction b as shown in FIG. 4( d ).

[0164] Step three, the generated steel bar bundling point feature map enters the processing flow in direction a.

[0165] In the technical solution provided in the above step three of the present invention, after adding an enhancement module of deep network feature extraction performance to the generated steel bar bundling point feature map and then copying a copy of the generated steel bar bundling point feature map, the generated steel bar bundling point feature map enters the processing flow in direction a.

[0166] In this embodiment, the generated steel bar bundling point feature map is first subjected to a layer of convolution and upsampling operations (UpSamping), and then the steel bar bundling point feature map after convolution and upsampling and the output result of the R6 layer are merged, processed by the RFB module and convolved by 5 convolution layers. Then, one copy of the data after merging, processing by the RFB module and convolution by 5 convolution layers is copied to enter the b process, and the other copy continues the processing process in the a direction to perform a round of one layer of convolution and upsampling operations.

[0167] Step 4: The result of the second step in direction a is merged with the output result of layer R5, input into the RFB module for processing and convolution of 5 convolution layers, and then enters the processing flow of direction b as shown in Figure 4(d), and then downsampling is performed.

[0168] In the technical solution provided in the above step 4 of the present invention, after the generated steel bar bundling point feature map enters the processing flow in the a direction, the result of the second step in the a direction is merged with the output result of the R5 layer, input into the RFB module for processing and convolved with 5 convolution layers, and then enters the processing flow in the b direction as shown in FIG4(d), and then down-sampling is performed.

[0169] Step 5: Merge and perform RFB operation on the down-sampling result and the steel bar bundling point feature map output in the first step in the c direction.

[0170] In the technical solution provided in the above step five of the present invention, after downsampling, the result obtained by downsampling and the steel bar binding point feature map output in the first step in the c direction are merged and subjected to RFB operation.

[0171] Step six, input the obtained feature image of the steel bar bundling point with deep network features into the steel bar bundling point classifier module for classification.

[0172] In the technical solution provided in the above step six of the present invention, after merging and performing RFB operation on the result obtained by downsampling and the steel bar bundling point feature map output in the first step in the c direction, the obtained steel bar bundling point feature image with deep network characteristics is input into the steel bar bundling point classifier module for classification.

[0173] In this embodiment, video data of steel bars in a bundled state is collected and preprocessed, and then image features of the steel bars in a bundled state are extracted, and then an image of the steel bar bundling area is obtained, and the image features of the steel bar bundling area are enhanced. Finally, the enhanced image features are input into a steel bar bundling point classifier to perform a three-classification prediction, thereby determining the quality of the steel bar bundling, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling.

[0174] Example 3

[0175] According to an embodiment of the present invention, a quality detection device for steel bar bundling is also provided. It should be noted that the quality detection device for steel bar bundling can be used to execute a quality detection method for steel bar bundling in Example 1.

[0176] Figure 5 Schematic diagram of a steel bar bundling quality inspection device according to an embodiment of the present invention. Figure 5 As shown, the steel bar bundling quality detection device 500 may include: an acquisition unit 501, a determination unit 502, a convolution unit 503, a fusion unit 504 and a detection unit 505.

[0177] The acquisition unit 501 is used to acquire video data of the steel bars in a bundled state.

[0178] The determination unit 502 is configured to determine at least one video frame data to be tested in the video data.

[0179] The convolution unit 503 is used to input the video frame data to be tested into the quality detection model for convolution to obtain the characteristic information of the steel bar, wherein the quality detection model is obtained by classification training based at least on the image data samples, and the image data samples are obtained by dividing the video data samples.

[0180] The fusion unit 504 is used to fuse the first feature information and the second feature information in the feature information to obtain the target feature information, wherein the first feature information is used to represent the feature information output from the initial extraction layer in the quality detection model, and the second feature information is used to represent the feature information output from the final extraction layer in the quality detection model.

[0181] The detection unit 505 is used to detect the target feature information and obtain a detection result, wherein the detection result is used to indicate the quality of the steel bar bundling.

[0182] Optionally, the determination unit 502 may include: a traversal module, configured to traverse at least one video frame data in the video data to obtain at least one initial video frame data; and a first determination module, configured to determine the video frame data to be tested based on the initial video frame data.

[0183] Optionally, the determination module may include: a labeling submodule, which is used to label the initial video frame data according to the target boundary to obtain the video frame data to be tested, wherein the target boundary is used to display the steel bars in a bundled state.

[0184] Optionally, the convolution unit 503 may include: a conversion module, used to input the video frame data to be tested into the preprocessing layer in the quality detection model for conversion to obtain the converted video frame data to be tested; a convolution module, used to input the converted video frame data to be tested into the initial extraction layer for convolution to obtain the first feature information; a second determination module, used to determine the second feature information based on the first feature information; and a third determination module, used to determine the first feature information and the second feature information as feature information.

[0185] Optionally, the conversion module may include: a first convolution submodule, used to input the video frame data to be tested into the preprocessing layer for convolution to obtain the convolved video frame data to be tested; a conversion submodule, used to perform interval conversion on the convolved video frame data to be tested in the preprocessing layer to obtain the converted video frame data to be tested.

[0186] Optionally, the second determination module may include: a second convolution submodule, used to input the first feature information into a next extraction layer adjacent to the initial extraction layer for convolution to obtain the first feature information after convolution, wherein the next extraction layer includes at least one extraction layer; a third convolution submodule, used to input the first feature information after convolution into a final extraction layer for convolution to obtain the second feature information, wherein the final extraction layer is adjacent to the next extraction layer.

[0187] Optionally, the detection unit 505 may include: a detection module, used to detect target feature information using a detection sub-model to obtain a detection result, wherein the detection sub-model is set in the quality detection model, and the detection sub-model is a classification model set based on a regression algorithm.

[0188] Optionally, the steel bar bundling quality inspection device 500 may include: an uploading unit, configured to upload the inspection result to the cloud.

[0189] Optionally, the acquisition unit 501 may further include: a monitoring module for monitoring whether the steel bars are in a bundled state to obtain a monitoring result; and an acquisition module for acquiring video data in response to the monitoring result that the steel bars are in a bundled state.

[0190] In this embodiment, an acquisition unit is used to acquire video data of steel bars in a bundled state; a determination unit is used to determine at least one video frame data to be tested in the video data; a convolution unit is used to input the video frame data to be tested into a quality detection model for convolution to obtain feature information of the steel bars, wherein the quality detection model is obtained by classification training based on at least image data samples, and the image data samples are obtained by dividing the video data samples; a fusion unit is used to fuse first feature information and second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent feature information output from an initial extraction layer in the quality detection model, and the second feature information is used to represent feature information output from a final extraction layer in the quality detection model; a detection unit is used to detect the target feature information to obtain a detection result, wherein the detection result is used to represent the quality of the steel bar bundling, thereby achieving the purpose of performing a global visual scan of the on-site image, solving the technical problem of low efficiency of quality detection of steel bar bundling, and achieving the technical effect of improving the efficiency of quality detection of steel bar bundling.

[0191] Example 4

[0192] According to an embodiment of the present invention, a computer-readable storage medium is further provided, the storage medium comprising a stored program, wherein the program executes the quality detection method for steel bar bundling in Example 1.

[0193] Example 5

[0194] According to an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein the program executes the quality detection method for steel bar bundling in Example 1 when the program is run by the processor.

[0195] Example 6

[0196] According to an embodiment of the present invention, a computer program product is further provided. The computer program product includes a computer program. When the computer program is executed by a processor, the quality detection method for steel bar bundling in Embodiment 1 of the present invention is implemented.

[0197] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0198] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0199] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0200] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0201] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0202] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the relevant technology, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0203] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for detecting the quality of steel bar bundling, characterized in that: include: Acquire video data of the steel bars in a bundled state; Determine at least one video frame data to be tested in the video data; Input the video frame data to be tested into a quality inspection model for convolution to obtain feature information of the steel bar, wherein the quality inspection model is obtained by classification training based at least on image data samples, and the image data samples are obtained by dividing video data samples; fusing first feature information and second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent feature information output from an initial extraction layer in the quality detection model, and the second feature information is used to represent feature information output from a final extraction layer in the quality detection model; Detecting the target feature information to obtain a detection result, wherein the detection result is used to indicate the quality of the steel bar bundling; Wherein, the video frame data to be tested is input into the quality inspection model for convolution to obtain the characteristic information of the steel bar, including: inputting the video frame data to be tested into the preprocessing layer in the quality inspection model for conversion to obtain the converted video frame data to be tested; inputting the converted video frame data to be tested into the initial extraction layer for convolution to obtain the first characteristic information; inputting the first characteristic information into the next extraction layer adjacent to the initial extraction layer for convolution to obtain the first characteristic information after convolution, wherein the next extraction layer includes at least one extraction layer; inputting the first characteristic information after convolution into the final extraction layer for convolution to obtain the second characteristic information, wherein the final extraction layer is adjacent to the next extraction layer; and determining the first characteristic information and the second characteristic information as the characteristic information.

2. The method according to claim 1, characterized in that Determining at least one video frame data to be tested in the video data includes: Traversing at least one video frame data in the video data to obtain at least one initial video frame data; Based on the initial video frame data, the video frame data to be tested is determined.

3. The method according to claim 2, characterized in that Determining the video frame data to be tested based on the initial video frame data includes: The initial video frame data is annotated according to the target boundary to obtain the video frame data to be tested, wherein the target boundary is used to display the steel bars in the bundled state.

4. The method according to claim 1, characterized in that: Inputting the video frame data to be tested into the preprocessing layer in the quality detection model for conversion to obtain the converted video frame data to be tested, including: Inputting the video frame data to be tested into the preprocessing layer for convolution to obtain the video frame data to be tested after convolution; In the preprocessing layer, the convolved video frame data to be tested is interval-converted to obtain the converted video frame data to be tested.

5. The method according to claim 1, characterized in that Detecting the target feature information to obtain a detection result includes: The target feature information is detected by using a detection sub-model to obtain the detection result, wherein the detection sub-model is set in the quality detection model, and the detection sub-model is a classification model set based on a regression algorithm.

6. The method according to claim 1, further comprising: The detection results are uploaded to the cloud.

7. The method according to claim 1, characterized in that Get video data of steel bars in bundled state, including: Monitoring whether the steel bars are in the bundled state and obtaining a monitoring result; In response to the monitoring result that the steel bars are in the bundled state, the video data is acquired.

8. A quality inspection device for steel bar bundling, characterized in that: include: An acquisition unit, used for acquiring video data of the steel bars in a bundled state; A determination unit, configured to determine at least one video frame data to be tested in the video data; A convolution unit, used for inputting the video frame data to be tested into a quality detection model for convolution to obtain feature information of the steel bar, wherein the quality detection model is obtained by performing classification training based at least on image data samples, and the image data samples are obtained by dividing video data samples; a fusion unit, configured to fuse first feature information and second feature information in the feature information to obtain target feature information, wherein the first feature information is used to represent feature information output from an initial extraction layer in the quality detection model, and the second feature information is used to represent feature information output from a final extraction layer in the quality detection model; A detection unit, used to detect the target feature information to obtain a detection result, wherein the detection result is used to indicate the quality of the steel bar bundling; Wherein, the convolution unit is used to input the video frame data to be tested into the quality inspection model for convolution to obtain the characteristic information of the steel bar by executing the following steps: input the video frame data to be tested into the preprocessing layer in the quality inspection model for conversion to obtain the converted video frame data to be tested; input the converted video frame data to be tested into the initial extraction layer for convolution to obtain the first characteristic information; input the first characteristic information into the next extraction layer adjacent to the initial extraction layer for convolution to obtain the first characteristic information after convolution, wherein the next extraction layer includes at least one extraction layer; input the first characteristic information after convolution into the final extraction layer for convolution to obtain the second characteristic information, wherein the final extraction layer is adjacent to the next extraction layer; determine the first characteristic information and the second characteristic information as the characteristic information.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is executed, the device where the computer-readable storage medium is located is controlled to execute the steel bar bundling quality detection method according to any one of claims 1 to 7.

10. A processor, characterized in that: The processor is used to run a program, wherein the program, when run by the processor, executes the steel bar bundling quality detection method according to any one of claims 1 to 7.

11. A computer program product, comprising a computer program, characterized in that When the computer program is executed by a processor, the quality detection method for steel bar bundling described in any one of claims 1 to 7 is implemented.

Citation Information

Patent Citations

  • Data processing method and device, storage medium and equipment

    CN114332678A

  • Video data processing method and device, computer equipment and storage medium

    CN114782853A

  • Reinforcing steel bar binding state detection model training method, recognition method and equipment

    CN115272691A

  • Global and local feature reconstruction network-based medical image segmentation method

    US20230274531A1