Method, device, storage medium and processor for determining binding information

Abstract

The present invention discloses a method, device, storage medium and processor for determining binding information. The method comprises: obtaining image data of steel bar binding; extracting channel features of color channels from the image data; determining a first image feature map of steel bar binding based on the channel features, wherein the first graphic feature map is used to represent binding information of steel bar binding; extracting a first detail feature map of steel bar binding from the first image feature map, wherein the first detail feature map is used to represent binding information of binding points of sizes in the image data that are smaller than a size threshold; detecting the first detail feature map to obtain target binding information of steel bar binding. The present invention solves the technical problem of low recognition accuracy of binding information.

Description

Technical Field

[0001] The present invention relates to the technical field of image processing, and in particular to a method, device, storage medium and processor for determining binding information. Background Art

[0002] At present, steel bar tying is an important construction process in construction and civil engineering. The tying quality of steel bar tying is directly related to the safety and durability of concrete structure. Testing the quality of steel bar tying can help construction personnel check and adjust the quality of steel bar tying, ensure that the tying is firm and the connection is good, and improve the seismic resistance and bearing capacity of the overall structure.

[0003] In the related art, a depth camera is usually used to obtain image data of the steel bars to be tied; the pixel coordinates of the intersection points of the steel bars are determined based on the image data to determine the tying information. However, when there are multiple layers of steel bars in the image data, problems such as incorrect identification and calibration of the steel bar tying points are prone to occur, resulting in a technical problem of low accuracy in identifying the tying information.

[0004] Currently, no effective solution has been proposed to the technical problem of low accuracy in identifying binding information. Summary of the invention

[0005] Embodiments of the present invention provide a method, device, storage medium, and processor for determining binding information, so as to at least solve the technical problem of low recognition accuracy of binding information.

[0006] According to one aspect of an embodiment of the present invention, a method for determining binding information is provided, and the method may include: acquiring image data of steel bar binding; extracting channel features of a color channel from the image data; determining a first image feature map of the steel bar binding based on the channel features, wherein the first image feature map is used to represent the binding information of the steel bar binding; extracting a first detail feature map of the steel bar binding from the first image feature map, wherein the first detail feature map is used to represent the binding information of binding points of a size smaller than a size threshold in the image data; and detecting the first detail feature map to obtain target binding information of the steel bar binding.

[0007] Optionally, the first detail feature map is detected to obtain target binding information of steel bar binding, including: detecting the first detail feature map to obtain first binding information; converting the first detail feature map through a channel feature extractor to obtain a second image feature map, wherein the granularity of the second image feature map is smaller than that of the first image feature map; extracting a second detail feature map of the steel bar binding in the second image feature map through a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than that of the first detail feature map; detecting the second detail feature map to obtain second binding information; and determining the target binding information based on the second detail feature map, the first binding information and the second binding information.

[0008] Optionally, the target binding information includes at least the binding position of the steel bar binding and the confidence of the binding position, and the target binding information is obtained based on the second detail feature map, the first binding information and the second binding information, including: converting the second detail feature map through the channel feature extractor to obtain a third image feature, wherein the granularity of the third image feature map is smaller than that of the second image feature map; extracting the third detail feature map of the steel bar binding in the third image feature map through the SCFEM module, wherein the granularity of the third detail feature map is smaller than that of the second detail feature map; detecting the third detail feature map to obtain third binding information; and fusing the first binding information, the second binding information and the third binding information for the confidence to obtain the target binding information.

[0009] Optionally, based on the channel features, a first image feature map of the steel bar binding is determined, including: dividing the channel features according to the channel dimension to obtain two branch channel features; using multiple convolution kernels to extract features of a first branch channel feature of the two branch channel features to obtain multiple sub-branch channel features; using an activation function to activate the multiple sub-branch channel features to obtain a sub-image feature corresponding to the first branch channel feature; converting the sub-image feature and a second branch channel feature of the two branch channel features to obtain the first image feature map, wherein the second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

[0010] Optionally, the sub-image feature and the second branch channel feature of the two branch channel features are converted to obtain the first image feature map, including: fusing the sub-image feature and the second branch channel feature to obtain a fused feature; performing convolution processing on the fused feature; and extracting the first image feature map from the fused feature after the convolution processing.

[0011] Optionally, a first detail feature map of the steel bar binding is extracted from the first image feature map, including: compressing the first image feature map to obtain a compressed feature map; activating the compressed feature map, and filtering the activated compressed feature map to obtain a first output feature map; pooling the compressed feature map, and converting the pooled compressed feature map to obtain a second output feature map, wherein a size of the first output feature map is the same as a size of the second output feature map; and obtaining the first detail feature map based on the first output feature map and the second output feature map.

[0012] Optionally, obtaining the first detail feature map based on the first output feature map and the second output feature map includes: splicing the first output feature map and the second output feature map according to a channel dimension to obtain the first detail feature map.

[0013] Optionally, the compressed feature map is pooled and the compressed feature map after pooling is converted to obtain a second output feature map, including: performing average pooling on the compressed feature map to obtain a first pooling feature map, and performing maximum pooling on the compressed feature map to obtain a second pooling feature map; respectively enhancing the first pooling feature map and the second pooling feature map to obtain a third pooling feature map and a fourth pooling feature map; upsampling the third pooling feature map and the fourth pooling feature map; fusing the upsampled third pooling feature map and the upsampled fourth pooling feature map to obtain the second output feature map.

[0014] According to one aspect of an embodiment of the present invention, another method for determining binding information is provided. The method may include: acquiring image data of steel bar binding; detecting the image data using a detection model to obtain target binding information of steel bar binding, wherein the detection model is used to extract channel features of a color channel from the image data, determine a first image feature map of steel bar binding based on the channel features, extract a first detail feature map of steel bar binding from the first image feature map, and detect the first detail feature to obtain target binding information, the first image feature map is used to represent the binding information of steel bar binding, the first detail feature map is used to represent the binding information of a size binding point in the image data whose size is less than a size threshold, and the binding information includes target binding information.

[0015] Optionally, the method may also include: detecting historical image data through a sub-detection model to obtain historical binding information; determining the intersection-and-union ratio between the historical binding information and the real binding information corresponding to the historical image data; constructing a loss function based on the intersection-and-union ratio; and using the loss function to adjust the model parameters of the sub-detection model to obtain a detection model.

[0016] According to another aspect of an embodiment of the present invention, a device for determining binding information is also provided, and the device may also include: a first acquisition unit, used to acquire image data of steel bar binding; a first extraction unit, used to extract channel features of a color channel from the image data; a determination unit, used to determine a first image feature map of steel bar binding based on the channel features, wherein the first graphic feature map is used to represent the binding information of the steel bar binding; a second extraction unit, used to extract a first detail feature map of the steel bar binding from the first image feature map, wherein the first detail feature map is used to represent the binding information of binding points of a size in the image data that is smaller than a size threshold; and a processing unit, used to detect the first detail feature to obtain target binding information of the steel bar binding.

[0017] According to one aspect of an embodiment of the present invention, another device for determining binding information is provided. The device may include: a second acquisition unit, used to acquire image data of steel bar binding; a detection unit, used to detect the image data using a detection model to obtain target binding information of steel bar binding, wherein the detection model is used to extract channel features of a color channel from the image data, determine a first image feature map of steel bar binding based on the channel features, extract a first detail feature map of steel bar binding from the first image feature map, and detect the first detail feature to obtain target binding information, the first image feature map is used to represent the binding information of steel bar binding, the first detail feature map is used to represent the binding information of a size binding point in the image data whose size is less than a size threshold, and the binding information includes target binding information.

[0018] According to another aspect of an embodiment of the present invention, a computer-readable storage medium is provided, which includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the method for determining binding information of an embodiment of the present invention.

[0019] According to another aspect of an embodiment of the present invention, a processor is further provided. The processor is used to run a program. When the program is run by the processor, the method for determining binding information of an embodiment of the present invention is executed.

[0020] According to another aspect of an embodiment of the present invention, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the method for determining binding information according to an embodiment of the present invention is implemented.

[0021] In an embodiment of the present invention, image data of steel bar binding is acquired; channel features of a color channel are extracted from the image data; based on the channel features, a first image feature map of steel bar binding is determined, wherein the first image feature map is used to represent the binding information of steel bar binding; from the first image feature map, a first detail feature map of steel bar binding is extracted, wherein the first detail feature map is used to represent the binding information of size binding points in the image data whose size is less than a size threshold; the first detail feature map is detected to obtain target binding information of steel bar binding. That is to say, in an embodiment of the present application, multiple rounds of processing are performed on the acquired image data, the channel features of the color channel are extracted from the image data, the channel features are further screened and filtered to obtain a first image feature map, and the first image feature map is further filtered and extracted to obtain a first detail feature map, thereby obtaining target binding information, thereby solving the technical problem of low recognition accuracy of binding information and achieving the technical effect of improving the recognition accuracy of binding information. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0023] Figure 1 is a flow chart of a method for determining binding information according to an embodiment of the present invention;

[0024] Figure 2 is a flow chart of another method for determining binding information according to an embodiment of the present invention;

[0025] Figure 3 is a flow chart of constructing a steel bar binding quality detection model based on deep learning according to an embodiment of the present invention;

[0026] Figure 4 is a flow chart of a data processing process of a channel feature extractor according to an embodiment of the present invention;

[0027] Figure 5 is a flow chart of a space-channel filtering module according to an embodiment of the present invention;

[0028] Figure 6 is a flow chart of a data processing process of a CPM module according to an embodiment of the present invention;

[0029] Figure 7 is a schematic diagram of a detection result according to an embodiment of the present invention;

[0030] Figure 8 is a schematic diagram of an EORA algorithm according to an embodiment of the present invention;

[0031] Fig. 9 is a schematic diagram of a device for determining binding information according to an embodiment of the present invention;

[0032] Fig.10 is a schematic diagram of another device for determining binding information according to an embodiment of the present invention. DETAILED DESCRIPTION

[0033] In order to enable those skilled in the art to better understand the scheme of the present invention, the technical scheme in the embodiments of the present invention will be clearly and completely described below in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions, for example, a process, method, system, product or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products or devices.

[0035] Example 1

[0036] According to an embodiment of the present invention, an embodiment of a method for determining binding information is provided. The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0037] Figure 1 FIG. 1 is a flow chart of a method for determining binding information according to an embodiment of the present invention. Figure 1 As shown, the method may include the following steps:

[0038] Step S102, obtaining image data of steel bar binding.

[0039] In the technical solution provided in the above step S102 of the present invention, in the identification task of steel bar binding, image data containing steel bar binding can be collected. Optionally, the image data can be collected by real-time on-site monitoring, camera shooting, drone shooting, etc., and can be used to confirm the binding conditions of the steel bar binding. The image data can be a steel bar binding picture, a steel bar binding video frame, etc. It should be noted that this is only an example, and there is no specific limitation on the method of collecting image data.

[0040] Optionally, the image data of the steel bar binding can be obtained through methods such as real-time on-site monitoring, camera shooting, and drone shooting. To improve the processing efficiency of the image data, the obtained image data can be preprocessed, for example, by resizing, removing noise, enhancing contrast, etc., so that subsequent detection algorithms can better process the image data.

[0041] Step S104: extracting channel features of color channels from the image data.

[0042] In the technical solution provided in the above step S104 of the present invention, the channel features of the color channel in the acquired image data are extracted. The channel features may be channel features of a steel bar binding image with a size of length (Height, referred to as H) × width (Width, referred to as W) × number of channels (Channel Count, referred to as C), which may be used to represent information of different positions and different features in the image data. The color channel may include a red channel, a green channel, and a blue channel, which is only used as an example here, and the type of the color channel is not specifically limited.

[0043] Optionally, different color channels can represent different features of the image data, such as color, texture, shape and other features. The channel features corresponding to the color channels in the image data are extracted, and the features of steel bar binding points of different sizes can be further distinguished based on the channel features.

[0044] For example, the image data can be input into a two-dimensional convolution_batch normalization_activation function (Conv2D_BN_SiLU) module, and the image data can be subjected to convolution operations, batch normalization, and SiLU activation function processing through at least one Conv2D_BN_SiLU module to obtain channel features corresponding to the color channel.

[0045] Step S106, determining a first image feature map of steel bar binding based on the channel features, wherein the first image feature map is used to represent binding information of the steel bar binding.

[0046] In the technical solution provided in the above step S106 of the present invention, the channel features can be screened and converted to obtain a first image feature map of steel bar binding. The first image feature map can be a steel bar binding feature map, which can be used to represent the binding information of the steel bar binding.

[0047] Optionally, channel features corresponding to different color channels can be extracted from the image data, multiple channel features can be screened, and the channel features that ultimately need to be processed can be determined. The screened channel features can be converted using a channel feature extractor (e.g., CFE) to obtain a first image feature map of steel bar binding.

[0048] Step S108, extracting a first detail feature map of steel bar binding from the first image feature map, wherein the first detail feature map is used to represent binding information of binding points of sizes smaller than a size threshold in the image data.

[0049] In the technical solution provided in the above step S108 of the present invention, a first detail feature map of steel bar binding can be extracted from the first image feature map. The first detail feature map can be a feature map that excludes interference information in the image, and can be used to represent the binding information of size binding points in the image data whose size is less than the size threshold. The size binding points can be small-sized binding points that are not obvious and difficult to identify in the image data.

[0050] Optionally, the first image feature is input into a space-channel filtering (Space-Channel Feature Extraction Module, abbreviated as SCFEM) module. Using the SCFEM module, interference information can be effectively filtered out from the channel and space in the first image feature map, thereby achieving the purpose of enhancing the size binding points in the image data, and further achieving the purpose of accurately identifying the binding points in the steel bar binding.

[0051] Step S110, detecting the first detail feature map to obtain target binding information of steel bar binding.

[0052] In the technical solution provided in step S110 of the present invention, the first detail feature map can be detected to obtain target binding information of the steel bar binding. The target binding information can be the detection result of the steel bar binding, which can be used to determine the position of each binding point, the position of the binding point missing, and the binding quality of the binding point.

[0053] Through the above steps S102 and S110 of the present application, the image data of the steel bar binding is obtained; the channel features of the color channel are extracted from the image data; based on the channel features, the first image feature map of the steel bar binding is determined, wherein the first graphic feature map is used to represent the binding information of the steel bar binding; from the first image feature map, the first detail feature map of the steel bar binding is extracted, wherein the first detail feature map is used to represent the binding information of the size binding points in the image data whose size is less than the size threshold; the first detail feature map is detected to obtain the target binding information of the steel bar binding. That is to say, in the embodiment of the present application, the acquired image data is processed for multiple rounds, the channel features of the color channel are extracted from the image data, the channel features are further screened and filtered to obtain the first image feature map, and the first image feature map is further filtered and extracted to obtain the first detail feature map, thereby obtaining the target binding information, thereby solving the technical problem of low recognition accuracy of the binding information and achieving the technical effect of improving the recognition accuracy of the binding information.

[0054] The above method of this embodiment is further introduced below.

[0055] As an optional implementation, step S110, detecting the first detail feature map to obtain target binding information of steel bar binding, includes: detecting the first detail feature map to obtain first binding information; converting the first detail feature map through a channel feature extractor to obtain a second image feature map, wherein the granularity of the second image feature map is smaller than that of the first image feature map; extracting a second detail feature map of the steel bar binding in the second image feature map through a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than that of the first detail feature map; detecting the second detail feature map to obtain second binding information; and determining the target binding information based on the second detail feature map, the first binding information and the second binding information.

[0056] In this embodiment, the first detail feature map can be detected to obtain the first binding information; the first detail feature map is processed by a channel feature extractor to obtain a second image feature map, and the second detail feature map of the steel bar binding in the second image feature map is extracted by a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than that of the first detail feature map; the second detail feature map is detected to obtain the second binding information; and the target binding information is obtained based on the second detail feature map, the first binding information and the second binding information.

[0057] Optionally, both the first binding information and the second binding information can be used to determine the binding points in the steel bar binding, but the granularity of the second image feature is smaller than that of the first image feature, that is, the second image feature can be used to characterize the binding points of smaller sizes, and the number and accuracy of identifying binding points can be higher than that of the first image feature. The second detail feature map can be used to determine the binding information of the binding points of sizes smaller than the size threshold in the image data, but the second detail feature map has a higher level of detail than the first detail feature map, that is, the second detail feature map can identify binding points that the first detail feature map cannot identify.

[0058] For example, image data is acquired, channel features corresponding to multiple color channels are extracted from the image data, and channel features that can be used to identify binding points can be selected from the multiple channel features. The selected channel features can be transmitted to a channel feature extractor, and the channel features can be processed by the channel feature extractor to obtain a first image feature map. The SCFEM module can be used to extract a first detail feature map of steel bar binding from the first image feature map. The first detail feature map can be detected by a feature pyramid network (FPN) to obtain first binding information. At the same time, the first detail feature map can be processed by the channel feature extractor to obtain a second image feature map, and the second detail feature map of steel bar binding in the second image feature map can be extracted by the SCFEM module; the second detail feature map can be detected by the feature pyramid network to obtain second binding information; and the target binding information can be obtained based on the second detail feature map, the first binding information, and the second binding information.

[0059] Optionally, FPN can be used to extract and integrate features of different scales in the image data, thereby improving the ability of the neural network model to detect and locate the binding points. FPN can perform feature fusion at different scales, so that the neural network can simultaneously focus on information of different scales in the image data, thereby achieving the purpose of improving the performance and robustness of the model. It should be noted that in addition to FPN, other neural network models can also be used to identify the first detail feature map to obtain the binding information, and no specific restrictions are made on the module for obtaining the binding information.

[0060] As an optional implementation, the target binding information includes at least the binding position of the steel bar binding and the confidence of the binding position. Based on the second detail feature map, the first binding information and the second binding information, the target binding information is obtained, including: converting the second detail feature map through the channel feature extractor to obtain a third image feature, wherein the granularity of the third image feature map is smaller than that of the second image feature map; extracting the third detail feature map of the steel bar binding in the third image feature map through the SCFEM module, wherein the granularity of the third detail feature map is smaller than that of the second detail feature map; detecting the third detail feature map to obtain the third binding information; and fusing the first binding information, the second binding information and the third binding information for the confidence to obtain the target binding information.

[0061] Optionally, the target binding information may include at least the binding position of the steel bar binding and the confidence of the binding position, wherein the confidence may be used to determine the authenticity of the binding position.

[0062] In this embodiment, after obtaining the second detail feature map, the second detail feature map can be processed by a channel feature extractor to obtain a third image feature, and a third detail feature map of steel bar binding in the third image feature map can be extracted by an SCFEM module; the third detail feature map can be detected using a feature pyramid network to obtain third binding information; based on the confidence corresponding to the binding position in the first binding information, the confidence corresponding to the binding position in the second binding information, and the confidence corresponding to the binding position in the third binding information, the first binding information, the second binding information and the third binding information are fused to obtain the target binding information.

[0063] The granularity of the third detail feature map is smaller than the granularity of the second detail feature map.

[0064] In this embodiment, the channel features are processed multiple times to obtain the first binding information, the second binding information, and the third binding information. Optionally, the first binding information, the second binding information, and the third binding information may identify binding positions at the same position, or may identify multiple binding positions in a small area, but only one binding position in the small area is correct. In this case, the binding positions can be screened based on the confidence level to determine the binding positions with high confidence levels, and then the target binding information is obtained. In this way, the accuracy of the target binding information is improved. It should be noted that the number of times the channel features are processed can be selected according to actual needs, and no specific restrictions are made here. For example, the channel feature extractor and the SCFEM module can be used to repeatedly process the data multiple times to achieve the purpose of improving the detection accuracy.

[0065] As an optional implementation, step S106, based on the channel features, determines the first image feature map of steel bar binding, including: dividing the channel features according to the channel dimension to obtain two branch channel features; using multiple convolution kernels to extract features of the first branch channel feature of the two branch channel features to obtain multiple sub-branch channel features; using an activation function to activate the multiple sub-branch channel features to obtain a sub-image feature corresponding to the first branch channel feature; converting the sub-image feature and the second branch channel feature of the two branch channel features to obtain a first image feature map, wherein the second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

[0066] In this embodiment, a channel feature extractor (CFE) is designed, and the first image feature map of the steel bar binding is determined by the channel feature extractor. The channel feature extractor not only reduces the number of parameters of the steel bar binding quality detection model, but also combines the detailed position information of the low-order feature map of the steel bar binding image with the rich semantic information of the high-order feature map. In this way, not only the information redundancy problem in the image feature map of the image data is avoided, but also the performance of the steel bar binding quality detection model is improved.

[0067] Optionally, the channel feature extractor can obtain the first image feature map (or the second image feature map or the third image feature map) by the following steps: the channel feature can be segmented according to the channel dimension to obtain two branch channel features; the first branch channel feature of the two branch channel features is extracted using multiple convolution kernels to obtain multiple sub-branch channel features; the multiple sub-branch channel features are activated using an activation function to obtain a sub-image feature corresponding to the first branch channel feature; the sub-image feature and the second branch channel feature of the two branch channel features are processed to obtain the first image feature map. The second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

[0068] For example, CFE divides the channel features of the steel bar binding image input from the upper layer into two branch channel features X1 and X2 along the channel dimension. The number of channels in each branch is 1 / 2 of the original. The channel features of the first branch channel feature X2 are extracted using convolution kernels with kernels of 3×3 and 5×5, respectively, to obtain multiple sub-branch channel features. After the multiple sub-branch channel features are processed by the ReLu activation function, the sub-image feature X2′ is obtained. The processed sub-image feature X2′ and the unprocessed second branch channel X1 are then subjected to a 1×1 convolution operation after the fusion and splicing (Concat) module to extract the useful information of the unprocessed branch channel, and finally the steel bar binding feature map output by the CFE module is obtained, that is, the first image feature map. It should be noted that the size of the above numbers and the number of processing times are only for example purposes and can be adjusted according to actual conditions.

[0069] As an optional implementation, the sub-image features and the second branch channel features of the two branch channel features are converted to obtain a first image feature map, including: fusing the sub-image features and the second branch channel features to obtain a fused feature; performing convolution processing on the fused feature; and extracting the first image feature map from the fused feature after the convolution processing.

[0070] In this embodiment, a fusion and stitching module can be used to fuse the sub-image features and the second branch channel features to obtain fused features; convolution processing is performed on the fused features; and the first image feature map is extracted from the fused features after the convolution processing.

[0071] For example, a channel feature of size H×W×C is input into CFE. CFE divides the channel feature into a second branch channel feature X1 and a first branch channel feature X2 along the channel dimension, where the number of channels of each branch is 1 / 2 of the original. In this step, only one of the branches is processed, and the other branch remains unchanged. When performing the convolution operation, X2 is first linearly transformed and the dimension order is rearranged. Convolution kernels with kernel sizes of 3×3 and 5×5 can be used to extract features from the first branch channel feature respectively to obtain multiple sub-branch channel features. The obtained multiple sub-branch channel features are activated by the ReLU function to obtain sub-image features. At this time, the number of channels of the obtained sub-image features is restored to C / 2. Through the above steps, the target information of the steel bar binding points of different sizes can be obtained at different scales, and the multi-scale small target features can be integrated so that all sub-features are aggregated and then restored to the original dimension order. Furthermore, the processed X2′ and the unprocessed second branch channel feature X1 can be concatenated by Concat to obtain a fused feature, and a 1×1 convolution operation can be performed on the fused feature to achieve the purpose of extracting useful information of the unprocessed branch channel, and finally the steel bar binding feature map output by the CFE module is obtained, that is, the first image feature map.

[0072] Optionally, in the process of obtaining the first image feature map by the CFE module, while extracting the multi-scale information of the steel bar binding feature map, the calculation of redundant information is also reduced. The first image feature map (output) can be obtained by the following formula:

[0073] output=Conv([Channel1(Stem(X1));Channel2(Stem(X2′))])

[0074] X2′=β(f C3 (X2)+f C5 (X2))

[0075] Among them, output can be used to represent the first image feature map output after passing through the CFE module; [;] can be used to represent the splicing operation; Channel i It can be used to represent the channel grouping operation performed on the input reinforcement binding channel features; Stem(Xi) can be used to represent the channel feature branch structure obtained after the grouping operation; f C3 and f C5 It can be used to represent the convolution operations of 3×3 and 5×5 convolution kernels respectively; β can be used to represent the ReLU activation function.

[0076] Optionally, in the feature extraction stage, since there will be a large amount of similar background redundant information in the images taken by drones at high altitudes, the scale differences of the steel bar binding point targets are large, and too large a network depth will easily lead to gradient explosion, reducing the overall fluency of the network. It is more conducive to enriching the gradient flow by operating only on some channels, and using convolution kernels of different sizes for half of the channel features to obtain perception fields of different scales also improves the model's ability to extract small target features of steel bar binding points. In the feature fusion stage, since there is still useful local information in the unprocessed branch channels, 1×1 convolution is used to improve the nonlinear fitting ability of the model, thereby better learning and enhancing the steel bar binding point features in the background, thereby achieving the technical effect of improving the accuracy of binding point recognition and solving the technical problem of low accuracy of binding point recognition.

[0077] Optionally, for the input steel bar binding point feature of size H×W×C, the required parameter amount p and calculation amount F of the ordinary convolution of size K×K can be used as shown in the following formula, where c can be used to represent the number of channels of the input data.

[0078] p=k×k×c 2

[0079] F=h×w×k 2 ×c 2

[0080] As for CFE, since it only operates on some channels, if a convolution kernel of size K×K is used, the parameter p and the computation amount F can be determined by the following formula:

[0081]

[0082]

[0083] Among them, g can be used to represent the proportion of the input feature channel division. For example, g can be 2, which means that only partial convolution is performed on 1 / 2 of the input channels. Compared with ordinary convolution, through the application embodiment, the number of parameters and the amount of calculation of CFE can be reduced by about 1 / 4, which can enable the steel bar binding quality detection model to better adapt to the limited resource deployment of the steel bar binding detection platform, and thus meet the detection requirements of the steel bar binding quality detection task.

[0084] As an optional implementation, step S108, extracting a first detail feature map of steel bar binding from the first image feature map, includes: compressing the first image feature map to obtain a compressed feature map; activating the compressed feature map, and filtering the activated compressed feature map to obtain a first output feature map; pooling the compressed feature map, and converting the pooled compressed feature map to obtain a second output feature map, wherein the size of the first output feature map is the same as the size of the first output feature map; based on the first output feature map and the second output feature map, obtaining a first detail feature map.

[0085] In this embodiment, the first image feature map can be compressed to obtain a compressed feature map; the compressed feature map can be activated and filtered to obtain a first output feature map; the compressed feature map can be pooled and converted to obtain a second output feature map; based on the first output feature map and the second output feature map, a first detail feature map can be obtained, wherein the size of the first output feature map is the same as the size of the first output feature map.

[0086] Optionally, this embodiment proposes a space-channel filtering module, which can effectively extract the interference information of the steel bar binding feature map (that is, the first image feature) from the channel and space filtering to enhance the small target feature information (that is, the first detail feature map) of the steel bar binding point (which can be simply referred to as the binding point) through SCFEM.

[0087] Optionally, the SCFEM module may include a space purification module (Space Purification Module, abbreviated as SPM) and a channel purification module (Channel Purification Module, abbreviated as CPM). The SCFEM module may obtain a first detail feature map by the following steps: the feature map (that is, the first image feature) of the steel bar binding image data input by the CFE may be compressed to obtain a compressed feature map. The compressed feature map is processed by the SPM and CPM modules respectively to obtain a first output feature map and a second output feature map, and the first output feature map and the second output feature map are processed to obtain a first detail feature map.

[0088] Optionally, SPM can obtain the first output feature map through the following steps: SPM can activate the compressed feature map using an activation function (e.g., Log-Softmax), and perform filtering and enhancement processing on the activated compressed feature map to obtain the first output feature map.

[0089] Optionally, CPM can obtain a second output feature map through the following steps: the compressed feature map can be processed by average pooling and maximum pooling respectively to obtain a more detailed global feature map, and then two convolutional layers and activation functions (for example, Hardswish) are introduced to enhance the small target features of the rebar binding points in the global feature map, and finally the rebar binding feature map is restored to its size through a nearest neighbor upsampling operation, and then the two filtered rebar binding feature maps are spliced ​​to obtain the final output of the CPM module (i.e., the second output feature map).

[0090] Optionally, the output feature map of SPM (ie, the first output feature map) and the output feature map of CPM (ie, the second output feature map) are spliced ​​to obtain the output feature map of SCFEM (ie, the first detail feature map).

[0091] As an optional implementation, obtaining a first detail feature map based on the first output feature map and the second output feature map includes: splicing the first output feature map and the second output feature map according to the channel dimension to obtain the first detail feature map.

[0092] In this embodiment, output feature maps, that is, the first output feature map and the second output feature map, are obtained from the spatial filtering module (SPM) and the channel filtering module (CPM). At this time, the two output feature maps have the same size, and the two output feature maps can be spliced ​​along their channel dimension, which means adding the number of channels of the two feature maps and connecting them along this channel dimension. The spliced ​​feature map is used as the final output of the spatial-channel filtering module (SCFEM) for the steel bar binding point feature map, thereby obtaining the first detail feature map.

[0093] As an optional implementation, pooling is performed on the compressed feature map, and the pooled compressed feature map is converted to obtain a second output feature map, including: average pooling is performed on the compressed feature map to obtain a first pooled feature map, and maximum pooling is performed on the compressed feature map to obtain a second pooled feature map; the first pooled feature map and the second pooled feature map are enhanced respectively to obtain a third pooled feature map and a fourth pooled feature map; the third pooled feature map and the fourth pooled feature map are upsampled; and the upsampled third pooled feature map and the upsampled fourth pooled feature map are fused to obtain the second output feature map.

[0094] In this embodiment, the first image feature map can be compressed in the channel direction to obtain a compressed feature map, and the compressed feature map is input into the SPM and the channel filter module CPM respectively. The SPM can process the compressed feature map using an activation function (e.g., Log-Softmax), thereby filtering and enhancing the feature information of the compressed feature map in space. Among them, the Log-Softmax algorithm can be used to generate relative weights of all positions relative to the channel, and can be used to penalize errors in the likelihood space, so that the algorithm converges faster.

[0095] Optionally, after the compressed feature map is input into the CPM module, the compressed feature map can be average pooled and maximum pooled respectively to obtain a first pooled feature map and a second pooled feature map. Since the combination of the first pooled feature map and the second pooled feature map can obtain more detailed global features, two convolutional layers and Hardswish activation functions can be introduced to enhance the small target features of the steel bar binding points (that is, the first pooled features and the second pooled features) respectively to obtain a third pooled feature map and a fourth pooled feature map. The third pooled feature map and the fourth pooled feature map are upsampled, and the size of the steel bar binding feature map is restored through the nearest neighbor upsampling operation. Finally, the upsampled third pooled feature map and the upsampled fourth pooled feature map are spliced ​​to obtain the final output of the CPM module (the second output feature map).

[0096] In an embodiment of the present application, multiple rounds of processing are performed on the acquired image data, channel features of the color channel are extracted from the image data, the channel features are further screened and filtered to obtain a first image feature map, and the first image feature map is further filtered and extracted to obtain a first detail feature map, thereby obtaining target binding information, thereby solving the technical problem of low recognition accuracy of the binding information and achieving the technical effect of improving the recognition accuracy of the binding information.

[0097] According to an embodiment of the present invention, another embodiment of a method for determining binding information is also provided. The steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and although a logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in an order different from that shown here. Figure 2 is a flow chart of another method for determining binding information according to an embodiment of the present invention.

[0098] like Figure 2 As shown, the method may include the following steps:

[0099] Step S202, obtaining image data of steel bar binding.

[0100] In the technical solution provided in the above step S204 of the present invention, in the identification task of steel bar binding, image data containing steel bar binding can be collected. Optionally, the image data can be collected by on-site real-time monitoring, camera shooting, drone shooting, etc., and can be used to confirm the binding situation of the steel bar binding. The image data can be a steel bar binding picture, a steel bar binding video frame, etc. It should be noted that this is only an example, and there is no specific limitation on the method of collecting image data.

[0101] Step S204: Use the detection model to detect the image data to obtain target binding information of the steel bar binding.

[0102] In the technical solution provided in the above step S204 of the present invention, the collected image data can be transmitted to the detection model, and the image data can be detected using the detection model to obtain the target binding information of the steel bar binding.

[0103] In this embodiment, the detection model is used to extract channel features of the color channel from the image data, determine the first image feature map of the steel bar binding based on the channel features, and extract the first detail feature map of the steel bar binding from the first image feature map, and detect the first detail feature to obtain target binding information, the first graphic feature map is used to represent the binding information of the steel bar binding, the first detail feature map is used to represent the binding information of the size binding points in the image data whose size is less than the size threshold, and the binding information includes the target binding information.

[0104] Optionally, the detection model may at least include a channel feature extractor and a space-channel filtering module constructed using an object detection (EORA) algorithm. The detection model extracts channel features of the color channel from the image data, processes the channel features using the channel feature extractor to obtain a first image feature map of steel bar binding, extracts a first detail feature map from the first image feature map using the space-channel filtering module, and detects the first detail feature map through a feature pyramid network to obtain target binding information.

[0105] In order to improve the accuracy of the target binding information, the channel feature extractor and the space-channel filtering module can be used to process the image data multiple times. It should be noted that there is no specific restriction on the content contained in the detection model here, and the content contained in the model can be increased or reduced according to the actual situation.

[0106] As an optional implementation, the method may also include: detecting historical image data through a sub-detection model to obtain historical binding information; determining the intersection-and-union ratio between the historical binding information and the real binding information corresponding to the historical image data; constructing a loss function based on the intersection-and-union ratio; and using the loss function to adjust the model parameters of the sub-detection model to obtain a detection model.

[0107] In this embodiment, the historical image data and the real binding information may be pre-acquired and used to train the sub-detection model. The sub-model may be an EORA model, or a steel bar binding quality detection model, which may be a steel bar binding quality detection model constructed based on deep learning.

[0108] Optionally, historical image data is detected through a sub-detection model to obtain historical binding information; the intersection-and-union ratio between the historical binding information and the real binding information corresponding to the historical image data is determined; a loss function is constructed based on the intersection-and-union ratio; and the model parameters of the sub-detection model are adjusted using the loss function to obtain a detection model.

[0109] In this embodiment, a loss function (e.g., Wise-IOU) for target detection tasks is introduced, and the positioning performance of the EORA model for steel bar binding points is improved by using the loss function. Optionally, the loss function solves the possible deviation problem in the intersection over union (IOU) by weighting the area between the predicted box and the true box; when the target box and the true box overlap well, the geometric factor penalty is weakened, so that the disturbance generated by the steel bar binding quality detection model has better generalization. On this basis, distance attention is created, and Wise-IOUv1 with a two-layer attention mechanism is obtained.

[0110] For example, suppose that at the position (x, y) of the target box t ,y t ), then the Wise-IOUv1 formula is:

[0111]

[0112] Among them, L Wise-IOUv1 It can be determined that Wise-IOUv1;L IOU It can be used to represent the gradient gain of the monotonic focusing coefficient, w c 、h c The width and height of the minimum bounding rectangle that can be used to represent the predicted box (that is, the historical binding information) and the real box (that is, the real binding information) respectively, and exp can be used to represent the accuracy of the anchor box.

[0113] Optionally, in order to avoid harmful gradients caused by low quality of the steel bar binding dataset, the outlier degree β, L Wise-IOUvl The monotonic focusing coefficient and Wise-IOUv3 constructed from Wise-IOUv1. L can be calculated by the following formula Wise-IOUv3 :

[0114] L Wise-IOUv3=rL Wise-IOUv1

[0115]

[0116]

[0117] Among them, α and δ can be used as hyperparameters and can be set to 1.9 and 3.0 respectively. β can be the outlier degree, and the outlier degree parameter is introduced to describe the quality of the anchor box, which is negatively correlated with the quality of the anchor box.

[0118] The increased computational cost of Wise-IOU mainly lies in the calculation of the focus coefficient and the mean statistics of the IoU loss. Under the same experimental conditions, Wise-IOU has a faster speed and higher accuracy in recognizing steel bar binding images because it does not calculate the aspect ratio.

[0119] The aspect ratio refers to the ratio of the width to the height of a bounding box or an object. The traditional IoU calculation needs to consider the area of ​​the overlapping area of ​​the two bounding boxes and the overall area of ​​the two bounding boxes. In the calculation process, it is necessary to first find the overlapping part of the two bounding boxes and calculate its area, then calculate the overall area of ​​the two bounding boxes, and finally divide the area of ​​the overlapping area by the overall area to get the IoU value. The aspect ratio needs to be considered in this process, so the calculation is relatively complicated.

[0120] Optionally, after initializing the model parameters of the sub-detection model, a steel bar binding data set can be obtained, and the data set can be divided into a training set, a validation set and a test set, and the training set and validation set data can be divided into multiple batches, and a batch of training set data is input into the sub-detection model for training each time to obtain the training loss value (loss) of the batch. After completing a round of training for all batches of data in the entire training set (in the actual training process, there may be multiple rounds), the validation set can be input into the steel bar binding quality detection model in batches to obtain the corresponding batch loss value (batch_loss). During training and verification, the sub-detection model will automatically learn and adjust parameters according to each loss and batch_loss situation. When the training process has been carried out for one or more rounds until the batch_loss value converges, the sub-detection model training is completed and the detection model is obtained.

[0121] In an embodiment of the present application, multiple rounds of processing are performed on the acquired image data, channel features of the color channel are extracted from the image data, the channel features are further screened and filtered to obtain a first image feature map, and the first image feature map is further filtered and extracted to obtain a first detail feature map, thereby obtaining target binding information, thereby solving the technical problem of low recognition accuracy of the binding information and achieving the technical effect of improving the recognition accuracy of the binding information.

[0122] Example 2

[0123] The technical solution of the embodiment of the present invention is illustrated below in conjunction with preferred implementation modes.

[0124] Rebar tying is an important construction process in construction and civil engineering. It mainly involves connecting pre-cut steel bars according to design drawings and engineering requirements through tying tools and tying wires so that the steel bars can be tightly fixed together. This process is designed to ensure the overall strength and stability of the concrete structure. The quality of steel bar tying is directly related to the safety and durability of the concrete structure. Testing the quality of steel bar tying can help construction personnel check and adjust the quality of steel bar tying, ensure that the tying is firm and the connection is good, improve the seismic resistance and bearing capacity of the overall structure, and ensure that it meets the design and construction standards.

[0125] In the field of deep learning, target detection algorithms can be divided into two categories: two-stage detection and one-stage detection. Two-stage detection (for example, R-CNN series, SPPNet, etc.) implements detection tasks progressively according to the idea of ​​"from big to small". One-stage detection (for example, YOLO series, SSD, Retina Net, etc.) performs regional category judgment while generating prediction boxes. The model has low complexity and fast detection speed.

[0126] In the related art, in the recognition task of steel bar binding images, it is first necessary to collect image data containing steel bar binding, which comes from on-site real-time monitoring, camera shooting and drone shooting. Then, the steel bar binding image data is preprocessed, such as resizing, removing noise, and enhancing contrast, so that the subsequent detection algorithm can better process it. Then, a deep convolutional neural network is used to extract the steel bar binding image features. These features contain various information in the steel bar binding image, such as binding shape, binding position, etc., which are helpful for subsequent target detection. After feature extraction, the detection model can be used to identify whether there is a problem with the steel bar binding in the image data. The detection model can directly predict the category and location information of the target from the feature map. After completing the target detection, the detected steel bar binding position and other related information are output as results and displayed in the form of images. In actual projects, a large number of steel bar binding images need to be detected quickly, and at the same time, false detection or missed detection problems should be avoided to the greatest extent to ensure the quality of the project.

[0127] As an optional embodiment, a steel bar binding method based on machine vision is proposed, which uses a depth camera to obtain image information of the steel bars to be tied; determines the pixel coordinates of the steel bar intersections based on the image information; sorts the steel bar intersections to determine the steel bar binding planning route; controls the robotic arm to perform steel bar binding in sequence according to the steel bar binding planning route until the steel bar binding task of all shooting points is completed, thereby avoiding the problem of repeated binding and missed binding at the steel bar intersections and improving the steel bar binding efficiency. However, when there are multiple layers of steel bars in the steel bar binding image data, technical problems such as incorrect identification and calibration of the steel bar binding points are prone to occur.

[0128] In the related technology, since the target of the rebar binding point is small, the official model is directly used for detection. Its single grid prediction frame is difficult to accurately capture the position of the target, resulting in a large positioning error. There is a problem that the existing target detection algorithm has relatively low positioning accuracy for the target of the rebar binding image. In addition, the rebar binding image involves small-sized targets, such as thin rebars. The existing target detection algorithm has weak performance in small target detection, and is prone to missed detection and false detection. There is a problem of difficulty in detecting small targets in rebar binding. At the same time, in actual engineering, rebars usually need to be tied in multiple layers, and the rebar binding points may be blocked by other objects or structures, resulting in some rebar binding points cannot be detected. When there is target occlusion, it will lead to technical problems of low detection efficiency.

[0129] To solve the above problems, in this embodiment, the EORA algorithm is used to design a channel feature extractor (CFE), and a convolution-channel feature extractor (Conv_CFE) module is constructed, so that the network pays more attention to dense and homogeneous small-sized objects, solving the problem of difficulty in detecting small targets in the steel bar binding quality inspection task. At the same time, a space-channel filtering module is designed to improve the problem of small target feature information being submerged in the convolution process by filtering out interference information and enhancing small target feature information, thereby solving the problem of target occlusion in the steel bar binding quality inspection task. In order to solve the problem of missed detection of individual steel bar binding points, Wise-IOU is introduced as the loss function of this algorithm. The above improvements effectively solve the problem of insufficient accuracy of EORA for small target detection, achieve the technical effect of improving the model's detection accuracy for steel bar binding points, and solve the technical problem of low detection accuracy of the model for steel bar binding points.

[0130] The following will be combined with the accompanying drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments; the size of the numbers in the embodiments is only for illustration and is not specifically limited here. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present invention and are not specifically limited here.

[0131] In this embodiment, a steel bar binding quality detection model based on deep learning is proposed, including data set construction, channel feature extractor design, space-channel filter module design, and improved loss function module. The method first constructs a data set, and professionals create negative samples to balance the number of positive and negative samples. Then the data set is annotated, and the annotated data set is divided into a training set, a validation set, and a test set according to a certain ratio. At the same time, the channel feature extractor is designed to construct the Conv_CFE module, and then the space-channel filter module is designed. The space-channel filter module is placed after the Conv_CFE module, and then the Wise-IOU loss function is introduced as the loss function of this algorithm, and the training parameters are initialized. Then the training set and the validation set are loaded into the steel bar binding quality detection model based on deep learning for training and updating the network parameters, and then the test set is input into the trained network model to verify the effect of the model. Finally, a steel bar binding quality detection model is obtained, which can automatically detect the input steel bar binding image data.

[0132] Figure 3 is a flow chart of constructing a steel bar binding quality detection model based on deep learning according to an embodiment of the present invention, such as Figure 3As shown, the above detection model can be constructed by following the steps below:

[0133] Step S301, constructing a steel bar dataset.

[0134] In this embodiment, a basic steel bar binding data set is collected in an actual project, and a certain number of negative samples are reasonably created based on the collected data set to balance the number of positive and negative samples, and the constructed data set is annotated. The above data set may include multiple image data.

[0135] Step S302, designing a channel feature extractor.

[0136] In this embodiment, a channel feature extractor (CFE) is designed, and the first image feature map of the steel bar binding is determined by the channel feature extractor. The channel feature extractor not only reduces the number of parameters of the steel bar binding quality detection model, but also combines the detailed position information of the low-order feature map of the steel bar binding image with the rich semantic information of the high-order feature map. In this way, not only the information redundancy problem in the image feature map of the image data is avoided, but also the performance of the steel bar binding quality detection model is improved.

[0137] Optionally, CFE divides the channel features of the steel bar binding image input from the upper layer into two branch channel features X1 and X2 along the channel dimension, and the number of channels of each branch is 1 / 2 of the original. Then, the channel features of the first branch channel feature X2 are extracted using convolution kernels with kernels of 3×3 and 5×5, respectively, to obtain multiple sub-branch channel features. After the multiple sub-branch channel features are processed by the ReLu activation function, the sub-image feature X2′ is obtained. Then, the processed sub-image feature X2′ and the unprocessed second branch channel X1 are subjected to a 1×1 convolution operation after a fusion and splicing (Concat) module to extract the useful information of the unprocessed branch channel, and finally the steel bar binding feature map output by the CFE module is obtained, that is, the first image feature map. It should be noted that the size of the above numbers and the number of processing times are only for example purposes and can be adjusted according to actual conditions.

[0138] Figure 4 is a flow chart of a data processing process of a channel feature extractor according to an embodiment of the present invention. Figure 4 As shown, the channel feature extractor can process the data through the following steps:

[0139] Step S401, obtaining channel features.

[0140] In this embodiment, a channel feature with dimensions of H×W×C may be input into the CFE.

[0141] Step S402, splitting channel features.

[0142] In this embodiment, CFE divides the acquired channel features into second branch channel features X1 and first branch channel features X2 along the channel dimension, wherein the number of channels of each branch is 1 / 2 of the original. In this step, only one of the branches is processed and the other branch remains unchanged.

[0143] Step S403, performing convolution processing on the second branch channel feature.

[0144] In this embodiment, only the second branch channel feature can be convolved. When performing the convolution operation, X2 can be linearly transformed first and the dimensional order can be rearranged. Convolution kernels with kernel sizes of 3×3 and 5×5 can be used to extract features of the first branch channel features respectively to obtain multiple sub-branch channel features. The multiple sub-branch channel features obtained are subjected to the ReLU activation function to obtain sub-image features. At this time, the number of channels of the obtained sub-image features is restored to C / 2. Through the above steps, target information of steel bar binding points of different sizes can be obtained at different scales, and multi-scale small target features can be integrated so that all sub-features are aggregated and then restored to the original dimensional order.

[0145] Step S404, fusing the sub-image features and the first branch channel features.

[0146] In this embodiment, the processed X2′ and the unprocessed second branch channel feature X1 can be concatenated by Concat to obtain a fused feature, and a 1×1 convolution operation can be performed on the fused feature to achieve the purpose of extracting useful information of the unprocessed branch channel, and finally the steel bar binding feature map output by the CFE module is obtained, that is, the first image feature map.

[0147] Optionally, in the process of obtaining the first image feature map by the CFE module, while extracting the multi-scale information of the steel bar binding feature map, the calculation of redundant information is also reduced. The first image feature map (output) can be obtained by the following formula:

[0148] output=Conv([Channel1(Stem(X1));Channel2(Stem(X2′))])

[0149] X2′=β(f C3 (X2)+f C5 (X2))

[0150] Among them, output can be used to represent the first image feature map output after passing through the CFE module; [;] can be used to represent the splicing operation; Channel iIt can be used to represent the channel grouping operation performed on the input reinforcement binding channel features; Stem(Xi) can be used to represent the channel feature branch structure obtained after the grouping operation; f C3 and f C5 It can be used to represent the convolution operations of 3×3 and 5×5 convolution kernels respectively; β can be used to represent the ReLU activation function.

[0151] Optionally, in the feature extraction stage, since there will be a large amount of similar background redundant information in the images taken by drones at high altitudes, the scale differences of the steel bar binding point targets are large, and too large a network depth will easily lead to gradient explosion, reducing the overall fluency of the network. It is more conducive to enriching the gradient flow by operating only on some channels, and using convolution kernels of different sizes for half of the channel features to obtain perception fields of different scales also improves the model's ability to extract small target features of steel bar binding points. In the feature fusion stage, since there is still useful local information in the unprocessed branch channels, 1×1 convolution is used to improve the nonlinear fitting ability of the model, thereby better learning and enhancing the steel bar binding point features in the background, thereby achieving the technical effect of improving the accuracy of binding point recognition and solving the technical problem of low accuracy of binding point recognition.

[0152] Optionally, for the input steel bar binding point feature of size H×W×C, the required parameter amount p and calculation amount F of the ordinary convolution of size K×K can be used as shown in the following formula, where c can be used to represent the number of channels of the input data.

[0153] p=k×k×c 2

[0154] F=h×w×k 2 ×c 2

[0155] As for CFE, since it only operates on some channels, if a convolution kernel of size K×K is used, the parameter p and the computation amount F can be determined by the following formula:

[0156]

[0157]

[0158] Among them, g can be used to represent the proportion of the input feature channel division. For example, g can be 2, which means that only partial convolution is performed on 1 / 2 of the input channels. Compared with ordinary convolution, through the application embodiment, the number of parameters and the amount of calculation of CFE can be reduced by about 1 / 4, which can enable the steel bar binding quality detection model to better adapt to the limited resource deployment of the steel bar binding detection platform, and thus meet the detection requirements of the steel bar binding quality detection task.

[0159] Step S303, designing a space-channel filtering module.

[0160] In this embodiment, a space-channel filtering module is also proposed, which can effectively extract interference information from the channel and space filtering steel bar binding feature map (that is, the first image feature) through SCFEM to enhance the small target feature information (that is, the first detail feature map) of the steel bar binding point (which can be referred to as the binding point). The SCFEM module can include a spatial filtering module and a channel filtering module.

[0161] Optionally, Figure 5 is a flow chart of a space-channel filtering module according to an embodiment of the present invention. Figure 5 As shown, the method may include the following steps:

[0162] Step S501: compress the first image feature.

[0163] In this embodiment, the feature map of the steel bar binding image data input by the CFE (that is, the first image feature 51) can be compressed to obtain a compressed feature map. The compressed feature map is processed by the SPM module 52 and the CPM module 53 to obtain a first output feature map and a second output feature map, and the first output feature map and the second output feature map are processed to obtain a first detail feature map.

[0164] Optionally, SPM can obtain the first output feature map through the following steps: SPM can activate the compressed feature map using an activation function (e.g., Log-Softmax), and perform filtering and enhancement processing on the activated compressed feature map to obtain the first output feature map.

[0165] Optionally, CPM can obtain a second output feature map through the following steps: the compressed feature map can be processed by average pooling and maximum pooling respectively to obtain a more detailed global feature map, and then two convolutional layers and activation functions (for example, Hardswish) are introduced to enhance the small target features of the rebar binding points in the global feature map, and finally the rebar binding feature map is restored to its size through a nearest neighbor upsampling operation, and then the two filtered rebar binding feature maps are spliced ​​to obtain the final output of the CPM module (i.e., the second output feature map).

[0166] Step S502: concatenate the first output feature map and the second output feature map.

[0167] In this embodiment, the output feature map of SPM (ie, the first output feature map) and the output feature map of CPM (ie, the second output feature map) may be spliced ​​together to obtain the output feature map of SCFEM (ie, the first detail feature map 54).

[0168] In this embodiment, the first image feature map can be compressed in the channel direction to obtain a compressed feature map, and the compressed feature map is input into the SPM and the channel filter module CPM respectively. The SPM can process the compressed feature map using an activation function (e.g., Log-Softmax), thereby filtering and enhancing the feature information of the compressed feature map in space. Among them, the Log-Softmax algorithm can be used to generate relative weights of all positions relative to the channel, and can be used to penalize errors in the likelihood space, so that the algorithm converges faster.

[0169] Figure 6 is a flow chart of a data processing process of a CPM module according to an embodiment of the present invention. Figure 6 As shown, the spatial-channel filtering module can process the data through the following steps:

[0170] Step S601, performing pooling processing on the compressed feature map.

[0171] In this embodiment, after the compressed feature map 61 is input into the CPM module, average pooling and maximum pooling may be performed on the compressed feature map respectively, thereby obtaining a first pooling feature map and a second pooling feature map.

[0172] Step S602, introducing a convolutional layer and an activation function to process the first pooled feature map and the second pooled feature map respectively.

[0173] Optionally, since the combination of the first pooled feature map and the second pooled feature map can obtain more detailed global features, two convolutional layers and Hardswish activation functions can be introduced to enhance the small target features of the steel bar binding points (that is, the first pooled features and the second pooled features), respectively, and the enhanced first pooled feature map and the enhanced second pooled feature map are upsampled, and the size of the steel bar binding feature map is restored through the nearest neighbor upsampling operation.

[0174] Step S603, concatenating the upsampled first pooling feature map and the upsampled second pooling feature map.

[0175] In this embodiment, the upsampled first pooling feature map and the upsampled second pooling feature map are concatenated to obtain the final output of the CPM module (the second output feature map).

[0176] The output feature maps, i.e., the first output feature map and the second output feature map, are obtained from the spatial filtering module (SPM) and the channel filtering module (CPM). At this time, the two output feature maps have the same size and can be spliced ​​along their channel dimension, which means adding the number of channels of the two feature maps and connecting them along this channel dimension. The spliced ​​feature map is used as the final output of the spatial-channel filtering module (SCFEM) for the feature map of the steel bar binding point, thereby obtaining the first detail feature map.

[0177] Optionally, the Hardswish activation function may be an activation function used in a deep learning neural network to improve the performance and efficiency of the model.

[0178] Step S304, introducing a loss function.

[0179] In this embodiment, a loss function (e.g., Wise-IOU) for target detection tasks is introduced, and the positioning performance of the EORA model for steel bar binding points is improved by using the loss function. Optionally, the loss function solves the possible deviation problem in the intersection over union (IOU) by weighting the area between the predicted box and the true box; when the target box and the true box overlap well, the geometric factor penalty is weakened, so that the disturbance generated by the steel bar binding quality detection model has better generalization. On this basis, distance attention is created, and Wise-IOUv1 with a two-layer attention mechanism is obtained.

[0180] For example, suppose that at the position (x, y) of the target box t ,y t ), then the Wise-IOUv1 formula is:

[0181]

[0182] Among them, L Wise-IOUv1 It can be determined that Wise-IOUv1;L IOU It can be used to represent the gradient gain of the monotonic focusing coefficient, w c 、h c The width and height of the minimum bounding rectangle that can be used to represent the predicted box (that is, the historical binding information) and the real box (that is, the real binding information) respectively, and exp can be used to represent the accuracy of the anchor box.

[0183] Optionally, in order to avoid harmful gradients caused by low quality of the steel bar binding dataset, the outlier degree β, L Wise-IOUvl The monotonic focusing coefficient and Wise-IOUv3 constructed from Wise-IOUv1. L can be calculated by the following formula Wise-IOUv3 :

[0184] L Wise-IOUv3 =rL Wise-IOUv1

[0185]

[0186]

[0187] Among them, α and δ can be used as hyperparameters and can be set to 1.9 and 3.0 respectively. β can be the outlier degree, and the outlier degree parameter is introduced to describe the quality of the anchor box, which is negatively correlated with the quality of the anchor box.

[0188] The increased computational cost of Wise-IOU mainly lies in the calculation of the focus coefficient and the mean statistics of the IoU loss. Under the same experimental conditions, Wise-IOU has a faster speed and higher accuracy in recognizing steel bar binding images because it does not calculate the aspect ratio.

[0189] The aspect ratio refers to the ratio of the width to the height of a bounding box or an object. The traditional IoU calculation needs to consider the area of ​​the overlapping area of ​​the two bounding boxes and the overall area of ​​the two bounding boxes. In the calculation process, it is necessary to first find the overlapping part of the two bounding boxes and calculate its area, then calculate the overall area of ​​the two bounding boxes, and finally divide the area of ​​the overlapping area by the overall area to get the IoU value. The aspect ratio needs to be considered in this process, so the calculation is relatively complicated.

[0190] Step S305: obtain a detection model based on loss function training.

[0191] In this embodiment, the parameters of each layer can be trained and updated based on the sub-detection model. First, all neural network parameters (i.e., model parameters) are initialized, and hyperparameters related to the steel bar binding quality detection model are set, such as: training rounds, batch size, optimizer selection, learning rate, maximum value of gradient clipping strategy, etc.

[0192] Optionally, after initializing the model parameters of the sub-detection model, a steel bar binding data set can be obtained, and the data set can be divided into a training set, a validation set and a test set, and the training set and validation set data can be divided into multiple batches, and a batch of training set data is input into the sub-detection model for training each time to obtain the training loss value (loss) of the batch. After completing a round of training for all batches of data in the entire training set (in the actual training process, there may be multiple rounds), the validation set can be input into the steel bar binding quality detection model in batches to obtain the corresponding batch loss value (batch_loss). During training and verification, the sub-detection model will automatically learn and adjust parameters according to each loss and batch_loss situation. When the training process has been carried out for one or more rounds until the batch_loss value converges, the sub-detection model training is completed and the detection model is obtained.

[0193] Step S306, using the trained prediction model.

[0194] In this embodiment, the trained steel bar binding quality detection model can be used to detect the steel bar binding image data, thereby realizing automatic detection of the quality of the steel bar binding.

[0195] Figure 7 is a schematic diagram of a detection result according to an embodiment of the present invention, such as Figure 7 As shown, in actual engineering applications, the collected image data of steel bar binding can be input into the steel bar binding quality detection model, and then the steel bar binding quality detection model detects the image to obtain the detection result of the steel bar binding quality (that is, the target binding information), and outputs the detection result on the image, wherein the detection result may include a position box and confidence for characterizing the binding point, the missed binding position of the binding point, and the binding quality of the bound point.

[0196] For example, 4,000 images of steel bar binding can be used to construct a steel bar binding dataset for training a steel bar binding quality detection model. The binding points of the extracted steel bar binding area are annotated, and the quality score of each binding point is annotated to obtain the dataset annotation. The dataset can be randomly divided into a training set and a validation set in a ratio of 8:2, that is, 3,200 images are used as the training set and 800 images are used as the validation set. The data in the training set is input into the steel bar binding quality detection model for training. The entire training process is set to 300 epochs, and the parameters in the model are adjusted according to the loss value in each iteration process. After adjusting the relevant parameters, the accuracy of the steel bar binding quality detection model finally stabilized, mainly fluctuating between 90% and 95%.

[0197] In this embodiment, a channel feature extractor is designed to make the network pay more attention to dense and homogeneous small-sized objects. A space-channel filtering module is designed to improve the problem of small target feature information being submerged during the convolution process by filtering out interference information and enhancing small target feature information. At the same time, in order to solve the problem of missed detection of individual steel bar binding points, Wise-IOU is used instead of CIOU of the original algorithm as the loss function of this algorithm. By weighting the area between the predicted box and the real box, the possible deviation problem of IOU (Intersection over Union) is solved.

[0198] Optionally, the channel feature extractor (CFE) detects small target objects at the steel bar binding points, and divides the input steel bar binding feature map of size H×W×C into two branches X1 and X2, with the number of channels in each branch being half of the original. Among them, X2 is subjected to linear transformation and 3×3 and 5×5 convolution kernels for feature extraction to obtain target information of steel bar binding points of different scales, and then passes through the ReLU activation function. The processed X2′ is concat-fused with the unprocessed X1, and then subjected to a 1×1 convolution operation to extract useful steel bar binding target information. This process extracts multi-scale steel bar binding point information and reduces the calculation of redundant information.

[0199] Optionally, a method for filtering out interference information of a steel bar binding feature map by a space-channel filtering module (SCFEM) compresses the input steel bar binding feature map in the channel direction, and then inputs it into a space filtering module (SPM) and a channel filtering module (CPM) respectively, to obtain steel bar binding feature maps after two filters, and finally superimposes the steel bar binding feature maps after two filters as the final output of the space-channel filtering module.

[0200] Optionally, by using the Wise-IOU loss function, the area between the predicted box and the true box is weighted to solve the possible deviation problem of IOU; when the target box and the true box overlap well, the geometric factor penalty is weakened, so that the disturbance generated by the steel bar model has better generalization, thereby improving the prediction accuracy of the model.

[0201] For example, Figure 8 is a schematic diagram of an EORA algorithm according to an embodiment of the present invention, such as Figure 8As shown, for the input image data 801 of steel bar binding, it first passes through two layers of convolutional layers Conv2D_BN_SiLU, for example, convolutional layer 802 and convolutional layer 803 perform convolution processing to obtain the channel features of the steel bar binding image, and then the obtained channel features are input into the channel feature extractor 804, wherein the channel feature extractor Conv_CFE pays more attention to dense and homogeneous small-sized objects, and enhances the model's feature extraction ability for small targets. Further, the spatial-channel filtering module 805 (SCFEM) takes the steel bar binding feature map output by the Conv_CFE module as input, and improves the problem of small target feature information being submerged during the convolution process by filtering out interference information and enhancing small target feature information, and outputs a steel bar binding feature map with more detailed small target features. Subsequently, the steel bar binding feature map passes through the convolutional layer 806, the channel feature extractor 807, the spatial-channel filtering module 808, the convolutional layer 809, the channel feature extractor 810 and the spatial-channel filtering module 812 in sequence, and the above operation is repeated twice. Then, each steel bar binding feature map extracted by SCFEM is passed through feature pyramid network 813, feature pyramid network 814 and feature pyramid network 815 respectively, and then through the introduced Wise-IOU loss function to improve the problem of missed detection of steel bar binding points, and finally the EORA detection result of the steel bar binding image is obtained (that is, the target binding information 816).

[0202] In this embodiment, the rebar binding image dataset collected in the actual project is processed twice, the number of positive and negative samples in the dataset is balanced, and the robustness of the model is enhanced. The designed channel feature extractor, while extracting the multi-scale information of the rebar binding feature map, also greatly reduces the calculation of redundant information, thereby reducing the amount of training for the rebar binding quality detection model, better meeting the real-time requirements for rebar binding quality detection in actual projects, and significantly reducing the amount of training for the rebar binding quality detection model.

[0203] In this embodiment, a channel feature extractor, a space-channel filtering module and an improved Wise-IOU loss function are used to solve the problems of difficulty in detecting small targets at rebar binding points and missed detection caused by occlusion of rebar binding targets in the rebar binding quality detection model. The detection accuracy is greatly improved, and its performance is even better without sacrificing the detection speed, thereby solving the technical problem of low recognition accuracy of binding information and achieving the technical effect of improving the recognition accuracy of binding information.

[0204] Example 3

[0205] According to an embodiment of the present invention, a device for determining binding information is also provided. It should be noted that the device for determining binding information of this embodiment can be used to execute the method for determining binding information of embodiment 1 of the present invention.

[0206] Fig. 9 1 is a schematic diagram of a binding information determination device according to an embodiment of the present invention. As shown in the figure, the binding information determination device 90 may include: a first acquisition unit 91, a first extraction unit 92, a determination unit 93, a second extraction unit 94 and a processing unit 95.

[0207] The first acquisition unit 91 is used to acquire image data of steel bar binding.

[0208] The first extraction unit 92 is used to extract channel features of color channels from the image data.

[0209] The determination unit 93 is used to determine a first image feature map of steel bar binding based on the channel feature, wherein the first image feature map is used to represent binding information of the steel bar binding.

[0210] The second extraction unit 94 is used to extract a first detail feature map of steel bar binding from the first image feature map, wherein the first detail feature map is used to represent binding information of binding points of sizes smaller than a size threshold in the image data.

[0211] The processing unit 95 is used to detect the first detail feature to obtain target binding information of the steel bar binding.

[0212] According to an embodiment of the present invention, another device for determining binding information is provided. It should be noted that the device for determining binding information of this embodiment can be used to execute another method for determining binding information of embodiment 1 of the present invention.

[0213] Fig.10 FIG. 1 is a schematic diagram of another apparatus for determining binding information according to an embodiment of the present invention. Fig.10 As shown, the binding information determination device 100 may include: a second acquisition unit 101 and a detection unit 102 .

[0214] The second acquisition unit 101 is used to acquire image data of steel bar binding.

[0215] The detection unit 102 is used to detect the image data using the detection model to obtain target binding information of the steel bar binding, wherein the detection model is used to extract the channel features of the color channel from the image data, and based on the channel features, determine the first image feature map of the steel bar binding, and extract the first detail feature map of the steel bar binding from the first image feature map, and detect the first detail feature to obtain the target binding information, the first graphic feature map is used to represent the binding information of the steel bar binding, the first detail feature map is used to represent the binding information of the size binding points in the image data whose size is less than the size threshold, and the binding information includes the target binding information.

[0216] The device for determining binding information of this embodiment performs multiple rounds of processing on the acquired image data, extracts channel features of the color channel from the image data, further screens and filters the channel features to obtain a first image feature map, further filters and extracts the first image feature map to obtain a first detail feature map, and thus obtains target binding information, thereby solving the technical problem of low recognition accuracy of binding information and achieving the technical effect of improving the recognition accuracy of binding information.

[0217] Example 5

[0218] According to an embodiment of the present invention, a computer-readable storage medium is further provided. The computer-readable storage medium includes a stored program, wherein the program executes the method for determining binding information in Embodiment 1.

[0219] Example 6

[0220] According to an embodiment of the present invention, a processor is further provided, the processor being used to run a program, wherein when the program is run by the processor, the method for determining binding information in Embodiment 1 is executed.

[0221] Example 7

[0222] According to an embodiment of the present invention, a computer program product is further provided. The computer program product includes computer instructions. When the computer instructions are executed by a processor, the method for determining binding information in Embodiment 1 is implemented.

[0223] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0224] In the above embodiments of the present invention, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0225] In the several embodiments provided in this application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only schematic. For example, the division of units can be a logical function division. There may be other division methods in actual implementation. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of units or modules, which can be electrical or other forms.

[0226] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed over multiple units. Some or all of the units may be selected according to actual needs to achieve the purpose of the present embodiment.

[0227] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0228] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), mobile hard disk, magnetic disk or optical disk and other media that can store program codes.

[0229] The above are only preferred embodiments of the present invention. It should be pointed out that, for ordinary technicians in this technical field, several improvements and modifications can be made without departing from the principles of the present invention. These improvements and modifications should also be regarded as the scope of protection of the present invention.

Claims

1. A method for determining lashing information, characterized in that: include: Acquire image data of steel bar binding; Extracting channel features of color channels from the image data; Based on the channel features, determining a first image feature map of the steel bar binding, wherein the first image feature map is used to represent binding information of the steel bar binding; Extracting a first detail feature map of the steel bar binding from the first image feature map, wherein the first detail feature map is used to represent binding information of binding points of sizes smaller than a size threshold in the image data; Detecting the first detail feature map to obtain target binding information of the steel bar binding; Wherein, the detecting of the first detail feature map to obtain the target binding information of the steel bar binding includes: detecting the first detail feature map to obtain the first binding information; converting the first detail feature map through a channel feature extractor to obtain a second image feature map, wherein the granularity of the second image feature map is smaller than the granularity of the first image feature map; extracting the second detail feature map of the steel bar binding in the second image feature map through a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than the granularity of the first detail feature map; detecting the second detail feature map to obtain the second binding information; and determining the target binding information based on the second detail feature map, the first binding information and the second binding information; Wherein, determining the first image feature map of the steel bar binding based on the channel feature includes: dividing the channel feature according to the channel dimension to obtain two branch channel features; using multiple convolution kernels to extract features of the first branch channel feature of the two branch channel features to obtain multiple sub-branch channel features; using an activation function to activate the multiple sub-branch channel features to obtain a sub-image feature corresponding to the first branch channel feature; converting the sub-image feature and the second branch channel feature of the two branch channel features to obtain the first image feature map, wherein the second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

2. The method according to claim 1, characterized in that The target binding information at least includes the binding position of the steel bar binding and the confidence of the binding position. The target binding information is obtained based on the second detail feature map, the first binding information and the second binding information, including: Converting the second detail feature map by the channel feature extractor to obtain a third image feature, wherein the granularity of the third image feature map is smaller than the granularity of the second image feature map; Extracting, by means of the SCFEM module, a third detail feature map of the steel bar binding in the third image feature map, wherein a granularity of the third detail feature map is smaller than a granularity of the second detail feature map; Detecting the third detail feature map to obtain third binding information; For the confidence, the first binding information, the second binding information and the third binding information are fused to obtain the target binding information.

3. The method according to claim 1, characterized in that The sub-image feature and the second branch channel feature of the two branch channel features are converted to obtain the first image feature map, including: Fusing the sub-image feature and the second branch channel feature to obtain a fused feature; Performing convolution processing on the fused features; The first image feature map is extracted from the fused features after convolution processing.

4. The method according to claim 1, characterized in that Extracting a first detail feature map of the steel bar binding from the first image feature map includes: Compressing the first image feature map to obtain a compressed feature map; Performing activation processing on the compressed feature map, and performing filtering processing on the compressed feature map after the activation processing to obtain a first output feature map; Performing pooling processing on the compressed feature map, and converting the compressed feature map after pooling to obtain a second output feature map, wherein the size of the first output feature map is the same as the size of the second output feature map; The first detail feature map is obtained based on the first output feature map and the second output feature map.

5. The method according to claim 4, characterized in that Obtaining the first detail feature map based on the first output feature map and the second output feature map includes: The first output feature map and the second output feature map are concatenated according to the channel dimension to obtain the first detail feature map.

6. The method according to claim 4, characterized in that Performing pooling processing on the compressed feature map, and converting the pooled compressed feature map to obtain a second output feature map, including: Performing average pooling processing on the compressed feature map to obtain a first pooling feature map, and performing maximum pooling processing on the compressed feature map to obtain a second pooling feature map; Performing enhancement processing on the first pooling feature map and the second pooling feature map respectively to obtain a third pooling feature map and a fourth pooling feature map; Upsampling the third pooling feature map and the fourth pooling feature map; The upsampled third pooling feature map and the upsampled fourth pooling feature map are fused to obtain the second output feature map.

7. A method for determining lashing information, characterized in that: include: Acquire image data of steel bar binding; The image data is detected by using a detection model to obtain target binding information of the steel bar binding, wherein the detection model is used to extract channel features of a color channel from the image data, determine a first image feature map of the steel bar binding based on the channel features, extract a first detail feature map of the steel bar binding from the first image feature map, and detect the first detail feature to obtain the target binding information, the first image feature map is used to represent the binding information of the steel bar binding, the first detail feature map is used to represent the binding information of a size binding point in the image data whose size is less than a size threshold, and the binding information includes the target binding information; Wherein, the detection model is used to detect the first detail feature map through the following steps to obtain the target binding information of the steel bar binding: detect the first detail feature map to obtain the first binding information; convert the first detail feature map through a channel feature extractor to obtain a second image feature map, wherein the granularity of the second image feature map is smaller than the granularity of the first image feature map; extract the second detail feature map of the steel bar binding in the second image feature map through a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than the granularity of the first detail feature map; detect the second detail feature map to obtain the second binding information; determine the target binding information based on the second detail feature map, the first binding information and the second binding information; Wherein, determining the first image feature map of the steel bar binding based on the channel feature includes: dividing the channel feature according to the channel dimension to obtain two branch channel features; using multiple convolution kernels to extract features of the first branch channel feature of the two branch channel features to obtain multiple sub-branch channel features; using an activation function to activate the multiple sub-branch channel features to obtain a sub-image feature corresponding to the first branch channel feature; converting the sub-image feature and the second branch channel feature of the two branch channel features to obtain the first image feature map, wherein the second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

8. The method according to claim 7, characterized in that The method further comprises: The historical image data is detected through the sub-detection model to obtain the historical binding information; Determine an intersection-and-union ratio between the historical lashing information and the real lashing information corresponding to the historical image data; Constructing a loss function based on the intersection-over-union ratio; The loss function is used to adjust the model parameters of the sub-detection model to obtain the detection model.

9. A device for determining lashing information, characterized in that: include: A first acquisition unit is used to acquire image data of steel bar binding; A first extraction unit, used for extracting channel features of color channels from the image data; A determining unit, configured to determine a first image feature map of the steel bar binding based on the channel feature, wherein the first image feature map is used to represent binding information of the steel bar binding; A second extraction unit is used to extract a first detail feature map of the steel bar binding from the first image feature map, wherein the first detail feature map is used to represent binding information of binding points of a size smaller than a size threshold in the image data; A processing unit, configured to detect the first detail feature to obtain target binding information of the steel bar binding; Wherein, the processing unit is used to detect the first detail feature through the following steps to obtain the target binding information of the steel bar binding: detect the first detail feature map to obtain the first binding information; convert the first detail feature map through a channel feature extractor to obtain a second image feature map, wherein the granularity of the second image feature map is smaller than the granularity of the first image feature map; extract the second detail feature map of the steel bar binding in the second image feature map through a space-channel filtering SCFEM module, wherein the granularity of the second detail feature map is smaller than the granularity of the first detail feature map; detect the second detail feature map to obtain the second binding information; determine the target binding information based on the second detail feature map, the first binding information and the second binding information; Wherein, the determination unit is used to determine the first image feature map of the steel bar binding based on the channel feature through the following steps: dividing the channel feature according to the channel dimension to obtain two branch channel features; using multiple convolution kernels to extract features of the first branch channel feature of the two branch channel features to obtain multiple sub-branch channel features; using an activation function to activate the multiple sub-branch channel features to obtain a sub-image feature corresponding to the first branch channel feature; converting the sub-image feature and the second branch channel feature of the two branch channel features to obtain the first image feature map, wherein the second branch channel feature is another branch channel feature of the two branch channel features except the first branch channel feature.

10. A processor, characterized in that: The processor is used to run a program, wherein the program executes the method according to any one of claims 1 to 8 when run by the processor.

11. A computer program product, characterized in that The method comprises computer instructions, which implement the method according to any one of claims 1 to 8 when executed by a processor.

Images