Industrial cuboid extraction method, device, computer and storage medium

Abstract

A method, device, computer, and storage medium for extracting industrial cuboids address the need for a high-precision cuboid extraction method suitable for the industrial field. The method includes: rough cuboid extraction based on target detection and fine cuboid extraction based on line segment reorganization in a filtered area. The rough cuboid extraction based on target detection includes: using a parallel residual convolutional neural network as the backbone network for cuboid target detection and fusing it with a parallel residual block to obtain fused multidimensional features; fusing the multidimensional features with a deconvolution module to obtain a target detection network model; and fine cuboid extraction includes: clustering line segment features based on the target detection network model to obtain a line segment set; connecting line segments based on the line segment set to obtain a line set; dynamically filtering boundaries based on the line set to obtain the edges of the cuboid; and obtaining the image region containing the cuboid based on the cuboid edges and establishing a coordinate system to obtain the cuboid vertices, thereby completing the extraction of the industrial cuboid.

Description

Technical Field

[0001] The present invention relates to the field of industrial visual measurement, and in particular to an industrial cuboid extraction method. Background Art

[0002] The cuboid is a commonly used three-dimensional structural component. Its simple structure, ease of fabrication, and ability to withstand significant bending and torsional forces make it a key component in mechanical assembly and component connection, making it widely used in various industrial fields, including machinery and aerospace. Rapidly and accurately capturing the contour or dimensional information of a cuboid significantly facilitates mechanical fault detection, product performance evaluation, and quality inspection.

[0003] Extracting cuboids from real industrial images is a complex and difficult task. The presence of noise can segment the boundaries of the cuboid into discontinuous segments, and some noise can obscure edge information. Lighting can interfere with the imaging of parts of the cuboid, causing some edges to disappear. Furthermore, different shooting angles can easily cause the shape feature information of the cuboid to be lost, and the cuboid is imaged as multiple adjacent quadrilaterals of different sizes. Unlike circles and ellipses, the boundaries of the cuboid are relatively independent, and there is no theoretical basis for directly determining the size and position of the cuboid. Without accurately locating the boundaries, the cuboid cannot be determined, which places higher demands on its extraction.

[0004] With the rapid development of machine learning and graphics and image technology in recent years, basic shape extraction and detection solutions have been extensively researched and put into use. Directly related to cuboid extraction is the research on rectangle extraction. Existing rectangle extraction and detection methods can be categorized into rectangle extraction in natural scenes and rectangle extraction in industrial settings based on their application scenarios. Rectangle extraction methods can be broadly divided into rectangle extraction based on corner detection, rectangle extraction based on line detection, and rectangle extraction based on hybrid features.

[0005] Rectangle extraction based on corner detection is often combined with the Hough space transform. The core idea is to first reduce noise in the image using image preprocessing methods such as the Hough transform, Gaussian filtering, and wavelet transform. Corner points or vertices are then extracted using a corner detection algorithm. Finally, the extracted points are subjected to the Hough space transform. The vertices of the rectangle are determined based on their peaks in Hough space, and the edges are then determined from the vertices, thereby achieving rectangle extraction. Yueping He et al. proposed a multi-rectangle detection method based on the restricted Hough transform. This method extracts possible Hough peaks from the restricted Hough transform and image pixels. Then, possible rectangles are generated based on the properties of the rectangles. Finally, based on an evidence collection mechanism, a threshold for the rectangle boundary is pre-set. The generated rectangle boundary is compared with the pre-set threshold to determine whether a rectangle exists. Claudio Rosito Jung proposed a rectangle detection method based on a Hough transform window. This method uses a pre-set Hough transform sliding window to scan each pixel in the image. The vertices of the rectangle are determined based on the peaks extracted by the Hough transform and the characteristics of the rectangle, thereby determining the rectangular area. This type of method can relatively accurately detect a single rectangle in a simple image. However, for more complex situations, such as when multiple rectangles are present, this method requires further corner point screening and generated rectangle screening before further rectangle extraction can be performed, which is often very time-consuming. Furthermore, because the vertex information of a rectangle is the only supporting information for rectangle detection and is only represented as a local peak in Hough space, in the industrial field, rectangles or rectangular blocks that are not connected will inevitably be damaged, resulting in some vertices being unclear. In the absence of features, rectangle extraction based on corner detection often fails to fully realize its advantages.

[0006] With the rapid development of machine learning and graphics and image technology in recent years, basic shape extraction and detection solutions have been extensively researched and put into use. Directly related to cuboid extraction is the research on rectangle extraction. Existing rectangle extraction and detection methods can be categorized into rectangle extraction in natural scenes and rectangle extraction in industrial settings based on their application scenarios. Rectangle extraction methods can be broadly divided into rectangle extraction based on corner detection, rectangle extraction based on line detection, and rectangle extraction based on hybrid features.

[0007] Rectangle extraction based on hybrid features achieves rectangle extraction by combining multiple features, such as corner point extraction, line extraction, shape features, and color features. The presence of more than one supporting feature allows for more accurate rectangle detection and extraction. David Shaw proposed a real-time detector for perspective rectangle features in natural scene images. First, vanishing point detection is used to extract key points of geometric features in natural environments. Next, the edges of the perspective image are generated based on the gradient direction of the key points. Lines that may be rectangle edges are initially screened based on the intersecting characteristics of any one side of the rectangle with the other two sides. Finally, possible rectangles in the image are detected. Qian Li used line detection and clustering to detect the locations of possible garages in an image. Then, candidate garages were identified based on the parallelism of opposite sides and the width of the garage lines. Finally, the garage location and entrance were determined by identifying T-shaped or L-shaped feature regions at the garage entrance. Experimental results show that this method can identify typical rectangular parking spaces and achieve ideal detection results. X. Ascar Davix extracts the shape of rectangular license plates using RGB color channels and grayscale space. By using the fixed color of the license plate and determining candidate regions based on the components of the RGB color space, the method then determines the rectangular region through outlier removal and vertex detection, achieving rectangular license plate region detection in specific situations. This type of method compensates for the shortcomings of rectangle extraction based on corner detection to a certain extent and can achieve rectangle detection and extraction in complex environments. However, due to the use of points, lines, or other supplementary features, a large amount of redundant information is easily extracted during the rectangle extraction process, resulting in high algorithm complexity and low operational efficiency. In addition, features other than points and lines are often not suitable for accurate size measurement.

[0008] Compared to the two aforementioned rectangle detection methods, the rectangle extraction algorithm based on line detection avoids Hough space transformation because it does not utilize the vertex features of the rectangle, improving the efficiency of rectangle detection and extraction. Furthermore, during the rectangle detection process, the edge information of the rectangle is not completely lost (three edges cannot define a rectangle). To address the situation where some edge information of the rectangle is lost, after extracting the edge information, the edge information is restored through mathematical operations on the lines, such as extension, connection, length weighting, and edge gradient. The rectangle is then extracted by exploiting the properties of intersection of rectangle edges and parallelism of opposite sides. Eun-Sung Yang et al. detect possible rectangular boundaries in natural images through image preprocessing and line detection. They then assume that the rectangles projected onto the two-dimensional image plane have the same vanishing point on the same plane and determine quadrilaterals based on the vanishing points. Experimental results show that this algorithm can more accurately detect quadrilaterals in images. This type of algorithm performs well when the image information is relatively simple, but faces the problem of selecting rectangle vertices when multiple rectangles are present. Furthermore, complex environmental interference can also affect this method.

[0009] In practical applications, the same image often contains multiple rectangles. During rectangle extraction, different rectangles can interfere with each other, affecting extraction accuracy and stability. Hough transform window-based rectangle detection methods use a preset sliding window to traverse the image, selecting and extracting edges from multiple rectangles. This method is not only inefficient but also lacks autonomy in rectangle selection, and its accuracy is significantly affected by the preset sliding window. Robust quadrilateral detection methods use preset weights to determine the ratio of line segment extensions. Then, any quadrilateral in an image containing multiple quadrilaterals is identified based on the four intersection points of the lines within a certain range. This method improves quadrilateral detection in natural scenes. However, due to the varying sizes of quadrilaterals and unknown spacing between them, the preset weights and intersection detection range are prone to deviations, resulting in some missed detections. Furthermore, this method fails to obtain information about the quadrilateral's position within the image, making it unsuitable for shape dimensional measurement. Furthermore, in the transportation sector, there are methods that detect and distinguish rectangular license plates or traffic signs of specific shapes based on color features. However, these methods have significant limitations and are not applicable in the industrial sector.

[0010] In the industrial field, cuboids usually appear in the form of grooves, bosses, plates, metal keys, characteristic structures, etc. The cuboid has the following characteristics:

[0011] 1. Due to unknown limitations of the shooting scene and the camera, the camera usually cannot shoot directly at a cuboid or an object containing a cuboid. Therefore, the faces of the cuboid are usually displayed as quadrilaterals in the image.

[0012] 2. The area or position containing the cuboid will appear with other parts, causing mutual interference;

[0013] 3. There are multiple visible rectangles in a regular cuboid, and adjacent rectangles share a common boundary;

[0014] 4. Surface damage and edge wear of the cuboid can cause interference and the disappearance of some boundaries or vertices;

[0015] 5. The surface of the workpiece containing a rectangular block may have strong reflections.

[0016] Furthermore, in the industrial field, in addition to extracting the shape of a cuboid, the precise measurement of its dimensions is also a key concern. This places higher demands on cuboid extraction methods, but there is almost no research on cuboid extraction. Existing rectangle or quadrilateral detection methods have proven their feasibility, but due to the characteristics of industrial images and the spatial structural differences between rectangles and cuboids, rectangle extraction methods are not suitable for extracting cuboid shapes. In addition, existing rectangle extraction methods for industrial images generally have low computational efficiency when relying on Hough space transformation and are less robust to the presence of multiple rectangles in the image. For cuboid shapes, when the camera plane is imaged, they appear as multiple adjacent quadrilaterals, which not only loses the spatial feature information of the cuboid, but also the planar feature information such as parallelism of opposite sides and perpendicularity of adjacent sides. This also greatly interferes with the extraction of the cuboid.

[0017] Therefore, a high-precision cuboid extraction method suitable for the industrial field is needed. Summary of the Invention

[0018] The present invention solves the problem of needing a high-precision cuboid extraction method applicable to the industrial field.

[0019] The present invention provides a method for extracting an industrial cuboid, comprising:

[0020] Rough extraction of cuboids based on target detection and precise extraction of cuboids based on line segment reconstruction in the screening area;

[0021] The target detection-based cuboid rough extraction includes:

[0022] The parallel residual convolutional neural network is used as the backbone network for cuboid object detection.

[0023] According to the described rectangular object detection backbone network and the parallel residual block fusion, a fused multi-dimensional feature is obtained;

[0024] Obtaining a target detection network model based on the fusion of the multidimensional features and the deconvolution module;

[0025] The accurate extraction of the cuboid based on the reorganization of the line segments in the screening area includes:

[0026] Perform line segment feature clustering according to the target detection network model to obtain a line segment set;

[0027] Connect the line segments according to the line segment set to obtain a straight line set;

[0028] Performing dynamic boundary screening based on the straight line set to obtain the edge of the cuboid;

[0029] Acquire the image area where the cuboid is located according to the edge of the cuboid;

[0030] A coordinate system is established according to the image area where the cuboid is located, and the vertices of the cuboid are obtained to complete the industrial cuboid extraction.

[0031] Furthermore, a preferred embodiment is provided, wherein the parallel residual block includes: a residual block, a 3*3 maximum pooling layer and a 1*1 convolution layer;

[0032] The residual block is used to compress the image size;

[0033] The 3*3 maximum pooling layer is fused with the 1*1 convolutional layer to extract image features;

[0034] The extracted image features are fused with the residual block in terms of channel dimension to obtain an increasing channel dimension.

[0035] Furthermore, a preferred embodiment is provided, wherein the increasing channel dimension is [64, 128, 256, 512, 1024].

[0036] Furthermore, a preferred embodiment is provided, wherein the deconvolution module comprises: two 2×2 deconvolution layers with a stride of 1; each deconvolution layer is activated by a rectified linear unit, and the activated deconvolution layer performs batch normalization processing.

[0037] Furthermore, a preferred embodiment is provided, wherein the segment feature clustering is performed according to the classification network model to obtain a segment set, and the score function score of each category of segment is set. m :

[0038]

[0039] Among them, cluster(·) represents the clustering function, G m is a set of line segments, l L is the length of the line segment, k L is the slope of the straight line.

[0040] Furthermore, a preferred embodiment is provided, wherein connecting the line segments according to the line segment set to obtain the straight line set includes obtaining parameter estimates of the fitted straight line:

[0041]

[0042] Among them, W i is the line segment weight, S is the loss function, a, b and c are the parameters of the final fitted line, and (u, v) is the pixel coordinate.

[0043] Furthermore, a preferred embodiment is provided, wherein the dynamically screening the boundaries according to the straight line set to obtain the edges of the cuboid comprises:

[0044] Clustering the line set;

[0045] Set the threshold for the number of straight lines;

[0046] Acquire a high-confidence data set according to the clustered straight line set and the straight line confidence;

[0047] Determining the straight line similarity based on the comparison of the high confidence data set with the straight line quantity threshold;

[0048] Eliminate the similar straight lines.

[0049] Based on the same inventive concept, the present invention also provides an industrial cuboid extraction device, comprising:

[0050] A cuboid rough extraction module based on target detection and a cuboid precise extraction module based on line segment reconstruction in the screening area;

[0051] The target detection-based cuboid rough extraction module includes:

[0052] A backbone network unit, used to use a parallel residual convolutional neural network as the backbone network unit for cuboid object detection;

[0053] A multidimensional feature unit, configured to fuse the main network for detecting rectangular objects with the parallel residual blocks to obtain fused multidimensional features;

[0054] A target detection network model acquisition unit, configured to acquire a target detection network model by fusing the multidimensional features with a deconvolution module;

[0055] The cuboid accurate extraction module based on the screening area line segment reorganization includes:

[0056] A line segment set acquisition unit, configured to perform line segment feature clustering according to the target detection network model to acquire a line segment set;

[0057] a line set acquisition unit, configured to connect line segments according to the line segment set to acquire a line set;

[0058] A cuboid edge acquisition unit, configured to dynamically screen the boundaries according to the straight line set to acquire the cuboid edge;

[0059] An image region acquisition unit where the cuboid is located, configured to acquire the image region where the cuboid is located based on the edge of the cuboid;

[0060] The cuboid vertex acquisition unit is used to establish a coordinate system according to the image area where the cuboid is located, obtain the cuboid vertices, and complete the industrial cuboid extraction.

[0061] Based on the same inventive concept, the present invention also provides a computer device, including a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes an industrial cuboid extraction method described in any one of the above items.

[0062] Based on the same inventive concept, the present invention also provides a computer-readable storage medium, characterized in that a computer program is stored on the computer-readable storage medium, and when the computer program is run by a processor, an industrial cuboid extraction method as described in any one of the above items is executed.

[0063] The present invention is beneficial in that:

[0064] The present invention solves the problem of needing a high-precision cuboid extraction method applicable to the industrial field.

[0065] The present invention proposes an industrial cuboid extraction method, which includes a coarse-to-fine cuboid extraction process. First, the region of the cuboid to be detected in the image is preliminarily determined using a target detection method. The problem of extracting the edges of multiple cuboids is converted into local single cuboid detection and extraction. A filtered region segment recombination (SLR) method is then proposed to accurately extract the edges of the cuboid. Furthermore, a vertex selection method based on the intersection characteristics of spatial line segments and planes is proposed for vertex selection. In the target detection portion, the industrial cuboid extraction method redesigns the backbone network of the target detection algorithm and optimizes the feature fusion portion to ensure superior algorithm performance under a lightweight model, making the target detection algorithm more suitable for industrial applications. In the precise cuboid detection portion, the cuboid's boundary is first preliminarily extracted and screened through line feature clustering and adaptive weighted connection of line segments in the support region. Then, by screening interfering lines and similar lines, the cuboid's boundary is accurately extracted. Finally, the cuboid's vertices are determined based on the cuboid's characteristics to accurately extract the cuboid's shape. The method retains the advantages of the rectangle extraction method based on line detection, and combines it with the target detection scheme based on deep learning, thereby improving the efficiency of cuboid extraction while ensuring detection accuracy.

[0066] The present invention is applicable to the fields of industrial product quality inspection, industrial product measurement, etc. BRIEF DESCRIPTION OF THE DRAWINGS

[0067] Figure 1 This is a flow chart of an industrial cuboid extraction method according to embodiment 1;

[0068] Figure 2 This is a schematic diagram of the parallel residual block design described in the second embodiment;

[0069] Figure 3 Schematic diagram of feature fusion based on cross-layer connection according to embodiment three;

[0070] Figure 4 Schematic diagram of adaptive selection of support region line segments according to embodiment 5, where Support region is the support domain, and Pixel vote is the pixel voting strategy based on the number of pixels contained in the line segment;

[0071] Figure 5 This is a schematic diagram showing that the line segments described in embodiment 6 are all fitted into straight lines, where Total leastsquares is the total least squares method; Combination is the straight line fitting, Group is the line segment group, Line is the straight line obtained by fitting the line segment group using the total least squares method, and Effective line is the effective straight line obtained by the final fitting of the obtained straight line;

[0072] Figure 6 This is a schematic diagram of the rectangular parallelepiped imaging described in the seventh embodiment. Figure 6 (a) is a schematic diagram showing that the number of detected cuboid edges is 4. Figure 6 (b) is a schematic diagram showing that the number of detected cuboid edges is 7. Figure 6 (c) is a schematic diagram showing that the number of detected cuboid edges is 7. Figure 6 (d) is a schematic diagram showing that the number of detected cuboid edges is 9;

[0073] Figure 7 The interference line elimination and similar edge selection described in the seventh embodiment;

[0074] Figure 8 Schematic diagram of interference line elimination and similar edge selection according to embodiment seven;

[0075] Figure 9 This is a schematic diagram of vertex fitting according to the eleventh embodiment. Figure 9 (a) is a schematic diagram of the intersection of three adjacent sides of a cuboid, and (b) is a schematic diagram of the center of the cuboid with vertices as three points;

[0076] Figure 10A graph showing changes in TOP-1 values over training cycles in different networks according to the twelfth embodiment;

[0077] Figure 11 The data sample image described in Embodiment 12;

[0078] Figure 12 The target detection result described in the twelfth embodiment;

[0079] Figure 13 This is a schematic diagram of accurate extraction of the cuboid boundary according to the thirteenth embodiment; DETAILED DESCRIPTION

[0080] In order to make the technical solutions and advantages of the present invention more clearly described, several embodiments of the present invention are now described in further detail with reference to the accompanying drawings. However, the various embodiments described below are only a few preferred embodiments of the present invention and are not intended to limit the invention.

[0081] Implementation method 1, see Figure 1 This embodiment describes a method for extracting an industrial cuboid, comprising:

[0082] Rough extraction of cuboids based on target detection and precise extraction of cuboids based on line segment reconstruction in the screening area;

[0083] The target detection-based cuboid rough extraction includes:

[0084] The parallel residual convolutional neural network is used as the backbone network for cuboid object detection.

[0085] According to the described rectangular object detection backbone network and the parallel residual block fusion, a fused multi-dimensional feature is obtained;

[0086] Obtaining a target detection network model based on the fusion of the multidimensional features and the deconvolution module;

[0087] The accurate extraction of the cuboid based on the reorganization of the line segments in the screening area includes:

[0088] Perform line segment feature clustering according to the target detection network model to obtain a line segment set;

[0089] Connect the line segments according to the line segment set to obtain a straight line set;

[0090] Performing dynamic boundary screening based on the straight line set to obtain the edge of the cuboid;

[0091] Acquire the image area where the cuboid is located according to the edge of the cuboid;

[0092] A coordinate system is established according to the image area where the cuboid is located, and the vertices of the cuboid are obtained to complete the industrial cuboid extraction.

[0093] Specifically, the parallel residual convolutional neural network (PaRNet) is constructed by interweaving continuous convolutions with parallel residual blocks, improving model generalization and accuracy on a relatively lightweight network. The network first enhances feature learning through continuous multi-layer convolutions with a small number of receptive fields, broadening the receptive field of individual elements in the feature map. Parallel residual blocks are then stacked to form the backbone, enabling comprehensive feature extraction.

[0094] The method described in this embodiment retains the advantages of the rectangle extraction method based on line detection, and at the same time combines the target detection solution based on deep learning to improve the efficiency of cuboid extraction while ensuring detection accuracy.

[0095] Implementation method 2, see Figure 2 This embodiment further defines the method for extracting an industrial cuboid described in Embodiment 1, wherein the parallel residual block includes: a residual block, a 3*3 maximum pooling layer, and a 1*1 convolution layer;

[0096] The residual block is used to compress the image size;

[0097] The 3*3 maximum pooling layer is fused with the 1*1 convolutional layer to extract image features;

[0098] The extracted image features are fused with the residual block in terms of channel dimension to obtain an increasing channel dimension.

[0099] Specifically, the input of the parallel residual block structure is divided into two paths. The main trunk compresses the image size using traditional residual blocks, while the branch fully extracts image features through a 3x3 max pooling layer (Maxpool_3*3) and a 1x1 convolution layer (Con+ReLU_1*1). Finally, the main trunk and branch channels are fused to achieve an increase in channel dimension. The minimum number of residual blocks connected in series is used, with each module consisting of two residual blocks connected in series, resulting in the 35-layer classification network model PaRNet-35, which fully extracts features.

[0100] Generally speaking, interference factors in industrial images are more obvious, and target features are more subtle, requiring more refined features for effective feature fusion and target detection. Therefore, we extract the output features of the second, third, and fourth parallel residual blocks, respectively, and perform feature layer fusion by expanding the image size from bottom to top.

[0101] Implementation method three: This implementation method further limits the industrial cuboid extraction method described in implementation method two, and the incremental channel dimension is [64, 128, 256, 512, 1024].

[0102] In practical applications, the incremental channel dimension can be adjusted according to actual conditions.

[0103] Implementation method 4, see Figure 3 This embodiment further defines the method for extracting industrial cuboids described in Embodiment 1, wherein the deconvolution module comprises two 2×2 deconvolution layers with a stride of 1; each deconvolution layer is activated by a rectified linear unit, and the activated deconvolution layers are subjected to batch normalization.

[0104] Based on the traditional YOLOv3, this implementation replaces all upsampling operations with deconvolution, thereby more fully restoring the detailed features of small targets. At the same time, the output features of the lowest layer, that is, the feature layer with 2048 channels, are fused into each layer of the network after repeated convolution to ensure the effective use of subtle features. We also set up multiple layers of deconvolution and perform layer-by-layer and cross-layer connections on the channel dimension. Furthermore, the output features of the second parallel residual block are effectively fused with other feature layers after passing through the convolution layer and the maximum pooling layer to ensure the completeness and comprehensiveness of the features.

[0105] The deconvolution module consists of two 2×2 deconvolution layers with a stride of 1, each activated by a rectified linear unit (ReLU) followed by batch normalization. Compared to traditional bilinear upsampling methods, the deconvolution layer increases the resolution of the feature layer while focusing more on fine-grained image recognition, which helps improve the representativeness of network features. Small changes in the input lead to larger changes in the loss function, thereby increasing the gradient and alleviating the problem of vanishing gradients. The deconvolution module amplifies the feature map and fuses it with shallow features, obtaining the final prediction layer through continuous convolution.

[0106] To further reduce the number of network model parameters and improve the feasibility of the detection method described in this embodiment for industrial application, a method based on the batch normalization (BN) layer scaling factor is selected to prune the model. This method can achieve efficient convolution channel-level model pruning, is easy to operate, and has excellent pruning effects.

[0107] Implementation 5. This implementation is a further limitation of the industrial cuboid extraction method described in Implementation 1. The line segment feature clustering is performed according to the classification network model to obtain a line segment set, and the score function score of each category of line segment is set. m :

[0108]

[0109] Among them, cluster(·) represents the clustering function, G mis a set of line segments, l L is the length of the line segment, k L is the slope of the straight line.

[0110] Specifically, since line detection is the basis of cuboid edge extraction, LSD is used in image line detection. The obtained lines are composed of pixels and have two characteristic information: length and slope. In cuboid detection, the slopes of the edges can be regarded as different types of lines. That is, lines with large differences in slope will not be described as lines on the same side. The length reflects the reliability of the line. Relatively short line segments are usually interference. However, damage to the edge of the object will cause the detected line segments to be discontinuous. Using only length judgment will inevitably lead to the loss of effective information.

[0111] This embodiment integrates the information of length and slope to eliminate straight lines. Suppose the line segment set φ obtained by LSD is D ={L1,L2,...,L N}. For the nth line segment,

[0112]

[0113] in, The coordinates of the line segment endpoints obtained by the LSD algorithm.

[0114] Since there are lines with non-existent slopes, it is not necessary to obtain the exact slope value during the initial screening. Therefore, the deviation δ is introduced to obtain the quasi-slope to eliminate the situation where the slope does not exist. The quasi-slope of the nth line segment The length of the segment with the nth segment Calculated by the following formula:

[0115]

[0116] Since the sizes of different objects to be tested vary greatly, it is impossible to manually give a threshold to distinguish them. Therefore, this embodiment first uses a clustering method to perform preliminary screening. Suppose that N line segments are divided according to Divided into M categories, there are:

[0117] φ D ={L1,L2,...,L N}={G1,G2,...,G M},

[0118] G m is a line segment set, which contains several line segments L. Taking into account its feature information, the score function score of each category of line segments is set m as follows:

[0119]

[0120] Among them, cluster(·) represents the clustering function, and the sum of the lengths of the line segments in each class is defined as the score of this class. Then score m Sort from large to small and get the max([M / 3]+1,2) line segment set G with the highest score m1 Get the new line segment set φ C It should be noted that feature clustering is only a preliminary screening to improve the efficiency of program processing and reduce the workload of subsequent accurate identification. Therefore, it is feasible to obtain any of the previous items here. The key is to pay attention to not obtaining too few items to cause the effective line segments to be eliminated. The max([M / 3]+1 ,2 ) is the empirical value obtained from a large number of experiments, that is, the two-category line segment set G with the highest score when M < 5 m1 Otherwise, take the line segment set G according to the number of [M / 3]+1 m2 .

[0121] Implementation method six, see Figure 4 and Figure 5 This embodiment further defines the method for extracting an industrial cuboid described in Embodiment 1, wherein connecting line segments according to the line segment set to obtain a straight line set includes obtaining parameter estimates of the fitted straight line:

[0122]

[0123] Among them, W i is the line segment weight, S is the loss function, a, b and c are the parameters of the final fitted line, and (u, v) is the pixel coordinate.

[0124] In practical applications, a line segment of one side of the cuboid may contain a combination and splicing of multiple supporting line segments, and we need to group and connect the discovered line segments. Any two consecutive supporting line segments must meet the continuity and consistency conditions before they can be connected, and the calculation method is shown in the following formula. Under the continuity condition, the distance between the head and tail of the two line segments should be close enough. Under the consistency condition, the angle between the two line segments should be less than the tolerance angle threshold. In each group of line segment support groups, consider any line segment L. For continuity, there must be another line segment that meets the continuity with it, and for consistency, all other line segments must meet it. If there are multiple line segments that meet the above conditions in the support area at the same time, such as L1 and L2, then the line segment containing the largest number of pixels can be selected according to the voting principle (such as Figure 4 The specific algorithm is shown in Algorithm 1.

[0125]

[0126]

[0127]

[0128] Algorithm 1 Line segment adaptive selection algorithm

[0129]

[0130]

[0131] After selection, the straight lines are divided into different line segment sets according to their continuity and consistency, and each line segment set contains more than one line segment. In industrial images, due to the influence of noise, wear, scratches, lighting and other factors, the number of line segment sets is usually more than the number of boundaries of the rectangular blocks in the field of view. Therefore, it is necessary to sort the filtered line segment sets to determine the possibility that each line segment set is the boundary of the rectangular block. In actual research, in the process of coarse extraction of the edge of the rectangular block, since the boundary of the rectangular block is the main feature, the total length of the line segments in the line segment set presented by the edge of the rectangular block is usually higher than the total length of the line segments in the line segment set composed of interference factors. For each line segment set g i The total length and confidence of the line segments in the are calculated. The larger the total length, the higher the confidence. The line segment set is sorted based on this. The line segment set with high confidence is more likely to be the boundary of the cuboid. The line segment set with high confidence is obtained as φ G ={g1,g2,…,g n}.

[0132] Any segment set g after the above classification contains more than one segment. In most segment sets g, a single segment cannot represent the complete boundary of the cuboid. This embodiment uses the regional least squares method to fit the segments in the same segment set g into a straight line, thereby achieving a rough extraction of the cuboid boundary. The method is as follows:

[0133] 1. Take the upper left corner of the image as the coordinate origin and establish the image coordinate system uov;

[0134] 2. According to the starting point of the line segment (u s , v s ) and the end point (u e , v e ), determine the equation of the line where the line segment lies in the image coordinate system uov, a·u+b·v+c=0;

[0135] 3. Using the segment set as the calculation area, the segments in the same segment set g are fitted into a straight line using the weighted least squares method with adaptive weights based on the length of the segment, and the equation of the line on which the segment lies is calculated;

[0136] When weighted fitting is performed on multiple line segments, the parameter estimates of the fitted straight line should satisfy the following formula:

[0137]

[0138] Right now

[0139] Among them, W i is the weight, S is the loss function. When S takes the minimum value, the values of a, b, and c are the parameters of the final fitted straight line. and The corresponding parameters can be obtained by taking partial derivatives respectively.

[0140]

[0141] Among them, p i is the length of any line segment in the line segment set g, n is the number of line segments in the line segment set g, n and p i The values of are adaptively determined by the line detection algorithm and the line feature clustering results.

[0142] After fitting by the above method, all line segments in the line segment set are fitted into straight lines. The confidence of the corresponding line segment set is the confidence of the straight line fitted by the corresponding line segment set. All the straight lines obtained by fitting are used as the edges of the cuboid obtained by preliminary screening, which are recorded as the line set φ PR .

[0143] Implementation method seven, see Figure 6 、 Figure 7 、 Figure 8 and Figure 9 This embodiment further defines the method for extracting an industrial cuboid described in Embodiment 1, wherein the dynamic boundary screening based on the line set to obtain the edge of the cuboid includes:

[0144] Clustering the line set;

[0145] Set the threshold for the number of straight lines;

[0146] Acquire a high-confidence data set according to the clustered straight line set and the straight line confidence;

[0147] Determining the straight line similarity based on the comparison of the high confidence data set with the straight line quantity threshold;

[0148] Eliminate the similar straight lines.

[0149] This embodiment is described in conjunction with the sixth embodiment. The line set φ obtained after the preliminary screening is PR ={L1,L2,…,L I}, theoretically, it can contain countless straight lines, but the actual number of straight lines is unknown but finite. As for the cuboid, in the visual imaging process, no matter how the camera and the cuboid are placed relative to each other, the 12 sides of the cuboid can never be fully detected. Figure 6 This shows several possible situations that may occur in theory. According to different shooting angles, the number of edges of the cuboid that can be detected is theoretically 4 (such as Figure 6 (a)), 7 (as shown Figure 6 (b) and Figure 6 (c)) or 9 (as Figure 6 (d)), and due to the different camera shooting angles, there may be two edges ( Figure 6 In (c), B2 and C2) are close to coincident, and the number of detected edges should be 5. In practice, for a single cuboid, the camera shooting angle can be easily adjusted to ensure that 9 edges of the cuboid can be clearly imaged ( Figure 6 (d)), and when multiple cuboids appear at the same time, it is impossible to completely guarantee that all cuboids are imaged in the best state. In addition, for industrial images, there will be the influence of lighting, complex environment, etc., and some boundaries may disappear. Therefore, when performing dynamic screening of lines, the threshold of the number of lines should be set. The range should be 4-9, determined according to the actual imaging situation.

[0150] Normally, due to the wear and tear of the edges and corners on the surface of industrial objects, edge features are usually not prominent. During the extraction process, multiple lines on one side are often present. In addition, there may be some interfering lines after the straight lines are connected. Figure 7 Here, l1 and l2 are repeated boundary interferences, and l3 and l4 are interfering lines. These repeated boundaries and interferences will interfere with the detection and extraction of the cuboid. Therefore, this embodiment further filters the lines obtained by the initial screening by dynamically expanding the lines to obtain the precise edges of the cuboid.

[0151] When screening, first the line set φ PR The straight line L:ax+by+c=0 in the image is used to remove the interfering straight line. Generally, compared with the edge straight line of the cuboid, the line segments fitted into the interfering straight line have fewer feature segments and the confidence of the interfering straight line is lower. In order to reduce the interference of the interfering straight line as much as possible in the subsequent screening of the edge of the cuboid, the φ PR The straight lines in the cluster are clustered based on the angle of the straight line Normally, a cuboid appears as an arbitrary quadrilateral when imaged, but since its essence is still a rectangle, the angle difference between the opposite sides of the corresponding imaged quadrilateral is small. Based on this, interfering lines are further eliminated. That is:

[0152]

[0153]

[0154] In the clustering process, the angle difference threshold is set to δθ, that is, the angles of any two straight lines satisfy δθ ij <δθ, the lines are determined to be of the same type. After clustering is completed, isolated lines are removed to eliminate interfering lines.

[0155] After the interference lines are removed, according to the preset line number threshold and the confidence level of the line, the line is selected from φ PR Preselect the n straight lines with the highest confidence, denoted as the straight line set φ L When repeated edge line removal, first L The feature similarity of the n straight lines is judged based on the angle and intercept of the straight line. A coordinate system xoy is established with the center of the image as the coordinate origin. The angles of the n straight lines with the highest confidence and their intercepts on the x-axis in the xoy coordinate system are calculated respectively. The angle difference and intercept difference of any two straight lines are compared, and their absolute values are taken and compared with the pre-set thresholds δθ and δb. If the calculated values are less than the thresholds at the same time, the two straight lines are judged to be similar. After the similarity judgment is completed, the similar straight lines are eliminated, that is, only one similar straight line is selected. When selecting, considering the common cases of workpiece wear, damage and chamfering in the industrial field, the actual boundary will be between similar straight lines. Therefore, the similar straight lines are fitted into a straight line as the edge line of the cuboid. For similar lines l1: a1x+b1y+c1=0 and l2: a2x+b2y+c2=0, at the same time, The fitted straight line l:ax+by+c=0 satisfies a=(a1+a2) / 2,b=(b1+b2) / 2,c=(c1+c2) / 2. After similar straight lines are eliminated, the straight line set φ is selected based on the calculated straight line confidence. PR Select a line to fill in the line set φ L In, until φ L The n straight lines in are not similar to each other, and the similar straight lines are screened and eliminated. The confidence calculation formula of the straight line is:

[0156] Confidence = the number of images in which all cuboids are detected correctly / the total number of images.

[0157] Specifically, the dynamic segment screening algorithm is as follows:

[0158]

[0159] Embodiment 8: An industrial cuboid extraction device according to this embodiment includes:

[0160] A cuboid rough extraction module based on target detection and a cuboid precise extraction module based on line segment reconstruction in the screening area;

[0161] The target detection-based cuboid rough extraction module includes:

[0162] A backbone network unit, used to use a parallel residual convolutional neural network as the backbone network unit for cuboid object detection;

[0163] A multidimensional feature unit, configured to fuse the main network for detecting rectangular objects with the parallel residual blocks to obtain fused multidimensional features;

[0164] A target detection network model acquisition unit, configured to acquire a target detection network model by fusing the multidimensional features with a deconvolution module;

[0165] The cuboid accurate extraction module based on the screening area line segment reorganization includes:

[0166] A line segment set acquisition unit, configured to perform line segment feature clustering according to the target detection network model to acquire a line segment set;

[0167] a line set acquisition unit, configured to connect line segments according to the line segment set to acquire a line set;

[0168] A cuboid edge acquisition unit, configured to dynamically screen the boundaries according to the straight line set to acquire the cuboid edge;

[0169] An image region acquisition unit where the cuboid is located, configured to acquire the image region where the cuboid is located based on the edge of the cuboid;

[0170] The cuboid vertex acquisition unit is used to establish a coordinate system according to the image area where the cuboid is located, obtain the cuboid vertices, and complete the industrial cuboid extraction.

[0171] Implementation method 9. A computer device described in this implementation method includes a memory and a processor, wherein a computer program is stored in the memory. When the processor runs the computer program stored in the memory, the processor executes an industrial cuboid extraction method described in any one of implementation methods 1 to 7.

[0172] Implementation method 10: A computer-readable storage medium described in this implementation method stores a computer program, and when the computer program is executed by a processor, an industrial cuboid extraction method as described in any one of implementation methods 1 to 7 is executed.

[0173] Implementation method 11, see Figure 8 and Figure 9This embodiment further defines the method for extracting an industrial cuboid described in the embodiment, wherein a coordinate system is established based on the image region where the cuboid is located to obtain the cuboid vertices, specifically:

[0174] Combine Figure 6 The selection of the vertices of the cuboid shown in (d) is explained when there are 9 boundaries in the cuboid imaging.

[0175] The detected cuboid boundary has infinite extension, but the area where the cuboid is located is limited after target detection. First, the uov coordinate system is established based on the image area where the cuboid is located. Figure 8 As shown in (a), the intersection of the rectangular boundary outside the image area cannot be the vertex of the rectangular parallelepiped. The 9 boundaries obtained by screening are divided into 3 categories {A1, A2, A3}, {B1, B2, B3}, {C1, C2, C3} according to the differences in the 3 groups of opposite side angles of the rectangular parallelepiped, and the classified lines are numbered according to their positions in the coordinate system. The purpose of numbering the lines here is to determine the order of each line in each category of lines, and the numbering basis does not affect the subsequent vertex selection. For the rectangular parallelepiped, if any three lines A, B, and C have an intersection, then the point must be a vertex of the rectangular parallelepiped. This is determined by the properties of the rectangular parallelepiped. For example, the lines A2, B1, and C1 intersect at the same point p3, which is a vertex of the rectangular parallelepiped. Similarly, points p9 and p 10 , p 14 In addition, for a cuboid, since it is actually a three-dimensional structure, there are only two points on any boundary that are vertices of the cuboid. Among the vertices determined above, points p3p9, p9p 10 , p9p 14 The corresponding cuboid boundary segments A2, B2, and C2 can be determined by collinearity, and the interference points p2, p4, p7, p8, and p 12 , p 16 , Figure 8 Finally, calculate the distances between the known points and determine the remaining vertices of the cuboid based on the distances from the other points to the known points. For example, calculate p3p9, p9p 10 , p3p 10 The distance between Then calculate the other intersection points p on line A1 to points p3, p9, p 10 distance If both meet It can be determined that point p5 is the vertex of the cuboid. Similarly, point p 11 , p 15 are the vertices of the cuboid, such as Figure 8As shown in (c), accurate detection and extraction of the cuboid is completed.

[0176] In practice, due to the existence of interference, the three adjacent sides of the cuboid do not necessarily intersect at one point. Figure 9 As shown in (a), in the selection of the intersection points of the three types of straight lines, the intersection screening threshold ρ is pre-set. If the pixel distance of the three intersection points of any three straight lines of different classes is less than the preset threshold, then the three intersection points are determined to be a vertex of a cuboid, and the center of the three points is taken as the vertex of the cuboid, as shown in Figure 9 (b) shown.

[0177] This approach accurately detects the edges of industrial cuboids. For objects in different scenarios, a supplemental lighting system and a binocular camera imaging system can be built, and parameters can be designed and adjusted through simulation experiments. Ultimately, this results in object contour detection tailored to a specific scenario.

[0178] Implementation Method 12: See Figure 10 、 Figure 11 、 Figure 12 、 Figure 13 This embodiment provides a specific example of the industrial cuboid extraction method described in the first embodiment, and is also used to explain the first to seventh embodiments. Specifically:

[0179] This implementation applies PaRNet to the well-known CIFAR-10 dataset. We evaluate our network using common metrics such as top-1 error rate, top-5 error rate, and accuracy. Finally, we compare our approach with popular networks such as ResNet to verify its effectiveness. We train using mini-batch gradient descent with a learning rate of 0.1, momentum of 0.1, and a batch size of 16 for 1000 epochs. The experimental environment consists of a GTX 2070, Ubuntu 16.04, PyTorch 1.6.1, CUDA 10.1, CUDNN 7.6.5, and Python 3.6.9.

[0180] Table 1 shows the model parameters, floating-point computational load (FLOPs), TOP-1 error rate, and TOP-5 error rate of PaRNet-35. We compared it with ResNet-35, ResNet-50, ResNet-101, and GoogLeNet. It is not difficult to see from the number of parameters and computational load that compared to networks with similar numbers of layers, PaRNet has more parameters but less computational load, thanks to the advantages of parallelism and reflects a similar sparsity to the GoogLeNet network. As the number of network layers increases, ResNet's accuracy improves, and our network has absorbed this advantage. In the first 1,000 training runs, both PaRNet and PaRNet were able to obtain more accurate classification results with low network load.

[0181] Table 1 Performance of different networks on CIFAR-10

[0182]

[0183]

[0184] Figure 10 This article shows the dynamic trend of the accuracy of the aforementioned network on CIFAR-10 as it iterates. PaRNet outperforms traditional networks in both the rate of decline and oscillation in the early stages, as well as stability and accuracy in the later stages. Firstly, the inclusion of parallel modules allows for network offloading, resulting in a larger descent gradient in the early stages. Consequently, the PaRNet network's classification error exhibits a faster rate of decline overall. Secondly, the PaRNet network model is more stable and robust. After 500 cycles, the PaRNet curve becomes smoother.

[0185] Furthermore, target detection performance verification is performed:

[0186] There are many target detection data sets, but there is no data set for specific rectangular industrial shapes. Therefore, this embodiment combines the actual situation and preliminarily shoots and produces the rectangular target detection data set used in this embodiment in a laboratory environment. The data set consists of 4000 images, each of which includes 2-7 rectangular blocks. It contains 4 types of rectangular blocks that may be used in industry, and their materials are metal (2 types, obtained by wire cutting), acrylic (1 type, laser cutting) and abs0 plastic (3D printing). The images contain common images of different situations such as target blur, target wear, weak targets and strong light interference, such as Figure 11As shown. The dataset images are randomly divided into training set and test set in a ratio of 3:1, and are used for training and testing of YOLOv3, YOLOv4, YOLOv5 and the method described in this embodiment. During training, the Adam (Adaptive Moment Estimation) algorithm is used for optimization, the initial learning rate is designed to be 0.001, and the StepLR mechanism is used to dynamically reduce the learning rate. Due to the lack of corresponding pre-training weights caused by the change in network structure, the pre-training weights are also removed when training with the YOLOv3 algorithm. Frozen training is used in the training method to prevent the weights from being destroyed in the early stage of training while speeding up the training speed. The frozen training generations are set to 50 times and the non-frozen training generations are set to 100 times.

[0187] For cuboid object detection, the main focus is on whether the cuboid can be correctly detected and whether multiple cuboids in the image can be fully detected. Therefore, in addition to using the traditional mAP metric to evaluate the results, the false alarm rate (FAR) and missed detection rate (MDR) are introduced as supplementary evaluation metrics. They can be calculated using Equation (1).

[0188]

[0189] Among them, AlarmPic is the number of images with false alarms; MissedPic is the number of images with missed detections; TotalPic is the total number of images.

[0190] The cuboid detection results are as follows Figure 12 As shown in Table 2, balancing detection accuracy and real-time performance, we obtained four metrics for cuboid detection: mean average detection accuracy (mAP), false alarm rate (FAR), missed detection rate (NDR), and frame rate per second (FPS) (Table 2). Under the same conditions, all four detection methods achieved excellent detection results, with mAP exceeding 95%, false alarm rates below 0.02%, and false alarm rates below 2%. The proposed method achieved higher accuracy (mAP: 96.28%, FAR: <0.01%, MDR: 1.36%), while YOLOv5m achieved a speed advantage (FPS up to 43.03%). YOLOv4 also performed more balanced. In industrial applications, both object detection accuracy and real-time performance are crucial factors influencing the value of an algorithm. Therefore, different detection methods should be selected based on the specific application. Maximizing detection accuracy while ensuring adequate detection speed is key to ensuring its value.

[0191] Table 2 Performance evaluation of target detection algorithms

[0192]

[0193] Furthermore, to ensure that the model is more suitable for industrial scenarios, we use a model optimization method based on the BN layer coefficient to prune the proposed object detection model, set the sparse weight to 0.0001, set different pruning rates, and fine-tune the model after pruning. The performance of the model under different pruning rates is shown in Table 3:

[0194] Table 3 Model pruning results

[0195]

[0196] The table above shows that the object detection algorithm proposed in this embodiment shows minimal change in various evaluation metrics when the pruning rate is within 50%. The model maintains detection accuracy, significantly reduces model size and parameters, and significantly improves detection speed. However, at a pruning rate of 60%, mAP and FAR show significant changes, indicating a decline in model performance. Based on the data in the table, this embodiment selected a model with a 40% pruning rate as the object detection model for subsequent experiments, as this model achieved optimal detection accuracy.

[0197] Furthermore, the target detection area is used as the segmentation unit to segment the cuboid area obtained by the preliminary detection, and on this basis, the accurate cuboid extraction effect is verified.

[0198] In this embodiment, the evaluation indicators of the cuboid extraction method performance adopt the evaluation indicators for multi-target extraction and the evaluation indicators for multi-target extraction in industrial images, which are precision, recall, F1-measure and reliability. Their definitions are as follows:

[0199] Precision = number of extracted cuboids / total number of cuboids

[0200] Recall = the number of correct cuboids extracted / the total number of cuboids appearing in the test image

[0201] F1=2*precious*recall / (precision+recall)

[0202] Confidence = the number of images in which all cuboids are detected correctly / the total number of images

[0203] The larger the values of all the above evaluation indicators, the better the performance of the algorithm. The confidence level also affects other evaluation indicators. That is, the higher the confidence level, the higher the corresponding precision, recall, and f1-measure values. Precision is the accuracy, and Recall is the recall rate.

[0204] This embodiment selects 100 image data as test data, each image includes 2-9 different types of cuboids, and there are damage or scratches on the surface of the metal cuboids, which is closer to the actual industrial scene. In addition, in order to ensure the consistency of the settings, the imaging conditions of the cuboids in the selected test images are roughly the same. The detection results are shown in Table 4. The accuracy and confidence of the cuboid detection and extraction method proposed in this embodiment can reach more than 95%, and the extraction time of a single cuboid is only 0.27s, which is feasible for practical application. In addition, since the precise extraction of cuboids is based on target detection, all extracted cuboids are of correct cuboid shape, so the accuracy, Recall and F1 values are the same (98.35%). This is also one of the advantages of this method, that is, the extracted cuboids are most likely the target objects, ensuring that the detection and extraction results are more reliable.

[0205] Table 4. Performance evaluation of cuboid boundary extraction

[0206] Precision Recall F1-measure Reliability Time 98.35 98.35 98.35 95％ 0.27s

[0207] The above describes the present application in detail through specific implementation methods, but the above is only a preferred implementation method of the present application and is not intended to limit the present application. Any modifications, combinations of implementation methods, equivalent replacements and improvements made within the spirit and principle of the present application should be included in the scope of protection of the present application.

Claims

1. A method for extracting industrial cuboids, characterized in that: The method comprises: Rough extraction of cuboids based on target detection and precise extraction of cuboids based on line segment reconstruction in the screening area; The target detection-based cuboid rough extraction includes: The parallel residual convolutional neural network is used as the backbone network for cuboid object detection. According to the described rectangular object detection backbone network and the parallel residual block fusion, a fused multi-dimensional feature is obtained; Obtaining a target detection network model based on the fusion of the multidimensional features and the deconvolution module; The accurate extraction of the cuboid based on the reorganization of the line segments in the screening area includes: Perform line segment feature clustering according to the target detection network model to obtain a line segment set; Connect the line segments according to the line segment set to obtain a straight line set; Performing dynamic boundary screening based on the straight line set to obtain the edge of the cuboid; Acquire the image area where the cuboid is located according to the edge of the cuboid; A coordinate system is established according to the image area where the cuboid is located, and the vertices of the cuboid are obtained to complete the industrial cuboid extraction.

2. The method for extracting an industrial cuboid according to claim 1, characterized in that: The parallel residual block includes: a residual block, a 3*3 maximum pooling layer and a 1*1 convolutional layer; The residual block is used to compress the image size; The 3*3 maximum pooling layer is fused with the 1*1 convolutional layer to extract image features; The extracted image features are fused with the residual block in terms of channel dimension to obtain an increasing channel dimension.

3. The method for extracting an industrial cuboid according to claim 2, characterized in that: The increasing channel dimensions are [64, 128, 256, 512, 1024].

4. The method for extracting an industrial cuboid according to claim 1, characterized in that: The deconvolution module includes: two 2×2 deconvolution layers with a stride of 1; each deconvolution layer is activated by a rectified linear unit, and the activated deconvolution layers are subjected to batch normalization processing.

5. The method for extracting an industrial cuboid according to claim 1, characterized in that: The method of clustering line segment features according to the target detection network model to obtain a line segment set also includes setting a score function for each category of line segments. : ， in, represents the clustering function, is a set of line segments, is the length of the line segment, is the slope of the line segment.

6. The method for extracting industrial cuboids according to claim 1, characterized in that: Connecting the line segments according to the line segment set to obtain a straight line set includes obtaining parameter estimates of the fitted straight line: ， in, is the line segment weight, is the loss function, , and The value is the parameter of the final fitted straight line. is the pixel coordinate.

7. The method for extracting industrial cuboids according to claim 1, characterized in that: The dynamically screening the boundaries according to the line set to obtain the edges of the cuboid includes: Clustering the line set; Set the threshold for the number of straight lines; Acquire a high-confidence data set according to the clustered straight line set and the straight line confidence; Determining the straight line similarity based on the comparison of the high confidence data set with the straight line quantity threshold; Eliminate the similar straight lines.

8. An industrial cuboid extraction device, characterized in that: The device comprises: A cuboid rough extraction module based on target detection and a cuboid precise extraction module based on line segment reconstruction in the screening area; The target detection-based cuboid rough extraction module includes: A backbone network unit, used to use a parallel residual convolutional neural network as the backbone network unit for cuboid object detection; A multidimensional feature unit, configured to fuse the main network for detecting rectangular objects with the parallel residual blocks to obtain fused multidimensional features; A target detection network model acquisition unit, configured to acquire a target detection network model by fusing the multidimensional features with a deconvolution module; The cuboid accurate extraction module based on the screening area line segment reorganization includes: A line segment set acquisition unit, configured to perform line segment feature clustering according to the target detection network model to acquire a line segment set; a line set acquisition unit, configured to connect line segments according to the line segment set to acquire a line set; A cuboid edge acquisition unit, configured to dynamically screen the boundaries according to the straight line set to acquire the cuboid edge; An image region acquisition unit where the cuboid is located, configured to acquire the image region where the cuboid is located based on the edge of the cuboid; The cuboid vertex acquisition unit is used to establish a coordinate system according to the image area where the cuboid is located, obtain the cuboid vertices, and complete the industrial cuboid extraction.

9. A computer device, characterized in that: The invention comprises a memory and a processor, wherein a computer program is stored in the memory, and when the processor runs the computer program stored in the memory, the processor executes the industrial cuboid extraction method described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the method for extracting an industrial cuboid according to any one of claims 1 to 7 is executed.

Images