Intelligent detection method for hydraulic support side guard plate bending processing

Abstract

This invention belongs to the technical field of intelligent detection, specifically relating to an intelligent detection method for bending processing of hydraulic support side guard plates. It addresses the technical problem that limitations in data acquisition and enhancement in existing technologies restrict the generalization ability of the model. The method includes the following steps: S1, based on the object mask of image A, cropping the target image from image A and pasting it into image B to generate a hybrid image; S2, performing pixel-level weighted fusion of the hybrid image and image C to generate a hybrid image; S3, replacing the corresponding high-frequency subbands in the hybrid image with perturbation high-frequency subbands; S4, pairing the enhanced image with a second hybrid label to form enhanced training samples for training the detection model. The enhanced training samples generated by this invention are not only visually highly faithful but also richer in semantics and features.

Description

Technical Field

[0001] This invention belongs to the technical field of intelligent detection, specifically relating to an intelligent detection method for bending processing of hydraulic support side guard plates. Background Technology

[0002] Hydraulic supports are supporting equipment for fully mechanized coal mining faces. The side plates of hydraulic supports are mainly used for protection and load bearing. The dimensional accuracy, surface flatness, and absence of defects in the side plates during the bending process are crucial to the stability and service life of the hydraulic supports.

[0003] In traditional technology, the quality inspection of hydraulic support side guard plates mainly relies on manual visual inspection. Manual inspection is not only labor-intensive and inefficient, but the results are also easily affected by subjective factors (such as experience and fatigue), which poses a risk of missed or false inspections. This makes it difficult to meet the higher requirements of modern coal mines for production quality and efficiency.

[0004] Utilizing machine vision and deep learning technologies to automate and intelligently inspect the side plates of hydraulic supports has become an inevitable trend in the industry. However, the performance of deep learning-based intelligent inspection models is highly dependent on large-scale, high-quality, and diverse training datasets. In real industrial environments, limitations such as production conditions, lighting variations, dust interference, and the scarcity of defect samples make acquiring and labeling sufficient training image data costly and time-consuming. This results in limited training dataset size and insufficient sample diversity, hindering the generalization ability of intelligent inspection models.

[0005] Data augmentation techniques are used to expand datasets, primarily including basic methods based on geometric transformations (such as flipping, rotating, and scaling) and color space transformations (such as adjusting brightness or contrast), as well as some more complex hybrid augmentation methods. While basic methods are simple to operate, the generated new samples are overly correlated with the original images, failing to introduce new contextual information and thus limiting their ability to improve the model's ability to handle complex backgrounds and unconventional poses. Hybrid augmentation methods (such as Mixup) often result in semantically ambiguous and feature-unclear images by linearly interpolating two images. Another hybrid augmentation method (CutMix) pastes a rectangular region from one image onto another, preserving the clarity of local regions but creating harsh boundaries at the stitching points, causing visual discontinuities and unrealistic appearances. Furthermore, simple label mixing strategies struggle to accurately reflect the complex composition of targets in the new image. In addition, some methods based on frequency domain processing or Generative Adversarial Networks (GANs) either suffer from operational complexity and high computational costs, or from difficulties in controlling the fidelity and stability of the generated images, resulting in unsatisfactory performance in specific industrial scenarios. Therefore, existing data augmentation techniques still have limitations in generating high-quality, high-fidelity samples with rich contextual information, making it difficult to simulate the various complex situations that may occur in the hydraulic support side guard plate under real working conditions. Summary of the Invention

[0006] This invention provides an intelligent detection method for bending the side guard plate of a hydraulic support, in order to solve the technical problem that the limitations of existing technologies in data acquisition and enhancement restrict the generalization ability of the model.

[0007] A smart detection method for bending of hydraulic support side guard plate includes the following steps:

[0008] S1, randomly select images A, B, and C from the hydraulic support side plate image set, and obtain the classification labels for each image. , and Based on the object mask of image A, the target image is cropped from image A and pasted into image B to generate a blended image. Based on the object mask of image A in the blended image Area ratio According to the formula Calculate the mixed label ;

[0009] S2 will mix the images Image C is input into a weight generation network, which outputs a pixel-wise nonlinear mixing weight matrix. According to the nonlinear mixed weight matrix For mixed images The image is then pixel-weighted and fused with image C to generate a hybrid image. According to the formula Calculate the mixed label ,in Nonlinear mixed weight matrix The global mean;

[0010] S3, for mixed images Image C is subjected to multi-scale discrete wavelet transform to decompose it into low-frequency subbands and multiple high-frequency subbands. A high-frequency subband is randomly selected from image C, and its wavelet coefficients are permuted to generate a perturbed high-frequency subband. The perturbed high-frequency subband is then used to replace the mixed image. The corresponding high-frequency sub-band;

[0011] S4, using a mixed image The low-frequency subband, the unreplaced high-frequency subband, and the perturbed high-frequency subband are reconstructed using inverse discrete wavelet transform to obtain the enhanced image. ; will enhance the image With the second mixed label Pairing is used to form enhanced training samples for training the detection model, and the trained detection model is used to detect defects in the side guard plate of the hydraulic support.

[0012] Furthermore, in S1, an image set of the hydraulic support side guard plate is acquired by deploying an industrial camera on the hydraulic support side guard plate bending production line; an instance segmentation model is used to pre-annotate the object mask and classification label, and the image set is checked and confirmed manually using annotation software.

[0013] Furthermore, in S1, three non-repeating random numbers are generated by computer as indices to extract three images without replacement from the hydraulic support side plate image set, and the corresponding object mask and classification label are obtained.

[0014] Furthermore, the process of pasting the target image to a random location in image B includes the following steps:

[0015] If the size of the cropped target image is larger than image B, then the target image is scaled.

[0016] Calculate the effective area in image B that can be pasted, and randomly select a location within the effective area to ensure that the target image is completely pasted within the boundary of image B.

[0017] Furthermore, the weight generation network stitches together the blended image along the channel dimension. Using image C as input, an encoder-decoder network is employed, and a nonlinear hybrid weight matrix is ​​output based on the encoder-decoder process. Among them, the nonlinear mixed weight matrix The value of each element is normalized to the interval [0, 1].

[0018] Furthermore, generate a mixed image. The formula is: ;

[0019] in This indicates element-wise multiplication, where 1 represents AND. A matrix of all ones with the same shape.

[0020] Furthermore, category labels , and One-hot encoded vectors, mixed labels and mixed tags A vector representing the probability distribution of each category.

[0021] Furthermore, the multi-scale discrete wavelet transform is decomposed into N levels using the Daubechies wavelet basis or the Symlet wavelet basis, where N is an integer between 2 and 5.

[0022] Furthermore, when randomly selecting one of the high-frequency subbands, it is randomly drawn from all N×3 high-frequency subbands according to a uniform distribution.

[0023] Furthermore, during the permutation operation, the selected high-frequency subband is divided into multiple non-overlapping sub-blocks of preset size, and all sub-blocks are randomly rearranged to generate perturbed high-frequency subbands.

[0024] The beneficial effect is that using object masks for target pasting avoids harsh boundaries and visual discontinuities, generating a blended image with clear targets and natural edges. The blended image is then processed through a weighted generation network. Pixel-by-pixel nonlinear fusion with image C introduces rich and realistic information, mitigating the semantic ambiguity inherent in methods like Mixup. Perturbation and replacement of high-frequency subbands in the wavelet domain injects fine-grained texture details into the image, simulating subtle differences in lighting and materials in industrial settings. The enhanced training samples generated by this invention are not only visually highly faithful but also richer in semantics and features. These enhanced training samples are used to train the detection model, improving the generalization ability of the hydraulic support side guard plate detection model. Attached Figure Description

[0025] Figure 1 Generate a blended image in S1 A diagram showing the time. Detailed Implementation

[0026] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Those skilled in the art should understand that the embodiments described below are only some, not all, of the embodiments disclosed. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0027] An embodiment of the intelligent detection method for bending of hydraulic support side guard plate provided by the present invention:

[0028] A smart detection method for bending of hydraulic support side guard plate includes the following steps:

[0029] S1, randomly select images A, B, and C from the hydraulic support side plate image set, and obtain the classification labels for each image. , and Based on the object mask of image A, the target image is cropped from image A and pasted into image B to generate a blended image. Based on the object mask of image A in the blended image Area ratio According to the formula Calculate the mixed label .

[0030] Specifically, an image set of hydraulic support side guard plates is acquired by deploying industrial cameras on the hydraulic support side guard plate bending production line. Each image in the hydraulic support side guard plate image set has a corresponding object mask and classification label. Based on the association mapping between images and classification labels, the corresponding classification label for each image is obtained. , and An instance segmentation model is used to pre-annotate the object mask and classification label, and then the labeling software is used manually to verify and confirm the labeling to ensure that each side panel has an accurate pixel-level mask and category classification. The instance segmentation model can be Mask R-CNN, and the labeling software can be LabelMe.

[0031] Three unique random numbers are generated by computer as indices to extract three images without replacement from the hydraulic support side plate image set, and the corresponding object masks and classification labels are obtained.

[0032] like Figure 1 As shown, image A is multiplied pixel-by-pixel with the object mask of image A to obtain the target image, which contains only pixels of the target area, the hydraulic support side guard plate. A random location containing coordinates is generated on image B, and the target image is pasted onto this random location, overwriting the original pixels, to generate a hybrid image. Area ratio The number of pixels with a value of 1 in the object mask of image A is calculated by dividing by the number of pixels in the blended image. The total number of pixels is obtained.

[0033] In an optional embodiment, the process of pasting the target image to a random location in image B includes the following steps:

[0034] If the size of the cropped target image is larger than image B, then the target image is scaled.

[0035] Calculate the effective area in image B that can be pasted, and randomly select a location within the effective area to ensure that the target image is completely pasted within the boundary of image B.

[0036] For example, if the target image cropped from image A is 300×300 pixels, while image B is 256×256 pixels, then the target image needs to be scaled down to 256×256 pixels or smaller. Furthermore, if the scaled target image is 100×100 pixels, the width and height of the effective pasting area in image B are 256-100=156 pixels. Randomly selecting a coordinate point within this 156×156 pixel area as the starting position for pasting ensures that the target image remains completely within the boundaries of image B.

[0037] S2 will mix the images Image C is input into a weight generation network, which outputs a pixel-wise nonlinear mixing weight matrix. According to the nonlinear mixed weight matrix For mixed images The image is then pixel-weighted and fused with image C to generate a hybrid image. According to the formula Calculate the mixed label ,in Nonlinear mixed weight matrix The global mean.

[0038] In one embodiment, the weight generation network employs a lightweight U-Net structure to generate mixed images. The image C is concatenated with the image along the channel dimension and used as input. The weight generation network outputs a nonlinear hybrid weight matrix. The Sigmoid activation function is used to ensure that the weight values ​​are between 0 and 1; using the formula... Pixel-by-pixel weighted fusion is performed to obtain a composite image. ,in This indicates element-wise multiplication.

[0039] In an optional embodiment, the weight generation network stitches together the blended image along the channel dimension. Using image C as input, an encoder-decoder network is employed, and a nonlinear hybrid weight matrix is ​​output based on the encoder-decoder process. ,in The value of each element is normalized to the interval [0, 1].

[0040] Specifically, assuming a mixed image Image C and image D are both 256×256 pixel images with 3 color channels. They are concatenated along the channel dimension to form a 6×256×256 input tensor. This input tensor is fed into an encoder-decoder network (such as a U-Net structure). The encoder-decoder network performs a series of downsampling and upsampling operations to output a 256×256 pixel nonlinear mixing weight matrix with the same size as the input image. Nonlinear hybrid weight matrix Each element value (such as 0.2 or 0.9) is restricted to between 0 and 1 through methods such as the Sigmoid activation function, representing the fusion weight of the corresponding pixel.

[0041] In an optional embodiment, a blended image is generated. The formula is: ;

[0042] in This indicates element-wise multiplication, where 1 represents AND. A matrix of all ones with the same shape.

[0043] Formula for mixed images Each pixel position in the image is calculated independently. For example, for a pixel at coordinates (10, 20) in the image, if the pixel is in the blended image... The pixel values ​​in image C are [100, 150, 200], and the pixel values ​​in image C are [50, 50, 50], and the weight matrix... With a value of 0.8 at this coordinate, the generated blended image The pixel value at the pixel point is [90, 130, 170].

[0044] In an optional embodiment, classification labels , and For one-hot encoded vectors, the first mixed label Second mixed label A vector representing the probability distribution of each category.

[0045] For example, in the classification task of intelligent detection of hydraulic support side guard plates, suppose the side guard plates need to be distinguished into 3 categories (Category 1: qualified; Category 2: unqualified; Category 3: awaiting secondary detection). If the side guard plate captured by image A is initially determined to enter the "awaiting secondary detection" process, that is, its status corresponds to category 3. At this time, the one-hot encoded label of image A is a vector [0, 0, 1] with the same length as the number of categories, where only the 3rd bit of the vector is 1, and the other bits are 0.

[0046] The mixed label is used to reflect the probability distribution of samples in the multi-class classification results after sample fusion. For example, if the first mixed label... The dataset includes two categories, Category 1 and Category 2. Based on the requirements of production data fusion, it is generated by mixing the two categories at a ratio of 0.6 and 0.4. Therefore, its vector form is [0.6, 0, 0.4], representing the mixed image. There is a 60% probability that it will be classified as "qualified" and a 40% probability that it will be classified as "awaiting a second test".

[0047] S3, for mixed images Image C is subjected to multi-scale discrete wavelet transform to decompose it into low-frequency subbands and multiple high-frequency subbands. A high-frequency subband is randomly selected from image C, and its wavelet coefficients are permuted to generate a perturbed high-frequency subband. The perturbed high-frequency subband is then used to replace the mixed image. The corresponding high-frequency sub-band.

[0048] Specifically, wavelets are used to process the hybrid image. Both image C and image D undergo two-stage discrete wavelet transform to transform the mixed image. Image C is decomposed into a low-frequency subband LL and three high-frequency subbands LH, HL, and HH. One of the three high-frequency subbands is randomly selected from image C, for example, the horizontal high-frequency subband HL. The permutation operation involves randomly rearranging all wavelet coefficients within the horizontal high-frequency subband HL in image C to generate a perturbed high-frequency subband; this perturbed high-frequency subband replaces the mixed image. The corresponding horizontal high-frequency subband HL.

[0049] In an optional embodiment, the multiscale discrete wavelet transform is decomposed into N levels using the Daubechies wavelet basis or the Symlet wavelet basis, where N is an integer between 2 and 5.

[0050] Specifically, taking N=2 as an example: perform a 2-level Daubechies wavelet decomposition on the image C to generate 1 low-frequency sub-band and 6 high-frequency sub-bands. The high-frequency sub-bands capture the texture edge information of the image C at different scales and directions.

[0051] In an optional embodiment, when randomly selecting one of the high-frequency subbands, one is randomly selected from all N×3 high-frequency subbands according to a uniform distribution.

[0052] Specifically, the six high-frequency subbands are the first-level horizontal, vertical, and diagonal subbands, and the second-level horizontal, vertical, and diagonal subbands. One of these six high-frequency subbands is selected by uniform random sampling. For example, the first-level horizontal subband is selected. The selected subband will be used for subsequent perturbation operations.

[0053] In an optional embodiment, during the permutation operation, the selected high-frequency subband is divided into multiple non-overlapping sub-blocks of preset size, and all sub-blocks are randomly rearranged to generate a perturbed high-frequency subband.

[0054] Specifically, assuming the selected high-frequency subband size is 128×128 pixels and the preset sub-block size is 16×16 pixels, the high-frequency subband is divided into 64 non-overlapping sub-blocks of 8×8 pixels each. The index positions (i.e., 1-64) of these 64 sub-blocks are randomly shuffled, and all sub-blocks are reorganized according to the new position order, forming a perturbed high-frequency subband whose internal texture structure is destroyed but whose overall energy distribution remains unchanged. For example, the sub-block originally at position 1 is moved to position 50, and the sub-block originally at position 2 is moved to position 13.

[0055] S4, using a mixed image The low-frequency subband, the unreplaced high-frequency subband, and the perturbed high-frequency subband are reconstructed using inverse discrete wavelet transform to obtain the enhanced image. ; will enhance the image With the second mixed label Pairing is used to form enhanced training samples for training the detection model, and the trained detection model is used to detect defects in the side guard plate of the hydraulic support.

[0056] In an optional embodiment, the mixed images The low-frequency subband LL, the two unreplaced high-frequency subbands LH and HH, and the perturbed high-frequency subband from image C after permutation are taken as input and subjected to an inverse discrete wavelet transform corresponding to the multi-scale discrete wavelet transform to reconstruct a single complete enhanced image. .

[0057] The generated enhanced image and the calculated second mixed label As a data pair, it is stored in the enhanced training dataset. The enhanced training dataset is used to train the detection model, and the trained detection model is used to detect defects in the hydraulic support side guard plate, thus improving the detection accuracy of the detection model in complex scenarios. The detection model for the hydraulic support side guard plate can be a detection model such as YOLOv5 or Faster R-CNN.

[0058] In addition, in the description of this specification, "multiple" means at least two, such as two, three or more, etc., unless otherwise expressly and specifically defined.

Claims

1. An intelligent detection method for bending processing of hydraulic support side guard plates, characterized in that, Includes the following steps: S1, randomly select images A, B, and C from the hydraulic support side plate image set, and obtain the classification labels for each image. , and Based on the object mask of image A, the target image is cropped from image A and pasted into image B to generate a blended image. Based on the object mask of image A in the blended image Area ratio According to the formula Calculate the mixed label ; S2 will mix the images Image C is input into a weight generation network, which outputs a pixel-wise nonlinear mixing weight matrix. According to the nonlinear mixed weight matrix For mixed images The image is then pixel-weighted and fused with image C to generate a hybrid image. According to the formula Calculate the mixed label ,in Nonlinear mixed weight matrix The global mean; S3, for mixed images Image C is subjected to multi-scale discrete wavelet transform to decompose it into low-frequency subbands and multiple high-frequency subbands. A high-frequency subband is randomly selected from image C, and its wavelet coefficients are permuted to generate a perturbed high-frequency subband. The perturbed high-frequency subband is then used to replace the mixed image. The corresponding high-frequency sub-band; S4, using a mixed image The low-frequency subband, the unreplaced high-frequency subband, and the perturbed high-frequency subband are reconstructed using inverse discrete wavelet transform to obtain the enhanced image. ; will enhance the image With the second mixed label Pairing is used to form enhanced training samples for training the detection model, and the trained detection model is used to detect defects in the side guard plate of the hydraulic support.

2. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 1, characterized in that, In S1, an image set of hydraulic support side guard plates is acquired by deploying industrial cameras on the hydraulic support side guard plate bending production line; an instance segmentation model is used to pre-annotate the object mask and classification labels, and manual verification and confirmation are performed using annotation software.

3. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 2, characterized in that, In S1, three non-repeating random numbers are generated by computer as indices to extract three images without replacement from the hydraulic support side plate image set, and the corresponding object mask and classification label are obtained.

4. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 1, characterized in that, The process of pasting the target image to a random location in image B includes the following steps: If the size of the cropped target image is larger than image B, then the target image is scaled. Calculate the effective area in image B that can be pasted, and randomly select a location within the effective area to ensure that the target image is completely pasted within the boundary of image B.

5. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 1, characterized in that, The weight generation network stitches together a blended image along the channel dimension. Using image C as input, an encoder-decoder network is employed, and a nonlinear hybrid weight matrix is ​​output based on the encoder-decoder process. Among them, the nonlinear mixed weight matrix The value of each element is normalized to the interval [0, 1].

6. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 5, characterized in that, Generate blended images The formula is: ; in This indicates element-wise multiplication, where 1 represents AND. A matrix of all ones with the same shape.

7. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 1, characterized in that, Category Tags , and One-hot encoded vectors, mixed labels and mixed tags A vector representing the probability distribution of each category.

8. The intelligent detection method for bending of the side guard plate of the hydraulic support according to any one of claims 1-7, characterized in that, The multi-scale discrete wavelet transform is decomposed into N levels using the Daubechies wavelet basis or the Symlet wavelet basis, where N is an integer between 2 and 5.

9. The intelligent detection method for bending the side guard plate of the hydraulic support according to claim 8, characterized in that, When randomly selecting one of the high-frequency subbands, it is randomly selected from all N×3 high-frequency subbands according to a uniform distribution.

10. The intelligent detection method for bending of the side guard plate of the hydraulic support according to any one of claims 1-7, characterized in that, During the permutation operation, the selected high-frequency subband is divided into multiple non-overlapping sub-blocks of preset size, and all sub-blocks are randomly rearranged to generate perturbed high-frequency subbands.

Images