Chain shaft diamond feature detection method, program product, storage medium and equipment

Abstract

The invention discloses a chain shaft diamond feature detection method, a program product, a storage medium and equipment. The method comprises the steps of collecting a chain shaft image, marking an area where a chain shaft end circle is located in the image, and generating a data set by taking the area as a sample; the data set is input into a Faster R-CNN network for training, and a training model is obtained; and obtaining a to-be-detected chain axis image, and inputting the to-be-detected chain axis image into the training model for detection and positioning. According to the method, the deep learning algorithm and the convolutional neural network are adopted, efficient and accurate detection and positioning of the chain shaft can be achieved under the complex background and different scales, the reliability and efficiency of chain state detection are improved, and therefore technical guarantee is provided for maintenance and safe operation of the chain.

Description

Technical Field

[0001] The present application relates to the field of computer vision technology, and in particular to a chain shaft diamond feature detection method, program product, storage medium and device. Background Art

[0002] In industrial production, chains may wear or break during transportation, so regular inspection of their status is essential. Due to the complex on-site environment, chain axis imaging is often affected. Traditional methods such as mathematical model-based detection have limited effects under complex backgrounds, and existing diamond detection and 3D vision methods are not ideal for chain axis applications; therefore, how to stably and accurately position the chain axis is an urgent problem to be solved in chain spacing detection.

[0003] Prior art 1, Chinese patent CN202211502035.6, discloses an ellipse detection method based on the KMPE loss function, including: obtaining point set data P, and preprocessing the point set data P using the 3σ criterion to obtain the preprocessed point set data P1; establishing an objective function f1 for ellipse detection, and iteratively solving the optimal solution of the objective function f1 using the SHADE algorithm; after determining a rough estimate of the parameters of the ellipse to be detected according to the optimal solution of the objective function f1, the point set data P1 is divided into an inner point set P2 and an outer point set using the rough estimate, and the objective function f2 is further established according to the inner point set P2, and the optimal solution of the objective function f2 is iteratively solved using the SHADE algorithm; determining a precise estimate of the parameters of the ellipse to be detected according to the optimal solution of the objective function f2; determining the focal center coordinates, semi-minor axis and rotation angle of the ellipse to be detected according to the precise estimate of the parameters of the ellipse to be detected.

[0004] Prior art 2, Chinese patent CN202110427803.5, discloses a method for detecting the filling amount of a bullet primer based on 3D vision, including: first, preprocessing the acquired primer depth image by histogram equalization, binarization, dilation, hole removal and corrosion to obtain a high-quality image; then, performing circular domain detection on the preprocessed primer depth image by a fast ellipse detection method based on an arc adjacency matrix, and fitting the center Ci of each circular domain; third, calculating the Euclidean distance ρi between the center Ci of each ellipse and the center of mass of the ellipse to determine the target circular domain.

[0005] However, the first and second prior arts still do not solve the computational problem of detecting and identifying the images of shaft parts in the visual system. Summary of the invention

[0006] The main purpose of this application is to provide a chain shaft diamond feature detection method, program product, storage medium and device to solve the problems of image recognition difficulties and low detection accuracy in the existing technology in chain shaft surface detection. By adopting deep learning algorithms and convolutional neural networks, it is possible to achieve efficient and accurate detection and positioning of chain shafts under complex backgrounds and different scales, improve the reliability and efficiency of chain status detection, and thus provide technical guarantees for chain maintenance and safe operation. To achieve the above purpose, this application adopts the following technical solutions:

[0007] In a first aspect, the present application provides a chain shaft diamond feature detection method, comprising:

[0008] Collect chain shaft images, mark the area where the chain shaft end circle is located in the image, and use this as a sample to generate a data set;

[0009] Input the data set into the Faster R-CNN network for training to obtain a training model;

[0010] The chain shaft image to be detected is obtained and input into the training model for detection and positioning.

[0011] The training process of the training model specifically includes:

[0012] Perform feature extraction to obtain a feature map;

[0013] Add several convolutional layers to the feature map to select the region and obtain the candidate target region selected by the candidate box;

[0014] Mapping the candidate box from the original image of the chain axis image to a feature map;

[0015] The accurate position of the chain shaft end circle is obtained by diamond regression.

[0016] In at least one embodiment, the feature extraction uses ResNet-FPN as a feature extractor, constructs a multi-scale feature pyramid, and extracts features at each level of the feature pyramid.

[0017] In at least one embodiment, three convolutional layers are added to the feature map for region selection; the first convolutional layer is used to adjust the number of channels of the feature layer, the second convolutional layer is used to distinguish the foreground and background of the anchor box generated for each anchor point, and the third convolutional layer is used to generate a border regression weight of the anchor box for each anchor point for border regression.

[0018] In at least one embodiment, the training requires the use of positive samples, negative samples and neutral samples for training; the positive samples refer to candidate boxes with a high overlap rate with the true target box, the negative samples refer to candidate boxes that do not overlap with the true target box; the neutral samples refer to candidate boxes with a medium overlap rate with the true target box.

[0019] In at least one embodiment, the mapping is performed through the RoI Align module, which uses a bilinear interpolation method to obtain the image values ​​at the pixel points whose coordinates are floating point numbers, thereby converting the entire feature aggregation process into a continuous operation, reselecting a small feature map, and accurately aligning the extracted features with the input of the previous step.

[0020] In at least one embodiment, the process of the diamond regression is: adding diamond regression on the basis of the bounding box regression conventionally adopted by the Faster R-CNN network, wherein the diamond regression is provided with an irregular quadrilateral box, which is superimposed on the bounding box of the bounding box regression, and the five relative offset parameters of the diamond are directly regressed with the regression of the bounding box; the five relative offset parameters are the center coordinates (x0, y0) of the diamond, the lengths a and b of the diagonals, and the rotation angle θ.

[0021] In a second aspect, the present application provides a computer device, including a memory and a processor, wherein the memory stores computer program instructions; when the computer program instructions are read and executed by the processor, the above-mentioned chain shaft diamond feature detection method is executed.

[0022] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, wherein the computer program executes the above-mentioned chain shaft diamond feature detection method when executed by a computer.

[0023] In a fourth aspect, the present application provides a computer program product, including a computer program, which, when executed by a processor, implements the chain shaft diamond feature detection method described above in the claims.

[0024] Compared with the prior art, the present invention achieves at least the following beneficial effects:

[0025] This application uses deep learning algorithms and convolutional neural networks to achieve efficient and accurate detection and positioning of the diamond features of the chain shaft under complex backgrounds and at different scales, thereby improving the reliability and efficiency of chain status detection. Specifically, firstly, ResNet-FPN and RPN are used to extract features from the original image of the chain shaft, thereby improving the detection capability of targets of different scales. Secondly, RoI Align is used to ensure accurate alignment of feature maps, thereby improving detection accuracy. Thirdly, diamond regression can accurately identify and locate the edge of the diamond contour, thereby improving the accuracy and reliability of detection. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] One or more embodiments of the present application will now be described, by way of example only, with reference to the accompanying drawings, in which:

[0027] Figure 1 A flowchart of a chain shaft diamond feature detection method provided in an embodiment of the present application;

[0028] Figure 2 A flowchart of the model training method provided in the embodiment of the present application;

[0029] Figure 3 A schematic diagram of diamond regression provided in an embodiment of the present application;

[0030] Figure 4 A schematic diagram of the structure of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0031] The present application will be described in detail below with reference to the exemplary embodiments in the accompanying drawings. However, it should be understood that the present application can be implemented in a variety of different forms and should not be construed as being limited to the embodiments described herein. These embodiments are provided here to make the application content of the present application more complete and to fully convey the concept of the present application to those skilled in the art.

[0032] refer to Figure 1 The present application provides a chain shaft diamond feature detection method, comprising the following steps:

[0033] S1, collect the chain shaft image, mark the area where the chain shaft end circle is located in the image, and use this as a sample to generate a data set;

[0034] S2, inputting the data set into the Faster R-CNN network for training to obtain a training model;

[0035] S3. Obtain the chain shaft image to be detected and input it into the training model for detection and positioning.

[0036] refer to Figure 2 In at least one embodiment, the training process of the training model includes the following steps:

[0037] S21, extracting features to obtain a feature map;

[0038] S22, adding several convolutional layers to the feature map to perform region selection, and obtaining the candidate target region selected by the candidate box;

[0039] S23, mapping the candidate frame from the original image of the chain axis image to a feature map;

[0040] S24. Obtain the accurate position of the chain shaft end circle through diamond regression.

[0041] In step S21:

[0042] The feature extraction uses ResNet-FPN as a feature extractor, constructs a multi-scale feature pyramid, and extracts features at each level of the feature pyramid.

[0043] Since ResNet only uses the top-level features of the image for prediction, small target detection will have low accuracy. The solution to this problem is to construct a multi-scale feature pyramid FPN, extract features and predict at each level of the pyramid; by building an FPN structure on ResNet, the multi-scale feature mapping function is completed, the last layer output of each stage is used as the feature, and the features of each stage are fused to detect targets at different scales; FPN is the forward calculation of CNN from bottom to top, which is composed of multi-scale feature mapping, and the last layer output of each stage is used as the feature, and a pyramid is created for it. The deeper the stage, the stronger the feature. When detecting targets of different scales, this method can obtain feature maps with appropriate scales and strong semantic information.

[0044] In step S22:

[0045] Three convolutional layers are added to the feature map for region selection; the first convolutional layer is used to adjust the number of channels of the feature layer, the second convolutional layer is used to distinguish the foreground and background of the anchor frame generated for each anchor point, and the third convolutional layer is used to generate the border regression weight of the anchor frame for each anchor point for border regression.

[0046] The RPN module is a deep learning module for target detection, which is used to generate candidate target areas in images. The RPN module is a key component in the Faster R-CNN algorithm, which is used to extract areas in images that may contain targets for subsequent target classification and position regression. The RPN module uses a deep convolutional neural network, with a feature map as input and a set of candidate boxes as output. Specifically, the RPN module slides a small convolution window on the input feature map, and then performs binary classification on each window, that is, judging whether the window contains the target object, and also performs position regression on the windows containing the target. The areas obtained after sliding the window and position regression are the candidate target areas. The RPN module uses a sliding window method to quickly generate a large number of candidate areas, thereby improving the efficiency of detection.

[0047] The training requires positive samples, negative samples and neutral samples for training; the positive samples refer to candidate boxes with a high overlap rate with the real target box, the negative samples refer to candidate boxes with no overlap with the real target box; the neutral samples refer to candidate boxes with a medium overlap rate with the real target box. Through training with a large number of positive and negative samples, the RPN module can learn the ability to generate high-quality candidate boxes, thereby improving the accuracy of target detection.

[0048] In step S23:

[0049] The mapping is performed through the RoI Align module, which uses bilinear interpolation to obtain the image values ​​at the pixel points with floating-point coordinates, thereby converting the entire feature aggregation process into a continuous operation, reselecting a small feature map, and accurately aligning the extracted features with the input of the previous step.

[0050] The RoI Align module is a regional feature aggregation method. This layer is based on the extended prediction area and uses bilinear interpolation to obtain the image values ​​at the pixel points with floating-point coordinates, thereby converting the entire feature aggregation process into a continuous operation, reselecting a small feature map, and accurately aligning the extracted features with the FPN input. The extended features outside the prediction area have an adverse effect on the accuracy of the predicted diamond parameters).

[0051] The RoI Align module is used for feature alignment in target detection to solve the information loss problem in RoIPooling. In target detection, RoI refers to the candidate target area output by the RPN module; in RoIPooling, each RoI is divided into several sub-regions, and then a maximum value is taken from each sub-region as the representative feature of the sub-region; however, this method will cause information loss, because the maximum value may only come from a few pixels around a certain pixel, while ignoring the information of other pixels. The RoI Align module can solve this problem; its basic idea is to evenly divide the RoI area into several small areas, and then interpolate in each small area to obtain the features of the small area; then splice the features of all small areas together to obtain the feature representation of the RoI; this method can make full use of all pixel information in the RoI area and avoid the problem of information loss.

[0052] The implementation of the RoI Align module can be divided into the following steps:

[0053] R1, transform the coordinates of the RoI area and convert it to the coordinate system of the corresponding feature map;

[0054] R2, divide the RoI area evenly into several small areas, and the size of each small area is an integer pixel on the feature map;

[0055] R3, interpolate the pixels in each small area to obtain the feature representation of the small area;

[0056] R4, concatenate the feature representations of all small regions together to obtain the feature representation of the RoI.

[0057] In step S24:

[0058] The process of the diamond regression is as follows: a diamond regression is added on the basis of the bounding box regression conventionally adopted by the Faster R-CNN network, wherein the diamond regression is provided with an irregular quadrilateral box, which is superimposed on the bounding box of the bounding box regression, and the five relative offset parameters of the diamond are directly regressed with the regression of the bounding box; the five relative offset parameters are the center coordinates (x0, y0) of the diamond, the lengths a and b of the diagonals, and the rotation angle θ.

[0059] The diamond regression module is a module in target detection, which is used to predict the diamond parameters of the target and detect the position and size of the target. For the diamond, it is necessary to predict the diamond parameters of the target to more accurately describe the position and size of the target; the diamond parameters usually include the coordinates of the center point of the diamond, the length of the major axis, the length of the minor axis and the rotation angle; the main task of the diamond regression module is to predict these parameters from the image; the training data of the diamond regression module consists of some manually annotated diamond marks; during the training process, the model receives the image as input and outputs the diamond parameters of the target; the loss function needs to be defined during the training process, using the mean square error (MSE) or Smooth L1 loss; after the training is completed, the model can be used to predict the diamond parameters of the target in the test data; the use of the diamond regression module can improve the precision and accuracy of target detection and more accurately describe the position and size of the target.

[0060] When regressing the diamond, the Faster R-CNN basic model is used as the front-end network structure, and the five relative offset parameters of the diamond are regressed to achieve the regression of the chain axis contour; the relative offset parameters of the predicted bounding box normalize the five parameters of the diamond box, so that objects of different sizes in the image can contribute equally to the regression loss; the front-end network structure refers to a type of neural network structure in deep learning, which is used to extract and process features of the input image; parameter normalization refers to the standardization of the parameters in the model to make it easier to converge and optimize during the training process.

[0061] Specifically, normalization ensures that when the fitting area is close to the true value, all predicted offset values ​​are close to zero, making the training process stable and avoiding the situation of unbounded values; using this method of predicting the bounding box relative offset parameters to regress the bounding box allows the five parameters of the diamond box to be normalized, so that objects of different sizes in the image can contribute equally to the regression loss, making the loss unaffected by the size of the object; in addition, normalization ensures that when the fitting area is close to the true value, all predicted offset values ​​are close to zero, making the training process stable and avoiding the situation of unbounded values;

[0062] From a geometric point of view, there is a general rhombus in any direction that can be defined by its five parameters: the center coordinates (x0, y0), the lengths of the diagonal a, b, and the rotation angle θ (from the positive horizontal axis to the long axis of the rhombus). The standard form of the general rhombus is as follows: x′=x-x0, y′=y-y0, where the rotation angle θ∈(-π / 2,π / 2]. Through a complete regression training, the five rhombus parameters are regressed from the feature area to the true value of the rhombus; given a square feature area Q=(Qx, Qy, Ql) obtained by expanding the proposal P, the goal is to learn a set of relative offset parameters between the regression feature Q and the true value of the rhombus E. Here we use five parameters δx, δy, δa, δb, and δθ to parameterize the regression process.

[0063] The range of the true diamond angle is Eθ∈(-π / 2,π / 2], δ* is the regression target of the diamond, E′ is the predicted diamond calculated by δ, (δx,δy) specifies the scale-invariant translation from Q to E′, δa and δb specify the logarithmic space translation from Q to the diagonal of E′, δθ is the normalized diamond angle direction prediction of E′, when the predicted region (P→Q) is close to the true diamond E, the prediction offset δ is bounded, and the regression process is as follows Figure 3 shown.

[0064] According to an embodiment of the present application, the present application also provides a computer device, a readable storage medium, and a computer program product.

[0065] refer to Figure 4 In at least one embodiment of the computer device 100, the computer device 100 includes a memory 110 and a processor 120. The memory 110 stores computer program instructions. When the computer program instructions are read and executed by the processor 120, the method described in the above embodiment is executed. Further, in order to facilitate interaction, the computer device 100 may also include a display 130.

[0066] The computer device described in the embodiment of the present application is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, personal digital assistants, servers, mainframe computers, and other suitable computers. The computer device can also represent various forms of mobile devices, such as personal digital processing equipment, cellular phones, smart phones, wearable devices, and other similar computing devices. However, based on the demand for stable and safe processing of large-scale data, the computer device should be suitable for desktop computers, workbenches, servers, mainframe computers, etc. Further, the components shown in this article, their connections and relationships, and their functions are only examples, and are not intended to limit the implementation of the present disclosure described and / or required herein. Moreover, the memory, processor, and display included in the computer device are also common components in the art, and the types and models contained therein are conventional selections, and the application is not repeated here.

[0067] In at least one embodiment, a readable storage medium stores a computer program, and the computer program is executed by a computer to perform the method described in the above embodiments.

[0068] The computer-readable medium described in the embodiment of the present application may be a computer-readable signal medium or a computer-readable storage medium or any combination of the above two. The computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, a computer-readable storage medium may be any tangible medium containing or storing a program that may be used by an instruction execution system, device or device or used in combination with it. In the present application, a computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a computer-readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. Computer-readable signal media may also be any computer-readable medium other than computer-readable storage media, which may send, propagate or transmit a program for use by or in conjunction with an instruction execution system, apparatus or device. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wireless, wire, optical cable, RF, etc., or any suitable combination of the above.

[0069] In a computer program product of at least one embodiment, when the computer program product is executed by a processor, the method described in the foregoing embodiments is implemented.

[0070] The computer program product described in the embodiment of the present application is stored in a specific readable storage medium, and can be run by a computer device. The computer program product should not be limited to being able to run on a certain operating system, and these operating systems include: Unix-like systems (such as Linux), Microsoft Windows systems, Apple macOS, Chrome OS, Android, etc. The programming language used by the computer program product includes object-oriented programming languages ​​such as Java, Smalltalk, C++, and also includes conventional procedural programming languages-such as "C" language or similar programming languages, but should not be limited to a certain specific programming language. The program code can be executed completely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer, partially on the remote computer, or completely on the remote computer or server.

[0071] It should be understood that all the above embodiments are illustrative rather than restrictive, and various modifications or variations made by those skilled in the art to the specific embodiments described above based on the concept of the present application should all be within the protection scope of the present application.

Claims

1. A chain shaft diamond feature detection method, characterized in that: include: Collect chain shaft images, mark the area where the chain shaft end circle is located in the image, and use this as a sample to generate a data set; Input the data set into the Faster R-CNN network for training to obtain a training model; The chain shaft image to be detected is obtained and input into the training model for detection and positioning.

2. The method according to claim 1, characterized in that The training includes: Perform feature extraction to obtain a feature map; Add several convolutional layers to the feature map to select the region and obtain the candidate target region selected by the candidate box; Mapping the candidate box from the original image of the chain axis image to a feature map; The accurate position of the chain shaft end circle is obtained by diamond regression.

3. The method according to claim 2, characterized in that: The feature extraction uses ResNet-FPN as a feature extractor, constructs a multi-scale feature pyramid, and extracts features at each level of the feature pyramid.

4. The method according to claim 2, characterized in that: Three convolutional layers are added to the feature map for region selection; the first convolutional layer is used to adjust the number of channels of the feature layer, the second convolutional layer is used to distinguish the foreground and background of the anchor frame generated for each anchor point, and the third convolutional layer is used to generate the border regression weight of the anchor frame for each anchor point for border regression.

5. The method according to claim 2, characterized in that: The training requires positive samples, negative samples and neutral samples for training; the positive samples refer to candidate boxes with a high overlap rate with the true target box, the negative samples refer to candidate boxes with no overlap with the true target box; the neutral samples refer to candidate boxes with a medium overlap rate with the true target box.

6. The method according to claim 2, characterized in that The mapping is performed through the RoIAlign module, which uses bilinear interpolation to obtain the image values ​​at the pixel points with floating-point coordinates, thereby converting the entire feature aggregation process into a continuous operation, reselecting a small feature map, and accurately aligning the extracted features with the input of the previous step.

7. The method according to claim 2, characterized in that The process of the diamond regression is as follows: a diamond regression is added on the basis of the bounding box regression conventionally adopted by the Faster R-CNN network, wherein the diamond regression is provided with an irregular quadrilateral box, which is superimposed on the bounding box of the bounding box regression, and the five relative offset parameters of the diamond are directly regressed with the regression of the bounding box; the five relative offset parameters are the center coordinates (x0, y0) of the diamond, the lengths a and b of the diagonals, and the rotation angle θ.

8. A computer device comprising a memory and a processor, wherein the memory stores computer program instructions, characterized in that: When the computer program instructions are read and executed by the processor, the method according to any one of claims 1 to 7 is executed.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a computer, the method according to any one of claims 1 to 7 is executed.

10. A computer program product, comprising a computer program, characterized in that: When the computer program is executed by a processor, the method according to any one of claims 1 to 7 is implemented.

Images