Threaded connection defect detection method and device, electronic equipment and storage medium

Abstract

The embodiment of the present application provides a method and device for detecting defects of threaded connectors, an electronic device and a storage medium, and the method for detecting defects of threaded connectors comprises: the electronic device performs pixel-by-pixel segmentation on a picture to be detected containing threaded connectors to obtain a segmentation feature map containing at least one subcomponent; performs defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map; and determines whether at least one subcomponent has a defect according to the dimension defined as a defect in the feature vector. The embodiment of the present application can locate the defect position of the threaded connector.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to a method and device for detecting defects of threaded connectors, an electronic device and a storage medium. Background Art

[0002] Threaded connectors such as bolts, nuts, screws, and studs are basic parts that tightly connect mechanical equipment. They are currently widely used in multiple manufacturing fields such as rail transportation, power equipment, automobile manufacturing, and construction industries. Due to the complex and changeable production process and the different operation and maintenance conditions of equipment, there is a certain proportion of defects in the threaded connectors actually produced.

[0003] However, the current solution can only determine whether there is a defect but cannot locate the defect. Summary of the invention

[0004] Embodiments of the present application provide a method and device for detecting defects in a threaded connection, an electronic device, and a storage medium, which can locate the defect position of the threaded connection.

[0005] A first aspect of an embodiment of the present application provides a defect detection method for a threaded connection, comprising:

[0006] Segmenting the image to be inspected containing the threaded connector pixel by pixel to obtain a segmentation feature map containing at least one subcomponent;

[0007] Performing defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map;

[0008] It is determined whether the at least one subcomponent has a defect based on the dimension defined as a defect in the feature vector.

[0009] Optionally, the pixel-by-pixel segmentation of the image to be detected containing the threaded connector to obtain a segmentation feature map containing at least one subcomponent includes:

[0010] The trained segmentation network is used to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map containing at least one sub-component.

[0011] Optionally, the performing defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map includes:

[0012] The trained defect recognition network is used to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0013] Optionally, the pixel-by-pixel segmentation of the image to be detected containing the threaded connector using the trained segmentation network to obtain a segmentation feature map containing at least one subcomponent includes:

[0014] Performing feature extraction on the image to be detected containing the threaded connector to obtain bottom-level features and high-level features of the image to be detected;

[0015] The bottom-level features and the high-level features are integrated to obtain a feature map of the image to be detected;

[0016] The feature map of the image to be detected is classified pixel by pixel, the category of each pixel in the image to be detected is predicted, and a background segmentation feature map of the image to be detected and a segmentation feature map containing at least one subcomponent are obtained.

[0017] Optionally, the extracting features of the image to be detected containing the threaded connector to obtain bottom-level features and high-level features of the image to be detected includes:

[0018] The feature extraction network is used to extract features of the image to be detected containing the threaded connector, and the bottom-level features and high-level features of the image to be detected are obtained.

[0019] Optionally, the fusing the bottom-level features and the high-level features to obtain a feature map of the image to be detected includes:

[0020] The underlying features and the high-level features are fused using a feature fusion network to obtain a feature map of the image to be detected.

[0021] Optionally, the pixel-by-pixel classification of the feature map of the image to be detected, predicting the category of each pixel in the image to be detected, and obtaining the background segmentation feature map of the image to be detected and the segmentation feature map including at least one subcomponent includes:

[0022] The feature map of the image to be detected is classified pixel by pixel according to the feature category label information obtained by supervised training, and the category of each pixel in the image to be detected is predicted to obtain a background segmentation feature map of the image to be detected and a segmentation feature map containing at least one subcomponent, wherein the feature category label information includes a background category label and category labels of each subcomponent of the threaded connector.

[0023] Optionally, determining whether the at least one subcomponent has a defect according to a dimension defined as a defect in the feature vector includes:

[0024] The dimensions defined as defects in the feature vector are added to obtain a defect score of the at least one subcomponent. If the defect score is greater than a target threshold, it is determined that the at least one subcomponent has a defect.

[0025] Optionally, the at least one subcomponent includes at least one of a bolt head, a stud, and a nut.

[0026] Optionally, the method further includes:

[0027] Determine whether the image to be detected meets the image input requirements of the segmentation network;

[0028] If the image to be detected meets the image input requirement of the segmentation network, the step of using the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector is performed.

[0029] Optionally, the method further includes:

[0030] If the image to be detected does not meet the image input requirement of the segmentation network, the image to be detected is preprocessed to obtain the image to be detected containing the threaded connector.

[0031] Optionally, the preprocessing of the image to be detected to obtain the image to be detected containing the threaded connector includes:

[0032] The size of the image to be detected is adjusted to a specific size required by the image size of the segmentation network, and the variance and mean of the pre-trained network are used to normalize the image to be detected after the size adjustment to obtain the image to be detected containing the threaded connection.

[0033] A second aspect of an embodiment of the present application provides a defect detection device for a threaded connection, comprising:

[0034] A segmentation unit, used to segment the image to be detected containing the threaded connector pixel by pixel using a trained segmentation network to obtain a segmentation feature map containing at least one subcomponent;

[0035] The defect recognition unit is used to use the trained defect recognition network to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map, and determine whether the at least one sub-component has a defect based on the dimension defined as a defect in the feature vector.

[0036] A third aspect of an embodiment of the present application provides an electronic device, comprising a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the step instructions in the first aspect of the embodiment of the present application.

[0037] The fourth aspect of the embodiments of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, wherein the computer program enables a computer to execute part or all of the steps described in the first aspect of the embodiments of the present application.

[0038] A fifth aspect of the embodiments of the present application provides a computer program product, wherein the computer program product includes a non-transitory computer-readable storage medium storing a computer program, and the computer program is operable to cause a computer to execute some or all of the steps described in the first aspect of the embodiments of the present application. The computer program product may be a software installation package.

[0039] In an embodiment of the present application, a trained segmentation network is used to perform pixel-by-pixel segmentation on a picture to be inspected containing a threaded connection, so as to obtain a segmentation feature map of at least one subcomponent containing the threaded connection; a trained defect recognition network is used to perform defect recognition on the segmentation feature map, so as to obtain a feature vector corresponding to the segmentation feature map; and it is determined whether the first part has a defect based on the dimension defined as a defect in the feature vector.

[0040] The defect detection method for threaded connectors in the embodiment of the present application can perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector, segment the various subcomponents of the threaded connector in the image to be detected, obtain a segmentation feature map of the various subcomponents of the threaded connector, perform defect identification on the segmentation feature map of each subcomponent, identify whether each subcomponent of the threaded connector has defects, locate the subcomponent of the threaded connector where the defect is located, thereby locating the defect position of the threaded connector, and can narrow the range of the defect, making it easier to quickly find the defect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0042] Figure 1 It is a schematic flow chart of a defect detection method for a threaded connection provided in an embodiment of the present application;

[0043] Figure 2 It is a schematic diagram of pixel-by-pixel segmentation of a picture to be inspected containing a threaded connector provided in an embodiment of the present application;

[0044] Figure 3It is a flow chart of another defect detection method for threaded connection provided in an embodiment of the present application;

[0045] Figure 4 It is a structural diagram of a segmentation network and a defect recognition network provided in an embodiment of the present application;

[0046] Figure 5 A schematic structural diagram of a defect detection device for a threaded connection provided in an embodiment of the present application;

[0047] Figure 6 It is a structural schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0048] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0049] The terms "first", "second", etc. in the specification and claims of this application and the above-mentioned drawings are used to distinguish different objects, rather than to describe a specific order. In addition, the terms "including" and "having" and any variations thereof are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices.

[0050] Reference to "embodiments" in this application means that a particular feature, structure, or characteristic described in conjunction with the embodiments may be included in at least one embodiment of the present application. The appearance of the phrase in various locations in the specification does not necessarily refer to the same embodiment, nor is it an independent or alternative embodiment that is mutually exclusive with other embodiments. It is explicitly and implicitly understood by those skilled in the art that the embodiments described in this application may be combined with other embodiments.

[0051] The electronic devices involved in the embodiments of the present application may include various handheld devices with computing capabilities, vehicle-mounted devices, wearable devices, computing devices or other processing devices connected to a wireless modem, as well as various forms of user equipment (UE), mobile stations (MS), electronic devices (terminal devices), etc. For the convenience of description, the above-mentioned devices are collectively referred to as electronic devices.

[0052] See also Figure 1 , Figure 1 Schematic diagram of a defect detection method for a threaded connection provided in an embodiment of the present application. Figure 1 As shown, the defect detection method of the threaded connection may include the following steps.

[0053] 101. An electronic device performs pixel-by-pixel segmentation on a to-be-detected image containing a threaded connector to obtain a segmentation feature map containing at least one subcomponent.

[0054] In an embodiment of the present application, the electronic device may use a neural network model to perform pixel-by-pixel segmentation, or may use pixel binarization processing to perform pixel-by-pixel segmentation.

[0055] Optionally, step 101 may include the following steps:

[0056] The electronic device uses the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector to obtain a segmentation feature map containing at least one subcomponent.

[0057] The neural network model may be a trained segmentation network. The trained segmentation network may include a trained semantic segmentation network, such as a trained fully convolutional network (FCN), SegNet, U-Net, mask-region convolutional neural network (Mask-RCNN) and other deep learning models.

[0058] During the training process of the segmentation network, the segmentation network can be trained through a large number of supervised samples (pictures of defective threaded connections and pictures of non-defective threaded connections), each pixel is classified, and the classification loss is calculated pixel by pixel (softmax). When the training loss is less than a set value, the segmentation network is determined to be a trained segmentation network.

[0059] A threaded connector may include at least one subcomponent, which may include any one of a bolt, a stud, a screw, and a nut. A subcomponent may also include a portion of a bolt, a portion of a stud, a portion of a screw, or a portion of a nut. Bolts may include hexagonal head bolts, square head bolts, square neck bolts, etc. Studs may include studs, equal length studs, welding studs for manual welding, welding studs for motorized arc welding, welding studs for energy storage welding, etc. Screws may include hexagon socket screws, set screws, self-tapping screws, etc. Nuts may include hexagonal nuts, round nuts, etc. The above-mentioned threaded connectors may be applied to high-speed rail contact networks, power grids, automobile production lines, and other places where threaded connectors are needed.

[0060] In one embodiment, the trained segmentation network can segment the threaded connector in the image to be detected according to subcomponents to obtain segmentation feature maps containing each subcomponent. Each segmentation feature map can contain only one subcomponent or two or more subcomponents at the same time.

[0061] Optionally, the at least one subcomponent includes at least one of a bolt head, a stud, and a nut.

[0062] The threaded connector may include a bolt, a stud and a nut, and the bolt may include a bolt head and a stud.

[0063] Optionally, the at least one subcomponent includes at least one of a bolt head, a threaded screw, an unthreaded screw, a regular hexagonal prism portion of a nut, a cylindrical portion of a nut, a head portion of a screw, and a screw portion of a screw.

[0064] Threaded connectors can be divided into studs, bolts, screws, and nuts according to the common categories currently available. Bolts are generally used in conjunction with nuts. One end of a stud can be embedded in a component, and the other end is connected to another part. It is generally used when the connected part cannot or is inconvenient to install a headed bolt. Screws are common fasteners that can be used alone and embedded in structural parts.

[0065] Furthermore, the bolt can be divided into the head portion of the bolt (i.e., the bolt head) and the (threaded) screw portion of the bolt (i.e., the stud). The stud can also be divided into a threaded screw portion and a non-threaded screw portion. The nut is divided into a regular hexagonal prism portion and a cylindrical portion. The screw can be divided into the head portion of the screw and the (threaded) screw portion of the screw.

[0066] If the threaded connection in the image to be inspected includes at least one of a bolt, a stud, a screw, and a nut, the threaded connection in the image to be inspected can be divided into at least two sub-components according to the bolt head part and the screw rod part included in the bolt, the threaded screw rod part and the unthreaded screw rod part included in the stud, the screw head part and the screw rod part included in the screw, and the regular hexagonal prism part and the cylindrical part included in the nut.

[0067] If the threaded connection in the detection image includes a bolt, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map of the head part of the bolt and a segmentation feature map of the screw part of the bolt.

[0068] If the threaded connection in the detection image includes a stud, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map of the threaded screw portion of the stud and a segmentation feature map of the unthreaded screw portion of the stud.

[0069] If the threaded connection in the inspection image includes a nut, the electronic device uses the trained segmentation network to perform pixel-by-pixel segmentation on the inspection image containing the threaded connection to obtain a segmentation feature map of the regular hexagonal prism part of the nut and a segmentation feature map of the cylindrical part of the nut.

[0070] If the threaded connections in the inspection image include bolts and studs, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the inspection image containing the threaded connections to obtain a segmentation feature map of the head part of the bolt, a segmentation feature map of the screw part of the bolt, a segmentation feature map of the threaded screw part of the stud, and a segmentation feature map of the unthreaded screw part of the stud.

[0071] If the threaded connection in the detection image includes bolts and nuts, the electronic device uses the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection, and obtains a segmentation feature map of the head part of the bolt, a segmentation feature map of the screw part of the bolt, a segmentation feature map of the regular hexagonal prism part of the nut, and a segmentation feature map of the cylindrical part of the nut.

[0072] If the threaded connection in the detection image includes a stud and a nut, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection, and obtains a segmentation feature map of the threaded screw part of the stud, a segmentation feature map of the stud and the unthreaded screw part, a segmentation feature map of the regular hexagonal prism part of the nut, and a segmentation feature map of the cylindrical part of the nut.

[0073] If the threaded connections in the detection image include bolts, studs and nuts, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connections, and obtains a segmentation feature map of the head part of the bolt, a segmentation feature map of the screw part of the bolt, a segmentation feature map of the threaded screw part of the stud, a segmentation feature map of the unthreaded screw part of the stud, a segmentation feature map of the regular hexagonal prism part of the nut, and a segmentation feature map of the cylindrical part of the nut.

[0074] In addition to threaded connectors, the image to be detected also contains other backgrounds. Pixel-by-pixel segmentation can not only segment the various sub-components of the threaded connector to obtain the segmentation feature maps of the various sub-components of the threaded connector, but also segment the background of the image to be detected (that is, the part of the image to be detected that does not belong to the threaded connector) to obtain the segmentation feature map of the background part.

[0075] The various subcomponents of the threaded connection can be clearly distinguished from the segmentation feature map. For example, all pixels of the segmentation feature map of the same part can be marked with the same color, and the colors of the pixels of the segmentation feature maps of any two different parts are different.

[0076] Pixel-by-pixel segmentation is to classify each pixel in the image to be detected into a category, segment the pixels of the same category, and form a segmentation feature map corresponding to the category. Figure 2 , Figure 2 1 is a schematic diagram of pixel-by-pixel segmentation of a to-be-detected image containing a threaded connector provided in an embodiment of the present application. Figure 2 As shown, after the image to be detected containing the threaded connection is segmented pixel by pixel, a segmentation feature map is obtained, and the threaded connection in the segmentation feature map is segmented into the screw part of the bolt, the cylindrical part of the nut and the regular hexagonal prism part of the nut. Figure 2 The segmentation feature maps include the segmentation feature maps of the screw part of the bolt, the segmentation feature maps of the cylindrical part of the nut and the segmentation feature maps of the regular hexagonal prism part of the nut. The segmentation feature maps of these three parts have different color depths (the color of the screw part of the bolt is relatively light, the color of the regular hexagonal prism part of the nut is relatively moderate, and the color of the cylindrical part of the nut is relatively darkest).

[0077] Optionally, the electronic device uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector to obtain a segmentation feature map of the first part containing at least one subcomponent, which may specifically include the following steps:

[0078] The electronic device extracts features of the image to be detected containing the threaded connector to obtain bottom-level features and high-level features of the image to be detected;

[0079] The electronic device fuses the bottom-level features and the high-level features to obtain a feature map of the image to be detected;

[0080] The electronic device classifies the feature map of the image to be detected pixel by pixel, predicts the category of each pixel in the image to be detected, and obtains the background segmentation feature map of the image to be detected and the segmentation feature map including at least one subcomponent.

[0081] The electronic device extracts features of the image to be detected containing the threaded connector to obtain the bottom-level features and high-level features of the image to be detected, specifically:

[0082] The electronic device uses a feature extraction network to extract features from a to-be-detected image containing a threaded connector, and obtains bottom-level features and high-level features of the to-be-detected image.

[0083] The electronic device fuses the underlying features and the high-level features to obtain a feature map of the image to be detected, specifically:

[0084] The electronic device uses a feature fusion network to fuse the underlying features and the high-level features to obtain a feature map of the image to be detected.

[0085] The electronic device classifies the feature map of the image to be detected pixel by pixel, predicts the category of each pixel in the image to be detected, and obtains the background segmentation feature map of the image to be detected and the segmentation feature map including at least one subcomponent, specifically:

[0086] The electronic device classifies the feature map of the image to be detected pixel by pixel according to the feature category label information obtained by supervised training, predicts the category of each pixel in the image to be detected, and obtains the background segmentation feature map of the image to be detected and the segmentation feature map containing at least one subcomponent, wherein the feature category label information includes the background category label and the category labels of each subcomponent of the threaded connector.

[0087] In the embodiment of the present application, the segmentation network may include a feature extraction network and a feature fusion network. The feature extraction network may include a residual network, and the feature fusion network may include a feature pyramid network (FPN).

[0088] Among them, the residual network is used to extract features from the image to be detected, and obtain the underlying features (also called low-level features) and high-level features of the image to be detected. The residual network can solve the problems of gradient disappearance, explosion and network degradation caused by the increase in the number of network layers. The feature pyramid network fuses the underlying features and high-level features extracted by the residual network, so that the entire segmentation network integrates features of different resolutions and different semantic strengths, and the segmentation features are richer, which is conducive to the classification of the subsequent defect recognition network. During the segmentation process, the segmentation network can learn the overall structure of the threaded connector and the relationship between each sub-component. At the same time, during the training process of the segmentation network, the bottom layer of the segmentation network can learn richer underlying features through the pixel-by-pixel supervision information of the entire image. Using the feature pyramid network, the underlying features and high-level features of the image to be detected can be fused, which can not only learn the overall semantic information of the high-level image, but also learn more detailed and more underlying features. The segmentation features are richer, the segmentation effect is better, and it is also conducive to the classification of the subsequent defect recognition network.

[0089] The residual network is a deep convolutional neural network. The different depths of the residual network correspond to the extraction of feature information at different levels, including low-level features, mid-level features, and high-level features. The more convolutional layers the residual network has, the more information at different levels can be extracted, and the more combinations of information between different levels can be obtained. For convolutional neural networks, different depths correspond to different levels of semantic features. The shallow network has a high resolution and learns more detailed features, while the deep network has a low resolution and learns more semantic features.

[0090] The low-level features of the image to be detected include: the contour, edge, color, texture and shape features of the image to be detected; the high-level features of the image to be detected include the overall information of the image to be detected. Generally speaking, the low-level features have higher resolution and contain more position and detail information, but because they have passed fewer convolutions, they have lower semantics and more noise. High-level features have stronger semantic information, but their resolution is very low and their perception of details is poor. Efficiently integrating the two and taking advantage of their strengths can improve the segmentation ability of the segmentation network.

[0091] The electronic device can use the residual network to extract features from the image to be detected containing the threaded connector, and the image to be detected outputs different features in different convolutional layers of the residual network. For example, the bottom-level features are output in the lower convolutional layer, and the high-level features are output in the higher convolutional layer.

[0092] The electronic device uses a feature pyramid network to fuse the bottom-level features and high-level features of the image to be detected. Each pixel in the feature map of the image to be detected contains the fused features, thereby obtaining a feature map of the image to be detected. The electronic device classifies the feature map of the image to be detected pixel by pixel based on the feature category label information obtained by the segmentation network during the supervised training process, and predicts the category of each pixel in the image to be detected, thereby obtaining a background segmentation feature map of the image to be detected and a segmentation feature map of at least one subcomponent contained in the threaded connector. The segmentation network can be trained using a large number of supervised training sample images containing threaded connectors, and the segmentation network can learn pixel-level features during the training process. Feature category label information may include: background category labels, category labels of various sub-components of threaded connectors (for example, category labels of various sub-components of threaded connectors may include: category label of the head part of the bolt, category label of the screw part of the bolt, category label of the threaded screw part of the stud, category label of the unthreaded screw part of the stud, category label of the regular hexagonal prism part of the nut and category label of the cylindrical part of the nut, category label of the head part of the screw and category label of the screw part of the screw).

[0093] 102. The electronic device performs defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0094] In an embodiment of the present application, the electronic device can use a neural network model or a traditional image segmentation algorithm to perform defect recognition on a segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0095] Optionally, step 102 may include the following steps:

[0096] The electronic device uses the trained defect recognition network to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0097] In the embodiment of the present application, the defect recognition network may include decision networks such as residual neural network (ResNet) and visual geometry group network (VGGNet). Residual neural network can also be referred to as residual network. VGGNet uses all 3x3 small convolution kernels and 2x2 pooling kernels. By continuously deepening the network to improve performance, VGGNet has good generalization ability and performs well on different image data sets.

[0098] Step 102 can use the segmentation feature map containing at least one subcomponent obtained in step 101 to further classify the segmentation feature map containing at least one subcomponent to obtain a feature vector. The defect recognition network and the segmentation network share a feature extraction network (e.g., a residual network). During the training process, the defect recognition network does not update the underlying parameters of the defect recognition network because the segmentation network has already learned the underlying detailed features. The defect recognition network only updates the high-level parameters to learn more advanced semantic features. After the training is completed, the feature vector output by the defect recognition network can be directly obtained, and the result map of the segmentation output by the segmentation network can be obtained at the same time.

[0099] 103 , the electronic device determines whether the first part has a defect according to the dimension defined as a defect in the feature vector.

[0100] In the embodiment of the present application, the feature vector may include multiple dimensions, and the values ​​of some dimensions in the multiple dimensions represent the categories of defects, and the values ​​of other dimensions represent the categories without defects. For example, if the feature vector P1 = (a1, a2, a3, a4, a5), if the second dimension, the third dimension and the fourth dimension of the vector P1 represent the categories of defects respectively (for example, the second dimension represents crack defects, the third dimension represents hole defects, and the fourth dimension represents rust defects), and the other dimensions represent the categories without defects, then it can be determined whether at least one subcomponent of the threaded connector has defects according to the values ​​of the second dimension, the third dimension and the fourth dimension of the feature vector P1. Among them, the value of each dimension in P1 can be between 0 and 1 (0 represents no defects, and the larger the value, the greater the degree of defect). For example, at least one subcomponent of the threaded connector is the head of the bolt. If a2, a3, and a4 are all 0, it means that the head of the bolt has no defects. If one of a2, a3, and a4 is not 0, or the number obtained by a2, a3, and a4 according to a certain operation rule is greater than a certain threshold, it indicates that the head of the bolt has defects. For example, if the sum of a2, a3, and a4 multiplied by the corresponding weighted coefficients is greater than a certain threshold, it indicates that there is a defect in the head of the bolt. If the sum of a2, a3, and a4 multiplied by the corresponding weighted coefficients is less than a certain threshold, it indicates that there is no defect in the head of the bolt.

[0101] If a subcomponent (for example, the first subcomponent) of the threaded connector has a defect, based on the pixel-by-pixel segmentation result, it can be located that the first subcomponent of the threaded connector in the image to be inspected containing the threaded connector has a defect, thereby locating the defect position.

[0102] Optionally, step 103 may specifically include the following steps:

[0103] (11) the electronic device adds the dimensions defined as defects in the feature vector to obtain a defect score for at least one subcomponent;

[0104] (12) if the defect score is greater than the target threshold, the electronic device determines that at least one subcomponent is defective;

[0105] (13) if the defect score is less than the target threshold, the electronic device determines that at least one subcomponent is not defective;

[0106] (14) If the defect score is equal to the target threshold, the electronic device determines whether at least one subcomponent is defective or not defective.

[0107] In the embodiment of the present application, for example, if the feature vector P1 = (a1, a2, a3, a4, a5), if the second dimension, the third dimension and the fourth dimension of the vector P1 represent the categories of defects respectively, then a2, a3 and a4 can be added to obtain the defect score of at least one subcomponent of the threaded connector, and if the defect score is greater than the target threshold, it is determined that at least one subcomponent has a defect. For example, if the defect score of the stud part of the threaded connector is greater than the target threshold, it is determined that the stud part of the threaded connector has a defect.

[0108] The target threshold may be preset and stored in a memory (eg, a non-volatile memory) of the electronic device.

[0109] Among them, step 103 can be a part of a defect recognition network, and the defect score of the threaded connection can be directly output through the defect recognition network without complex logical judgment.

[0110] The embodiment of the present application can obtain a defect score of at least one sub-component by adding the dimensions defined as defects in the feature vector. The defect score can accurately reflect the degree of defects of at least one sub-component and can comprehensively consider various possible defects of at least one sub-component, thereby accurately predicting whether at least one sub-component has defects.

[0111] In the embodiment of the present application, for the feature vector output by the defect recognition network, the scores of the dimensions defined as defects can be added to obtain the final defect score. If the defect score is greater than a given threshold, it is considered that the given image has a defect, and if it is less than the threshold, it is considered that the given image is a fault-free image.

[0112] Optionally, step 103 may specifically include the following steps:

[0113] (21) the electronic device calculates the probability that the at least one subcomponent has a defect based on the dimension defined as a defect in the feature vector;

[0114] (22) if the probability that the at least one subcomponent is defective is greater than a set threshold, the electronic device determines that the at least one subcomponent is defective;

[0115] (23) if the probability that the at least one subcomponent is defective is less than a set threshold, the electronic device determines that the at least one subcomponent is not defective;

[0116] (24) If the probability that the at least one subcomponent is defective is equal to a set threshold, the electronic device determines whether the at least one subcomponent is defective or not.

[0117] In the embodiment of the present application, the electronic device calculates the probability that the at least one subcomponent has a defect according to the dimension defined as a defect in the feature vector, specifically:

[0118] The electronic device adds the dimensions defined as defects in the feature vector to obtain a defect score of at least one subcomponent, and determines the probability that the at least one subcomponent has a defect according to the corresponding relationship between the defect score and the defect probability.

[0119] The set threshold value may be set in advance and stored in a memory (eg, a non-volatile memory) of the electronic device.

[0120] The embodiment of the present application can calculate the probability that the at least one sub-component has a defect based on the dimension defined as the defect in the feature vector. The probability that the at least one sub-component has a defect can accurately reflect whether the at least one sub-component has a defect, and can comprehensively consider various possible defects of the at least one sub-component, thereby accurately predicting whether the at least one sub-component has a defect.

[0121] The embodiment of the present application adopts a deep learning model, which is more accurate than traditional methods in complex scenes and can better adapt to complex situations such as different lighting, different angles, and different resolutions. The solution based on the segmentation network can output the pixel-by-pixel segmentation results of the image of the threaded connection compared to the simple classification solution, which is more interpretable and convenient for continuous analysis and solving samples with difficult defects.

[0122] The segmentation network can output feature maps of each sub-component of a threaded connection. It can not only learn high-level overall semantic information of the image, but also learn more detailed and low-level features. The segmentation network can segment out each sub-component of a threaded connection, and has good generalization performance. Compared with traditional methods, the deep learning framework is used to identify and detect threaded connections, and there is no need to manually design complex features. The overall framework is well interpretable. Given a picture of a threaded connection, the defect probability can be output, and at the same time, a pixel-by-pixel segmentation map of the threaded connection can be output to locate the location of the defect.

[0123] The segmentation network and defect recognition network have good scalability. The above networks can be updated in time and can quickly respond to new defect categories for detection. If there are new defects, they can be added to the category and then added to the training, and the score or probability of the defect corresponding to this category can be output.

[0124] In an embodiment of the present application, the image to be inspected containing the threaded connector can be segmented pixel by pixel, and the various sub-components of the threaded connector in the image to be inspected can be segmented to obtain a segmentation feature map of each sub-component of the threaded connector. Defect identification can be performed on the segmentation feature map of each sub-component to identify whether each sub-component of the threaded connector has defects. The sub-component of the threaded connector where the defect is located can be located, thereby locating the defective position of the threaded connector.

[0125] Figure 1 The method shown can be applied to the following scenarios: In the scenario where defect detection of threaded connections is required, the method can be used Figure 1 The method shown is used to determine whether the threaded connection is a defective image. Figure 1 The method shown in the figure outputs a pixel-by-pixel segmentation map of the threaded connection and locates the defect position of the threaded connection. Figure 1 The method shown helps improve the defect detection effect of threaded connections and is further adopted based on other methods. Figure 1 method for defect detection.

[0126] Figure 1 The method shown can be applied to industrial defect detection, such as inspection of high-speed rail contact network and power network.

[0127] See also Figure 3 , Figure 3 FIG. 1 is a flow chart of another method for detecting defects in threaded connectors provided in an embodiment of the present application. Figure 3 As shown, the defect detection method of the threaded connection may include the following steps.

[0128] 301, the electronic device determines whether the image to be detected meets the image input requirements of the segmentation network. If yes, execute steps 302 to 304; if no, execute step 305.

[0129] In the embodiment of the present application, the segmentation network has certain requirements on the size of the image, whether it is normalized, etc. If it does not meet the requirements, step 305 is executed to pre-process the image to be detected. If it meets the requirements, it can be directly input into the trained segmentation network for pixel-by-pixel segmentation. Among them, the trained segmentation network and the trained segmentation network have the same requirements for the input image. For example, in the training stage, if the trained segmentation network needs to normalize the input image for training, in the test stage, the trained segmentation network also needs to normalize the input image for testing.

[0130] 302, the electronic device uses the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector to obtain a segmentation feature map containing at least one subcomponent.

[0131] 303, the electronic device uses the trained defect recognition network to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0132] 304 , the electronic device determines whether at least one subcomponent has a defect according to the dimension defined as a defect in the feature vector.

[0133] The specific implementation of steps 302 to 305 can refer to the specific description of steps 101 to 103, which will not be repeated here.

[0134] Optionally, when at least one subcomponent is detected to be defective, the electronic device may give an alarm, for example, by sounding an alarm.

[0135] 305 , the electronic device pre-processes the image to be detected to obtain the image to be detected containing the threaded connector.

[0136] In the embodiment of the present application, preprocessing may include resizing the image to be detected, normalizing the image to be detected, etc.

[0137] For the image to be inspected that includes the threaded connector, the image can be input into the segmentation network for further processing. After executing step 305, steps 302 to 304 can be continued.

[0138] Optionally, step 305 may include the following steps:

[0139] (31) The electronic device adjusts the size of the image to be detected to a specific size required by the image size of the segmentation network;

[0140] (32) The electronic device uses the variance and mean of the pre-trained network to normalize the resized image to be detected, thereby obtaining the image to be detected including the threaded connector.

[0141] In the embodiment of the present application, the picture to be detected can be a picture containing threaded connectors taken directly, or it can be a picture containing threaded connectors taken from a specific scene. For the picture to be detected taken directly, the size of the picture to be detected is moderate, and the proportion of the threaded connectors in the picture to be detected is greater than the preset ratio threshold, then there is no need to crop the picture to be detected. Corresponding to the picture to be detected taken from a specific scene, the proportion of the threaded connectors in the picture to be detected is less than the preset ratio threshold, and the threaded connectors are relatively small, and it is necessary to first crop the threaded connectors from the large picture of the scene (picture to be detected). For the picture containing threaded connectors taken directly or the picture cropped out of the threaded connection, in order to meet the picture input requirements of the segmentation network, it is necessary to perform necessary image preprocessing operations, including adjusting the picture size to a specific scale, and using the variance and mean of the pre-trained network to normalize the picture to be detected.

[0142] Among them, the pre-trained network is pre-trained under a larger data set (not necessarily a data set of threaded connectors), and the mean and variance are the mean and variance of this larger data set. The mean and variance are the average and variance of some common features on the image. For example, assuming that the feature x1 ranges between 0-1, the range of x2 is between -1 and 1, and the range of x3 is between 1-2, these features are in a similar range, so they will perform well. The gradient descent method can more directly find the minimum value, and a larger step size can be used in the gradient descent method, which can help the learning algorithm of the segmentation network run faster. If some features are in very different value ranges, such as one feature x1 has a value between 1 and 1000, and another feature x2 has a value between -1 and 1, this is very unfavorable to the optimization algorithm of the segmentation network. At this time, the value of feature x1 of the image to be processed can be subtracted from the mean value of the feature x1 in the pre-trained network, the mean value of feature x1 of the image to be processed can be adjusted to 0, and the variance of feature x1 of the image to be processed can be adjusted to be the same as the mean value of feature x1 in the pre-trained network, thereby achieving normalization processing of feature x1. Similarly, a similar normalization processing is performed on feature x2.

[0143] In an embodiment of the present application, the image to be inspected containing the threaded connector can be preprocessed and then input into a segmentation network for pixel-by-pixel segmentation. The sub-components of the threaded connector in the image to be inspected are segmented to obtain a segmentation feature map of the sub-components of the threaded connector. The segmentation feature map of the sub-components is used to identify defects through a defect recognition network to identify whether the sub-components of the threaded connector have defects. The defective part of the threaded connector can be located, thereby locating the defective position of the threaded connector.

[0144] See also Figure 4 , Figure 4 Schematic diagram of a segmentation network and a defect recognition network provided by an embodiment of the present application. Figure 4 As shown, the segmentation network may include a residual network and a feature pyramid network. The defect recognition network includes a vector generation unit and a defect recognition unit. The residual network extracts features from a picture to be detected containing a threaded connection (the picture to be detected may be a preprocessed picture), and the feature pyramid network fuses the extracted underlying features and high-level features, predicts the category of each pixel in the picture to be detected, and obtains a background segmentation feature map of the picture to be detected and a segmentation feature map of a subcomponent of the threaded connection. The vector generation unit obtains a feature vector corresponding to the segmentation feature map of a subcomponent of the threaded connection based on the segmentation feature map of a subcomponent of the threaded connection and the features extracted by the residual network, and the defect recognition unit determines whether a subcomponent has a defect based on the dimension defined as a defect in the feature vector.

[0145] The above mainly introduces the scheme of the embodiment of the present application from the perspective of the execution process on the method side. It is understandable that, in order to realize the above functions, the electronic device includes a hardware structure and / or software module corresponding to the execution of each function. It should be easily appreciated by those skilled in the art that, in combination with the units and algorithm steps of each example described in the embodiments provided herein, the present application can be implemented in the form of hardware or a combination of hardware and computer software. Whether a function is executed in the form of hardware or computer software driving hardware depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present application.

[0146] The embodiment of the present application can divide the electronic device into functional units according to the above method example. For example, each functional unit can be divided according to each function, or two or more functions can be integrated into one processing unit. The above integrated unit can be implemented in the form of hardware or in the form of software functional units. It should be noted that the division of units in the embodiment of the present application is schematic and is only a logical function division. There may be other division methods in actual implementation.

[0147] In line with the above, see Figure 5 , Figure 5 A structural schematic diagram of a defect detection device for a threaded connection provided in an embodiment of the present application is provided. The defect detection device 500 for a threaded connection is applied to an electronic device. The defect detection device 500 for a threaded connection may include a segmentation unit 501 and a defect recognition unit 502, wherein:

[0148] The segmentation unit 501 is used to segment the image to be detected containing the threaded connector pixel by pixel to obtain a segmentation feature map containing at least one subcomponent;

[0149] The defect recognition unit 502 is used to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map, and to determine whether the at least one sub-component has a defect based on the dimension defined as a defect in the feature vector.

[0150] Optionally, the segmentation unit 501 performs pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map containing at least one sub-component, specifically: using a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map containing at least one sub-component.

[0151] Optionally, the defect recognition unit 502 performs defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map, specifically: using a trained defect recognition network to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

[0152] Optionally, the segmentation unit 501 uses a trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection to obtain a segmentation feature map containing at least one sub-component, specifically: feature extraction is performed on the image to be detected containing the threaded connection to obtain underlying features and high-level features of the image to be detected; the underlying features and the high-level features are merged to obtain a feature map of the image to be detected; the feature map of the image to be detected is classified pixel by pixel to predict the category of each pixel in the image to be detected to obtain a background segmentation feature map of the image to be detected and a segmentation feature map containing at least one sub-component.

[0153] Optionally, the segmentation unit 501 performs feature extraction on the image to be detected containing threaded connections to obtain underlying features and high-level features of the image to be detected, specifically: using a feature extraction network to perform feature extraction on the image to be detected containing threaded connections to obtain underlying features and high-level features of the image to be detected.

[0154] Optionally, the segmentation unit 501 fuses the underlying features and the high-level features to obtain a feature map of the image to be detected, specifically by using a feature fusion network to fuse the underlying features and the high-level features to obtain a feature map of the image to be detected.

[0155] Optionally, the segmentation unit 501 classifies the feature map of the image to be detected pixel by pixel, predicts the category of each pixel in the image to be detected, and obtains the background segmentation feature map of the image to be detected and the segmentation feature map containing at least one sub-component. Specifically, the feature map of the image to be detected is classified pixel by pixel according to the feature category label information obtained by supervised training, and the category of each pixel in the image to be detected is predicted to obtain the background segmentation feature map of the image to be detected and the segmentation feature map containing at least one sub-component, wherein the feature category label information includes the background category label and the category labels of each sub-component of the threaded connector.

[0156] Optionally, the defect identification unit 502 determines whether the at least one sub-component has a defect based on the dimensions defined as defects in the feature vector, specifically by adding the dimensions defined as defects in the feature vector to obtain a defect score of the at least one sub-component; if the defect score is greater than a target threshold, it is determined that the at least one sub-component has a defect.

[0157] Optionally, the defect identification unit 502 determines whether the at least one sub-component has a defect based on the dimension defined as a defect in the feature vector, specifically: the probability that the at least one sub-component has a defect is calculated based on the dimension defined as a defect in the feature vector; if the probability that the at least one sub-component has a defect is greater than a set threshold, it is determined that the at least one sub-component has a defect.

[0158] Optionally, the at least one subcomponent includes at least one of a bolt head, a stud, and a nut.

[0159] The threaded connector may include a bolt, a stud and a nut, and the bolt may include a bolt head and a stud.

[0160] Optionally, the at least one subcomponent includes at least one of a bolt head, a threaded screw, an unthreaded screw, a regular hexagonal prism portion of a nut, a cylindrical portion of a nut, a head portion of a screw, and a screw portion of a screw.

[0161] Optionally, the defect detection device 500 may further include a judgment unit 503, wherein:

[0162] A judging unit 503 is used to judge whether the image to be detected meets the image input requirement of the segmentation network;

[0163] The segmentation unit 501 is used to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connection by using a trained segmentation network when the judgment result of the judgment unit 503 is yes.

[0164] Optionally, the defect detection device 500 may further include a preprocessing unit 504, wherein:

[0165] The preprocessing unit 504 is used to preprocess the image to be detected to obtain the image to be detected containing the threaded connection part when the judgment result of the judgment unit 503 is no.

[0166] Optionally, the preprocessing unit 504 preprocesses the image to be detected to obtain the image to be detected containing the threaded connector, specifically:

[0167] The size of the image to be detected is adjusted to a specific size required by the image size of the segmentation network, and the variance and mean of the pre-trained network are used to normalize the image to be detected after the size adjustment to obtain the image to be detected containing the threaded connection.

[0168] Among them, the segmentation unit 501, the defect recognition unit 502, the judgment unit 503 and the preprocessing unit 504 in the embodiment of the present application can be processors in an electronic device.

[0169] In an embodiment of the present application, the image to be inspected containing the threaded connector can be segmented pixel by pixel, and the various sub-components of the threaded connector in the image to be inspected can be segmented to obtain a segmentation feature map of the various sub-components of the threaded connector. Defect identification can be performed on the segmentation feature map of each sub-component to identify whether there are defects in the various sub-components of the threaded connector. The sub-component of the threaded connector where the defect is located can be located, thereby locating the defective position of the threaded connector.

[0170] See also Figure 6 , Figure 6 is a schematic diagram of the structure of an electronic device provided in an embodiment of the present application, such as Figure 6 As shown, the electronic device 600 includes a processor 601 and a memory 602, and the processor 601 and the memory 602 can be connected to each other via a communication bus 603. The communication bus 603 can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus. The communication bus 603 can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 6 The memory 602 is used to store computer programs. The computer programs include program instructions. The processor 601 is configured to call the program instructions. The program includes instructions for executing Figure 2 The method shown.

[0171] The processor 601 may be a general purpose central processing unit (CPU), a microprocessor, an application-specific integrated circuit (ASIC), or one or more integrated circuits for controlling the execution of the above program.

[0172] The memory 602 may be a read-only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read-only memory (EEPROM), a compact disc read-only memory (CD-ROM) or other optical disc storage, optical disc storage (including compressed optical disc, laser disc, optical disc, digital versatile disc, Blu-ray disc, etc.), a magnetic disk storage medium or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of instructions or data structures and can be accessed by a computer, but is not limited thereto. The memory may exist independently and be connected to the processor via a bus. The memory may also be integrated with the processor.

[0173] In addition, the electronic device 600 may also include common components such as a communication interface and an antenna, which will not be described in detail here.

[0174] In an embodiment of the present application, the image to be inspected containing the threaded connector can be segmented pixel by pixel, and the various sub-components of the threaded connector in the image to be inspected can be segmented to obtain a segmentation feature map of the various sub-components of the threaded connector. Defect identification can be performed on the segmentation feature map of each sub-component to identify whether there are defects in the various sub-components of the threaded connector. The sub-component of the threaded connector where the defect is located can be located, thereby locating the defective position of the threaded connector.

[0175] An embodiment of the present application also provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program for electronic data exchange, and the computer program enables a computer to execute part or all of the steps of any one of the threaded connection defect detection methods recorded in the above method embodiments.

[0176] It should be noted that, for the aforementioned method embodiments, for the sake of simplicity, they are all expressed as a series of action combinations, but those skilled in the art should be aware that the present application is not limited by the described order of actions, because according to the present application, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily required by the present application.

[0177] In the above embodiments, the description of each embodiment has its own emphasis. For parts that are not described in detail in a certain embodiment, reference can be made to the relevant descriptions of other embodiments.

[0178] In the several embodiments provided in the present application, it should be understood that the disclosed device can be implemented in other ways. For example, the device embodiments described above are only schematic, such as the division of the units, which is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the device or unit can be electrical or other forms.

[0179] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0180] In addition, the functional units in the various embodiments of the application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated units may be implemented in the form of hardware or in the form of software program modules.

[0181] If the integrated unit is implemented in the form of a software program module and sold or used as an independent product, it can be stored in a computer-readable memory. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a memory, including a number of instructions to enable a computer device (which can be a personal computer, server or network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned memory includes: U disk, read-only memory (ROM), random access memory (RAM), mobile hard disk, disk or optical disk and other media that can store program codes.

[0182] A person skilled in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable memory, which can include: a flash drive, a read-only memory, a random access memory, a magnetic disk or an optical disk, etc.

[0183] The embodiments of the present application are introduced in detail above. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea. At the same time, for general technical personnel in this field, according to the idea of ​​the present application, there will be changes in the specific implementation method and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A method for detecting defects in threaded connections, characterized in that: include: Using the trained segmentation network, the image to be detected containing the threaded connection is segmented pixel by pixel to obtain a segmentation feature map containing at least one subcomponent; Performing defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map; determining whether the at least one subcomponent has a defect based on a dimension defined as a defect in the feature vector; The segmentation network includes a feature extraction network and a feature fusion network. The trained segmentation network is used to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector to obtain a segmentation feature map containing at least one subcomponent, including: Using the feature extraction network to extract features from the image to be detected containing the threaded connector, and obtaining the bottom-level features and high-level features of the image to be detected; Using the feature fusion network to fuse the bottom-level features and the high-level features, to obtain a feature map of the image to be detected; The feature map of the image to be detected is classified pixel by pixel according to the feature category label information obtained by supervised training, and the category of each pixel in the image to be detected is predicted to obtain a background segmentation feature map of the image to be detected and a segmentation feature map containing at least one subcomponent, wherein the feature category label information includes a background category label and category labels of each subcomponent of the threaded connector.

2. The method according to claim 1, characterized in that: The performing defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map includes: The trained defect recognition network is used to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map.

3. The method according to claim 1, characterized in that The determining whether the at least one subcomponent has a defect according to the dimension defined as a defect in the feature vector comprises: The dimensions defined as defects in the feature vector are added to obtain a defect score of the at least one subcomponent. If the defect score is greater than a target threshold, it is determined that the at least one subcomponent has a defect.

4. The method according to claim 1, characterized in that: The at least one subcomponent includes at least one of a bolt head, a stud, and a nut.

5. The method according to any one of claims 1 to 4, characterized in that: The method further comprises: Determine whether the image to be detected meets the image input requirements of the segmentation network; If the image to be detected meets the image input requirement of the segmentation network, the step of using the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector is performed.

6. The method according to claim 5, characterized in that The method further comprises: If the image to be detected does not meet the image input requirement of the segmentation network, the image to be detected is preprocessed to obtain the image to be detected containing the threaded connector.

7. The method according to claim 6, characterized in that The preprocessing of the image to be detected to obtain the image to be detected containing the threaded connector includes: The size of the image to be detected is adjusted to a specific size required by the image size of the segmentation network, and the variance and mean of the pre-trained network are used to normalize the image to be detected after the size adjustment to obtain the image to be detected containing the threaded connection.

8. A defect detection device for a threaded connection, characterized in that: include: A segmentation unit, used to segment the image to be detected containing the threaded connector pixel by pixel using a trained segmentation network to obtain a segmentation feature map containing at least one subcomponent; A defect recognition unit is used to perform defect recognition on the segmentation feature map to obtain a feature vector corresponding to the segmentation feature map, and determine whether the at least one subcomponent has a defect according to a dimension defined as a defect in the feature vector; The segmentation network includes a feature extraction network and a feature fusion network. The segmentation unit uses the trained segmentation network to perform pixel-by-pixel segmentation on the image to be detected containing the threaded connector to obtain a segmentation feature map containing at least one subcomponent, including: Using the feature extraction network to extract features from the image to be detected containing the threaded connector, and obtaining the bottom-level features and high-level features of the image to be detected; Using the feature fusion network to fuse the bottom-level features and the high-level features, to obtain a feature map of the image to be detected; The feature map of the image to be detected is classified pixel by pixel according to the feature category label information obtained by supervised training, and the category of each pixel in the image to be detected is predicted to obtain a background segmentation feature map of the image to be detected and a segmentation feature map containing at least one subcomponent, wherein the feature category label information includes a background category label and category labels of each subcomponent of the threaded connector.

9. An electronic device, characterized in that: The method comprises a processor and a memory, wherein the memory is used to store a computer program, the computer program comprises program instructions, and the processor is configured to call the program instructions to execute the method according to any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, wherein the computer program includes program instructions, and when the program instructions are executed by a processor, the processor is enabled to perform the method according to any one of claims 1 to 7.

Images