A method, device, equipment, medium and product for determining bolt state parameters

Abstract

The present invention provides a method, device, equipment, medium and product for determining bolt state parameters, relating to the technical field of automatic detection of the health state of structural bolts. The method specifically includes: determining a target detection model based on the detection requirements of the bolt to be detected, and inputting the bolt information of the bolt to be detected into the target detection model to obtain the first bolt image information of the bolt to be detected; the bolt to be detected includes at least one bolt; performing image calibration processing on the first bolt image information to obtain the second bolt image information; the marking line of the bolt in the second bolt image information faces the preset direction; determining the state parameters of the bolt to be detected based on the second bolt image information, where the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The present invention can realize automatic non-contact non-destructive detection of the bolt loosening angle and pre-tightening force, and greatly improve the efficiency and accuracy of bolt health state detection.

Description

Technical Field

[0001] The present invention relates to the technical field of automatic detection of the health status of structural bolts, and particularly to a method, device, equipment, medium and product for determining bolt state parameters. Background Art

[0002] Bolt connections have the advantages of convenient construction, safety and reliability, etc., and are widely used to connect various structural members. During the actual operation and use of bolted structures, they will be affected by the coupled action of complex environments and reciprocating dynamic loads, and are in a state of vibration and load for a long time. It is extremely easy to have problems such as bolt loosening and even fatigue fracture, which will then cause the reduction of bolt pre-tightening force and even affect the bearing capacity and service safety of the structure. Therefore, the detection of bolt state parameters is a problem that has attracted much attention in the field of automation.

[0003] Currently, the methods for determining bolt state parameters mainly include on-site manual detection and sensor detection. The on-site manual detection methods mainly include the scribing calibration detection method, the torque wrench method and the strain measurement method. This method has low detection efficiency, high missed detection rate and false detection rate, low operation safety, and the detection results highly depend on the detection experience of the detection personnel; the sensor detection method is to install various sensors at the bolts, and analyze the sensor data by combining various data analysis methods to evaluate the connection state of the bolts. However, the sensor data has strong non-linearity and randomness, the accuracy of the data analysis results is relatively low, and a large number of complex operations are involved in the analysis process, which requires high requirements for analysts and processing equipment, high detection costs and low universality. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to overcome the deficiencies of the prior art and provide a method, device, equipment, medium and product for determining bolt state parameters. The parameter determination method of the present invention is a method for detecting bolt loosening and pre-tightening force based on computer vision.

[0005] To solve the above technical problem, the present invention proposes a method for determining bolt state parameters, which specifically includes:

[0006] Based on the detection requirements of the bolt to be detected, determine the target detection model, and input the bolt information of the bolt to be detected into the target detection model to obtain the first bolt image information of the bolt to be detected; wherein, the bolt information of the bolt to be detected is the image data of the bolt to be detected determined by means of computer vision, and the bolt to be detected includes at least one bolt;

[0007] Perform image calibration processing on the first bolt image information to obtain the second bolt image information; wherein, the marking line orientation of the bolt in the second bolt image information is the preset orientation;

[0008] Based on the second bolt image information, determine the state parameters of the bolt to be detected. Among them, the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The bolt loosening angle is determined based on the included angle of the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle of the marking lines, the bolt loosening angle, and the bolt length increment of each bolt are determined according to the orientation of each bolt in the second bolt image information.

[0009] Compared with the prior art, the advantages of the present invention are as follows: It can realize the automatic non-contact and non-destructive detection of the bolt loosening angle and the pre-tightening force, greatly improving the efficiency and accuracy of the bolt health status detection. Secondly, the present invention uses computer vision to automatically identify the bolt marking method, replacing the manual inspection work, greatly improving the efficiency. Only by taking images can batch detection be completed, simplifying the workload and saving costs. Moreover, the present invention uses computer vision to calculate the bolt length increment, and then completes the calculation of the bolt pre-tightening force increment. Compared with the existing contact-type bolt pre-tightening force measurement method, a large number of expensive sensors are saved, and the complex analysis process of a large amount of sensor data is omitted. Non-contact non-destructive automatic detection can be realized, greatly improving the intelligent level and also reducing costs.

[0010] Based on the above method, the present invention also provides a device for determining bolt state parameters, an electronic device, a computer-readable storage medium, and a computer program product.

[0011] The device for determining bolt state parameters is used to implement the method for determining bolt state parameters in any embodiment of the present invention. The device includes:

[0012] The first determination module is used to determine the target detection model based on the detection requirements of the bolt to be detected, and input the bolt information of the bolt to be detected into the target detection model to obtain the first bolt image information of the bolt to be detected. Among them, the bolt information of the bolt to be detected is the image data of the bolt to be detected determined by the computer vision method, and the bolt to be detected includes at least one bolt.

[0013] The second determination module is used to perform image calibration processing on the first bolt image information by using the pre-set image calibration rules to obtain the second bolt image information. Among them, the orientation of the marking line of the bolt in the second bolt image information is the preset orientation.

[0014] A parameter determination module is configured to determine the state parameters of the bolt to be detected based on the second bolt image information, where the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The bolt loosening angle is determined based on the included angle between the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle between the marking lines of each bolt, the bolt loosening angle, and the bolt length increment are determined according to the orientation of each bolt in the second bolt image information.

[0015] The electronic device includes: at least one processor; and a memory communicatively connected to the at least one processor; the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor to enable the at least one processor to execute the method for determining the bolt state parameters in any embodiment of the present invention.

[0016] The computer-readable storage medium stores computer instructions for causing a processor to implement the method for determining the bolt state parameters in any embodiment of the present invention when executed.

[0017] The computer program product includes a computer program that implements the method for determining the bolt state parameters in any embodiment of the present invention when executed by a processor.

[0018] It should be understood that the content described in this part is not intended to identify the key or important features of the embodiments of the present invention, nor is it used to limit the scope of the present invention. Other features of the present invention will become easily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Hereinafter, the present invention will be described in more detail based on embodiments and with reference to the drawings. Among them:

[0020] Figure 1 is a schematic flowchart of a method for determining bolt state parameters provided in Embodiment 1 of the present invention;

[0021] Figure 2 is a schematic diagram of a YOLOv8 network structure provided in Embodiment 1 of the present invention;

[0022] Figure 3 is a schematic diagram of a group of bolt images provided in Embodiment 1 of the present invention;

[0023] Figure 4 is a schematic diagram of the structure of an attention mechanism model provided in Embodiment 1 of the present invention;

[0024] Figure 5 is a schematic diagram of the structure of channel attention provided in Embodiment 1 of the present invention;

[0025] Figure 6It is a schematic structural diagram of spatial attention provided in the first embodiment of the present invention;

[0026] Figure 7 It is a schematic diagram of an object detection model based on transfer learning provided in the first embodiment of the present invention;

[0027] Figure 8 It is a schematic diagram of a learning rate decay curve provided in the first embodiment of the present invention;

[0028] Figure 9 It is a schematic diagram of another group of bolt images provided in the first embodiment of the present invention;

[0029] Figure 10 It is a schematic diagram of a group of uncalibrated bolt images provided in the first embodiment of the present invention;

[0030] Figure 11 It is a schematic diagram of a group of calibrated bolt images provided in the first embodiment of the present invention;

[0031] Figure 12 It is a schematic diagram of a group of bolt images with marked lines provided in the first embodiment of the present invention;

[0032] Figure 13 It is a schematic diagram of a group of bolt images for extracting the contour of the marked line provided in the first embodiment of the present invention;

[0033] Figure 14 It is a schematic diagram of the minimum circumscribed rectangle of the marked line of a group of bolts provided in the first embodiment of the present invention;

[0034] Figure 15 It is a schematic diagram of a directed vector of the marked line provided in the first embodiment of the present invention;

[0035] Figure 16 It is a schematic diagram of a bolt structure provided in the first embodiment of the present invention;

[0036] Figure 17 It is a schematic diagram for calculating the length increment of a bolt provided in the first embodiment of the present invention;

[0037] Figure 18 It is a schematic structural diagram of a device for determining bolt state parameters provided in the second embodiment of the present invention;

[0038] Figure 19 It is a schematic structural diagram of an electronic device provided in the third embodiment of the present invention. Detailed implementation manners

[0039] The present invention will be further described in detail below with reference to the accompanying drawings of the specification and specific embodiments, but the protection scope of the present invention is not limited thereby.

[0040] In the description of the present invention, it should be understood that the orientation or positional relationship indicated by the terms "length", "width", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore should not be construed as a limitation to the present invention.

[0041] In addition, the terms "first" and "second" are only used for descriptive purposes and should not be construed as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include one or more of such features. In the description of the present invention, the meaning of "a plurality of" is two or more, unless otherwise specifically defined.

[0042] In the present invention, unless otherwise clearly specified and defined, the terms "assembled", "connected", "joined", "fixed", etc. should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or integrated; it may be a mechanical connection or an electrical connection; it may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present invention can be understood according to specific circumstances.

[0043] In recent years, the rapid rise of computer vision technology has provided new ideas for the evaluation of bolt loosening and pre-tightening force. Many scholars have used machine learning and image processing technologies to evaluate bolt status, including support vector machines, threshold segmentation, edge detection, and Hough (a feature extraction technology) line detection. This method has been improved compared with the manual on-site detection method, but the detection results are greatly affected by the external environment, and manual feature extraction is still required. The accuracy and intelligence level need to be improved. The deep learning method has the ability to directly extract features from images and is an advanced data analysis method. It is a feasible method to apply the deep learning method to the evaluation of bolt connection status. Some scholars directly extract the distance features between the screw and the connecting plate, and then classify loose and non-loose bolts. However, the minimum recognition distance of this method is about 4 mm, which is suitable for obvious large loosening; for minor bolt loosening, some scholars calculate the bolt loosening angle by detecting the coordinates of six key points of the bolt. However, this method is only applicable to the identification of bolts with a loosening angle range of 0-60°, and cannot reflect the change of bolt pre-tightening force. It has certain limitations for bolts with a loosening angle exceeding 60°. The overall detection accuracy needs to be improved. The present invention combines computer vision to analyze the included angle between the current marking line and the ideal marking line of the scribed marking bolt, so as to determine the loosening state of the bolt, and can further determine the change of bolt pre-tightening force according to the included angle and length increment of the bolt. Moreover, the present invention belongs to a non-contact non-destructive automatic detection method, which can significantly improve the efficiency and accuracy of bolt health status evaluation.

[0044] The following further elaborates on the present invention in detail with reference to the accompanying drawings of the specification and specific embodiments.

[0045] Embodiment 1:

[0046] Figure 1 It is a schematic flowchart of a method for determining bolt status parameters provided by the present invention. This embodiment is applicable to accurately and cost-effectively determining the bolt loosening angle and bolt pre-tightening force. This method can be executed by the bolt status parameter determination device provided by the present invention. The device can be implemented in the form of hardware and / or software. In a specific embodiment, the device can be integrated into an electronic device. The following embodiments will take the integration of the device into an electronic device as an example for illustration. Referring to Figure 1 The method specifically includes the following steps:

[0047] S101. Based on the detection requirements of the bolt to be detected, determine the target detection model, and input the bolt information of the bolt to be detected into the target detection model to obtain the first bolt image information of the bolt to be detected.

[0048] The method of the present invention can be used to detect the state parameters of a single bolt or a batch of bolts. That is, the bolts to be detected include at least one bolt, and the bolt information of the bolts to be detected is the image data of the bolts to be detected determined by means of computer vision. The first bolt image information can be understood as an image representing the current state of the bolts to be detected. The bolt information can be understood as the image data of the bolts to be detected. The target detection model can be understood as an algorithm that processes the bolt information to obtain the first bolt image information. The detection requirement is used to indicate the goal of the current bolt detection work, so as to perform the detection work more in line with the user's needs.

[0049] The target detection model of the present invention includes a single-stage algorithm model and a two-stage algorithm model, which are respectively used to perform detection work with different requirements. The detection requirements include detection speed requirements and detection accuracy requirements.

[0050] The single-stage algorithm model directly analyzes the input image to obtain the positions and categories of each target in the input image, including but not limited to You Only Look Once (YOLO series), RetinaNet, Single Shot MultiBox Detector (SSD), CenterNet, EfficientDet, and Swin Transformer; the two-stage algorithm model analyzes the input image to obtain multiple candidate regions, classifies and locates each candidate region respectively, and then determines the positions and categories of each target in the input image, including but not limited to RCNN, Fast-RCNN, Faster-RCNN, Mask-RCNN, SPP-Net, FPN, R-FCN, and DetectoRS. Different target detection models can be selected according to different application scenarios (i.e., detection requirements). Specifically, the two-stage target detection model has high accuracy, and the single-stage target detection model has high speed. When high detection accuracy is required but detection speed is not required, a two-stage target detection model can be selected; when high detection speed is required but detection accuracy is not required, a single-stage target detection model can be selected. Therefore, based on the detection requirements of the bolts to be detected, the target detection model is determined, including: when the detection requirement is a detection speed requirement, the target detection model is determined to be a single-stage algorithm model; when the detection requirement is a detection accuracy requirement, the target detection model is determined to be a two-stage algorithm model.

[0051] The present invention can select the single-stage target detection network YOLOv8 to build a bolt target detection model. Figure 2 It is a schematic diagram of a YOLOv8 network structure provided in Embodiment 1 of the present invention. From Figure 2It can be seen that the YOLOv8 network structure mainly consists of three parts: a feature extraction network, a feature fusion network, and a decoupled detection head. Determining the object detection model includes: obtaining an initial detection model and an initial bolt image dataset; using a pre-set dataset augmentation scheme to supplement the initial bolt image dataset to obtain a target bolt image dataset; using the target bolt image dataset to train the initial detection model, and when the initial detection model meets the preset model convergence condition, determining the trained initial detection model as the target detection model.

[0052] The initial bolt image dataset can be understood as bolt images with markings taken through multiple channels using intelligent devices. The target bolt image dataset can be understood as the augmented bolt image dataset. For example, it is a dataset made by augmenting the collected dataset using various conventional and deep learning methods and annotating the images. The dataset augmentation scheme includes augmenting the dataset using basic methods and using deep learning methods. The basic methods can be understood as common conventional dataset augmentation methods.

[0053] The intelligent devices for collecting images include smartphones, cameras, drones, etc. The devices are not limited, but different angles, distances, and light intensities should be considered when taking images to ensure the representativeness and diversity of the obtained image dataset. It should be noted that to ensure the training effect of the model, at least 500 images should be taken.

[0054] The methods for dataset augmentation include, but are not limited to, augmenting the collected dataset using conventional methods and using deep learning methods. The methods for augmenting the collected dataset using conventional methods include randomly rotating the images, randomly scaling, adding random noise, randomly occluding the images, changing the color temperature, and convolutional calculation. The methods for augmenting the collected dataset using deep learning methods include, but are not limited to, using RandAugment, AutoAugment, TrivialAugment, generative adversarial networks (GANs), and various variants to generate augmented datasets. When using a deep convolutional generative adversarial network to augment the dataset, the detailed network parameters of the method used are shown in Table 1.

[0055]

[0056] When annotating the bolt images to make a dataset, the augmented dataset should be at least of the order of 10 3 magnitude. Specifically, an image annotation software LabelImg can be used to annotate the comprehensively augmented dataset, including two categories: bolt targets with marking lines and bolt targets without marking lines. The dataset folder includes image files in jpg format and bolt target position files in txt format. The bolt images can be, for example,Figure 3 , Figure 3 is a schematic diagram of a set of bolt images provided in the first embodiment of the present invention. This figure includes 9 bolts. As can be seen from Figure 3 it, the three-dimensional structure and the orientation of the marking lines of each bolt are shown. The triangle on the bolt represents the bolt marking line, and the direction away from the bolt center is the orientation of the bolt marking line.

[0057] When training the model, a detection model (i.e., the initial detection model) will be built based on the deep learning platform. After dividing the dataset, it will be input into the initial detection model, and the transfer learning method will be combined to accelerate the model training to obtain the target detection model (i.e., the optimal detection model obtained through training). The optimal target detection model obtained through training can also be used for image inference to obtain the bolt target and the detection results of the bolt with markings.

[0058] The deep learning platform of the present invention can select TensorFlow, PyTorch, Keras, etc. After the detection is selected, the detection model can be further improved. For example, replace the feature extraction module, add a channel attention mechanism or a spatial attention mechanism, etc., to enhance the performance of the detection model. In the present invention, the CBAM (Convolutional Block Attention Module) attention mechanism can be added to the YOLOv8 feature extraction network. The structure of the attention mechanism model is as Figure 4 shown. CBAM is a model that combines channel attention and spatial attention, and can enhance the image attention ability of the convolutional neural network. The structure of the channel attention module is as Figure 5 shown, and the structure of the spatial attention module is as Figure 6 shown. Dataset division means dividing the images into a training set, a validation set, and a test set according to a certain ratio, which are respectively used for the training, validation, and testing of the model. In the present invention, the training set, the validation set, and the test set can be divided according to the ratio of 7:2:1. Transfer learning (TransferLearning, TL) is to transfer the knowledge of one domain (i.e., the source domain) to another domain (i.e., the target domain) so that the target domain can achieve better learning results. The source domain can select various large datasets, such as the COCO, ImageNet, CIFAR-10, etc. datasets. The target domain is the bolt image dataset. The feature extraction network that has been pre-trained in the source domain is transferred to the target domain, and then a new classifier is trained on the target domain. In order to make the source domain and the target domain have a high correlation, the present invention selects the COCO dataset as the source domain and the bolt image dataset as the target domain. The target detection model based on transfer learning is as Figure 7 shown. The datasets in the figure are: the COCO dataset and the bolt image dataset.

[0059] Specifically, during model training, the hyperparameters of the network need to be set in advance, and the values of the hyperparameters will affect the performance of the model after training. Common hyperparameters include the input image size, initial learning rate, optimizer, number of iterative training epochs, batch size, etc. The larger the input image size, the higher the model accuracy, but it will occupy more video memory and require higher computer requirements; the initial learning rate is an important hyperparameter for adjusting the network weights through the gradient of the loss function. If it is set too large, it is easy to produce oscillations, and if it is set too small, the convergence speed is too slow. Therefore, the initial learning rate should not be set too large, and it is appropriate to be between 0.01 and 0.001, and it should decay with the increase of the number of iterations during training. The initial learning rate of the present invention decays according to the cosine function, and the learning rate decay curve is as Figure 8 shown. The optimizer is used to optimize the parameter values of network neurons, and reasonable settings can make the model converge to the optimal solution faster and more accurately; the number of iterative training epochs is the total number of times the model is trained, and one training of all datasets is one epoch; the batch size is the number of samples selected in batches each time, and the size of this parameter will affect the convergence speed and optimization degree of the model, and it is generally set to the Nth power of 2. The training hyperparameters of the YOLOv8 object detection model of the present invention are shown in Table 2.

[0060]

[0061] In the present invention, the loss function of YOLOv8 is composed of three parts: detection box loss, confidence loss, and classification loss, and the total loss function , represents the detection box loss, represents the confidence loss, represents the classification loss, represents the weight coefficient of the detection box loss, represents the weight coefficient of the confidence loss, represents the weight coefficient of the classification loss. During the iterative training of the network, it is necessary to continuously update the network parameters according to the loss value of the model to minimize the loss value. The corresponding model parameters are the optimal parameters. In order to determine the optimal parameters when the loss value reaches the minimum, the gradient descent method is commonly used to update the weights and bias parameters of each layer of the network. The update method is: , represents the th layer of the network; are the weight matrix and bias vector before update respectively, is the learning rate, is the loss function; are the weight matrix and bias vector after update respectively.

[0062] After the model training is completed, it is necessary to evaluate the model performance to determine whether the trained model has good performance. Common evaluation metrics for deep learning models include precision, recall, average precision, etc. In the present invention, the precision p, recall r, and F1-score metrics are selected to evaluate the model performance. Specifically, the precision , the recall , the F1-score , where TP represents the positive samples that are truly positive predicted as positive, FP represents the negative samples that are truly negative predicted as positive, and FN represents the positive samples that are truly positive predicted as negative.

[0063] After the model is trained and tested successfully, the bolt image to be detected can be input into the trained optimal object detection model to obtain the detection results of the bolt targets with marked lines and the bolt targets without marked lines. The detection results include the position information and confidence of the predicted targets, as Figure 9 shown, Figure 9 is a schematic diagram of another group of bolt images provided in the first embodiment of the present invention, which is essentially the image information output by the object detection model. It can be seen the position and confidence of the prediction boxes of each bolt and other information.

[0064] S102. Perform image calibration processing on the first bolt image information to obtain the second bolt image information.

[0065] The second bolt image information can be understood as the corrected first bolt image information, and the orientation of the marked lines of the bolts in the second bolt image information is the preset orientation. The correction work can be to register the bolt center point coordinates based on the object detection results, calculate the perspective transformation matrix, and complete the image correction.

[0066] In one embodiment, S102 may specifically include: 1) Input the first bolt image information into the object detection model for processing to obtain the bolt prediction box position information, where the bolt prediction box position information includes at least one bolt prediction box, and the bolt prediction box corresponds to the bolt one by one; for example, the bolt image can be input into the trained optimal object detection model to obtain the detection results of the bolt targets without marked lines, and then the prediction box position information in the detection results can be obtained. 2) Based on at least one bolt prediction box, determine the target center point coordinates of at least one bolt; for example, the center point coordinates of each bolt can be calculated according to the prediction box position information of each bolt to determine the center point coordinates of the bolts in the corrected image. 3) Based on the target center point coordinates of at least one bolt and the initial center point coordinates, determine the calibration matrix; for example, according to the position relationship of the coordinate points before and after correction, use the perspective transformation formula to calculate the perspective transformation matrix. 4) Process the first bolt image information based on the calibration matrix to obtain the second bolt image information; for example, use the perspective transformation matrix to reproject the distorted image coordinate points to complete the correction of the distorted image.

[0067] The initial center point coordinates are the center point coordinates of the uncorrected bolts, that is, the center point coordinates of each bolt in the first bolt image information, which are used to represent the current state of the bolts. The target center point coordinates are the center point coordinates of the corrected bolts, that is, the center point coordinates of each bolt in the second bolt image information, which are used to represent the ideal state of the bolts. The calibration matrix can be determined based on the current state and the ideal state of the bolts.

[0068] In the present invention, the calibration matrix (i.e., the perspective transformation matrix) is solved as follows: , and They are the corresponding coordinate points of the views before and after correction, represents the initial center point coordinates, The first axis value representing the coordinates of the initial center point of the at least one bolt, that is, the coordinate value in the x-axis direction, The second axis value representing the coordinates of the initial center point of the at least one bolt, that is, the coordinate value in the y-axis direction, represents the coordinates of the target center point, a first axis value representing the target center point coordinates of the at least one bolt, a second axis value representing the target center point coordinates of the at least one bolt, represents the perspective transformation matrix, , ,…, all represent the element values of the perspective transformation matrix.

[0069] Figure 10 is a schematic diagram of a group of uncalibrated bolt images provided by the first embodiment of the present invention, Figure 11 is a schematic diagram of a set of calibrated bolt images provided in the first embodiment of the present invention, Figure 11 After the calibration matrix conversion Figure 10 , combined with Figure 10 and Figure 11 It can be seen that the calibration method of the present invention can well complete the correction of the image so as to perform subsequent parameter detection work, and the direction of the marking line is the preset direction, which can be understood as aligning the bolt in one direction.

[0070] S103: Determine state parameters of the bolt to be inspected based on the second bolt image information.

[0071] Among them, the state parameter is the bolt loosening angle and / or the bolt pre-tightening force. Determining the state parameter of the bolt to be detected based on the second bolt image information can be understood as determining the bolt loosening angle and / or the bolt pre-tightening force of each bolt based on the second bolt image information. Specifically, the bolt loosening angle is determined based on the included angle between the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle between the marking lines, the bolt loosening angle, and the bolt length increment of each bolt are determined according to the orientation of each bolt in the second bolt image information.

[0072] On the one hand, based on the second bolt image information, determining the bolt loosening angle of the bolt to be detected includes: determining the first marking line and the second marking line of at least one bolt based on the orientation of the marking lines of each bolt in the second bolt image information; determining the bolt loosening angle of at least one bolt based on the included angle information between the first marking line and the second marking line of at least one bolt.

[0073] The present invention extracts the straight line feature information of the marking lines on the bolt with marking lines, calculates the included angle between the two straight lines to determine the bolt loosening angle. One straight line is the current marking line of the bolt, and the other straight line is the ideal marking line of the bolt. The included angle between the two straight lines is the offset angle of the bolt, which can be used to represent the degree of bolt loosening.

[0074] Specifically, the process of determining the bolt loosening angle includes: 1) Inputting the corrected bolt image that simultaneously includes the first marking line and the second marking line into the trained optimal object detection model, obtaining the detection result of the bolt with marking lines, and cropping the bolt with marking lines according to the prediction box to obtain a sub-graph of the bolt with marking lines (see details in Figure 12 ). 2) Sequentially performing grayscale conversion, binarization, corrosion, and dilation operations on the sub-graph of the bolt with marking lines, and then using an edge detection algorithm to extract the contour information of the marking lines to obtain a marking line contour image (see details in Figure 13 ). 3) Using the minimum convex hull principle to calculate and obtain the minimum circumscribed rectangle of the marking line contour in step 2) to obtain the minimum circumscribed rectangle (see details in Figure 14 ). 4) Obtaining two points on the marking line by calculating the midpoint of the short side of the rectangle, and then determining the order according to the distance between the two coordinate points and the center point of the bolt, and constructing a directed direction vector from the near point to the far point (see details in Figure 15 ). 5) Calculating the included angle value between the directed direction vectors on the two marking lines to obtain the bolt loosening angle.

[0075] Furthermore, in combination with Figure 15, O is the coordinate of the center point of the bolt, which is calculated by the target detection frame; O1 and O2 can determine the straight line L1, O1 is closer to point O than O2, so the directed vector is O1 pointing to O2, and similarly, the directed vector from O3 to O4 can be obtained. After obtaining the equations of the two marked lines, the bolt loosening angle , and are the direction vectors of the two lines respectively; and They correspond to the modulus of the direction vector respectively.

[0076] On the other hand, based on the second bolt image information, the bolt preload of the bolt to be detected is determined, including: determining the nut surface loosening angle and the nut surface loosening angle of at least one bolt based on the second bolt image information; processing the second bolt image information based on the target detection model to obtain the nut surface prediction frame information and the nut surface prediction frame information of at least one bolt; determining the measured length of at least one bolt based on the nut surface prediction frame information and the nut surface prediction frame information of at least one bolt, and determining the bolt length increment of at least one bolt based on the measured length and the preset length; determining the bolt preload of at least one bolt based on the stiffness of the connector, the stiffness of the bolt, the looseness angle of the nut surface and the looseness angle of the nut surface of at least one bolt; or, determining the bolt preload of at least one bolt based on the stiffness of the connector, the stiffness of the bolt, the looseness angle of the nut surface of the at least one bolt and the bolt length increment.

[0077] The structure of the bolt detected by the present invention is detailed in Figure 16 . Specifically, the process of determining the bolt preload includes: 1) determining the nut face angle looseness value and the nut face angle looseness value in turn according to the calculation method of the bolt looseness angle, subtracting the nut face angle looseness value and the nut face angle looseness value to obtain the relative looseness angle increment of the bolt. 2) inputting the bolt image into the optimal target detection model for processing, obtaining the nut face prediction frame information and the nut face prediction frame information, obtaining the vertex coordinates of each detection frame, calculating the distance between the farthest vertices of the upper and lower prediction frames, obtaining the initial length of the bolt in the tightened state (which can be understood as the preset length), and then subtracting the initial length from the calculated bolt length (ie, the measured length) to obtain the bolt length increment. 3) obtaining the stiffness of the connection and the stiffness of the bolt itself, and then jointly calculating the preload change value (ie, the bolt preload) by the connection stiffness, the bolt stiffness, the bolt length increment or the relative looseness angle increment of the bolt.

[0078] Bolt loosening angle increment , Indicates the loose angle of the nut surface, Indicates the loose angle of the nut surface. Figure 17 is a schematic diagram of calculating the bolt length increment provided in the first embodiment of the present invention, with reference to Figure 17The coordinates of the four vertices of the nut face prediction box are P1~P4, and the coordinates of the four vertices of the nut face prediction box are P5~P8. The distances between the farthest vertices of the two prediction boxes are the distance between P1 and P7, and the distance between P2 and P8. The calculation method of the bolt length increment is the simultaneous formula , , and .

[0079] Among them, the predicted box vertex The coordinates of , is the length calculation error, the theoretical value is 0, is the error correction factor, which is equal to the length calculation error corresponding; is the calculated bolt length, the length unit is pixel; It is the conversion coefficient between pixel length and real length, and its unit is mm / pixel. The conversion coefficient can be obtained by calibrating the marker block. The bolt length converted into real value in millimeters; is the initial length of the bolt in the tightened state; The calculated bolt length increment.

[0080] The bolt preload is calculated as:

[0081] ,

[0082] .

[0083] in, is the bolt preload in the current state, is the preload force of the bolt in the initial state of tightening, is the change in bolt preload, is the thread pitch of the bolt, is the loose angle of the nut surface, is the loosening angle of the nut surface, is the bolt loosening angle increment, is the bolt length increment, is the bolt stiffness; is the stiffness of the connector, is the friction coefficient, which reflects the inhibitory effect of friction on the change of preload. The technical solution of this embodiment includes: 1) constructing an image data set, that is, using smart devices to take images of marked bolts in multiple ways, using a variety of conventional and deep learning methods to augment and expand the collected data set, and finally annotating the image to produce a data set. 2) Model training and reasoning, that is, building a target detection model based on a deep learning platform, dividing the data set and inputting it into the model, and combining the transfer learning method to accelerate model training, using the trained optimal target detection model to perform image reasoning, and obtain bolt target and marked bolt target detection results. 3) Perspective transformation to correct the image, that is, based on the target detection result, the coordinates of the center point of the bolt are aligned, and the perspective transformation matrix is calculated to complete the image correction. 4) Calculating the bolt loosening angle, that is, extracting the feature information of the marking line on the bolt with a marked line, and calculating the angle between the two straight lines to determine the bolt loosening angle. 5) Calculation of bolt preload: First, calculate the loose angles of the nut face and the nut face respectively, and subtract the two to obtain the relative loose angle increment of the bolt. Secondly, determine the bolt length increment according to the target detection results of the nut and the screw. Finally, calculate the preload change value by combining the stiffness of the connector, the stiffness of the bolt, the bolt length increment or the relative loose angle increment of the bolt. The aim is to achieve automatic non-contact non-destructive testing of the bolt loose angle and preload, and improve the efficiency and accuracy of bolt health status detection. Secondly, the present invention uses computer vision to automatically identify the bolt marking method, replacing manual inspection work, greatly improving efficiency, and only needs to take pictures to complete batch detection, simplifying the workload and saving costs. Finally, the present invention uses computer vision to calculate the bolt length increment, and then completes the calculation of the bolt preload increment. Compared with the existing contact bolt preload measurement method, it saves a large number of expensive sensors and eliminates the complex analysis process of a large amount of sensor data. It can realize non-contact non-destructive automatic detection, greatly improving the level of intelligence and reducing costs.

[0084] Embodiment 2:

[0085] Figure 18 Schematic diagram of the structure of a device for determining bolt state parameters provided by the present invention. Figure 18 As shown, the device includes: a first determination module 201, a second determination module 202 and a parameter determination module 203.

[0086] The first determination module 201 is used to determine the target detection model based on the detection requirements of the bolts to be detected, and input the bolt information of the bolts to be detected into the target detection model to obtain the first bolt image information of the bolts to be detected; wherein the bolt information of the bolts to be detected is the image data of the bolts to be detected determined by computer vision, and the bolts to be detected include at least one bolt.

[0087] The second determination module 202 is configured to perform image calibration processing on the first bolt image information by using a preset image calibration rule to obtain second bolt image information; wherein, the orientation of the marking line of the bolt in the second bolt image information is a preset orientation.

[0088] The parameter determination module 203 is configured to determine the state parameters of the bolt to be detected based on the second bolt image information, wherein the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The bolt loosening angle is determined based on the included angle of the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle of the marking lines, the bolt loosening angle, and the bolt length increment of each bolt are determined according to the orientation of each bolt in the second bolt image information.

[0089] Optionally, the target detection model includes a single-stage algorithm model and a two-stage algorithm model, and the detection requirements include detection speed requirements and detection accuracy requirements.

[0090] Optionally, the first determination module 201 is specifically configured to determine the target detection model as a single-stage algorithm model when the detection requirement is a detection speed requirement. The single-stage algorithm model is used to directly analyze the input image to obtain the positions and categories of the targets in the input image; when the detection requirement is a detection accuracy requirement, determine the target detection model as a two-stage algorithm model. The two-stage algorithm model is used to analyze the input image to obtain multiple candidate regions, and classify and locate each candidate region respectively to determine the positions and categories of the targets in the input image.

[0091] Optionally, the second determination module 202 is specifically configured to input the first bolt image information into the target detection model for processing to obtain bolt prediction box position information. The bolt prediction box position information includes at least one bolt prediction box, and each bolt prediction box corresponds to a bolt one by one; based on the at least one bolt prediction box, determine the target center point coordinates of at least one bolt; based on the target center point coordinates of at least one bolt and the initial center point coordinates, determine the calibration matrix; and process the first bolt image information based on the calibration matrix to obtain the second bolt image information.

[0092] Optionally, the correlation relationship among the calibration matrix, the target center point coordinates of at least one bolt, and the initial center point coordinates is: ;

[0093] Wherein, represents the first axis value of the initial center point coordinates of at least one bolt, represents the second axis value of the initial center point coordinates of at least one bolt, represents the first axis value of the target center point coordinates of at least one bolt, represents the second axis value of the target center point coordinates of at least one bolt, represents a calibration matrix, , , , , , , , and are the element values of the calibration matrix, = 1.

[0094] Optionally, the parameter determination module 203 is specifically configured to determine the first marking line and the second marking line of at least one bolt based on the orientation of the marking lines of each bolt in the second bolt image information; the first marking line and the second marking line are two vector lines determined based on the orientation of the marking lines of each bolt in the second bolt image information and the bolt center point; based on the included angle information between the first marking line and the second marking line of at least one bolt, determine the bolt loosening angle of at least one bolt.

[0095] Optionally, the parameter determination module 203 is specifically configured to determine the nut surface loosening angle and the nut surface loosening angle of at least one bolt based on the second bolt image information; process the second bolt image information based on the target detection model to obtain the nut surface prediction box information and the nut surface prediction box information of at least one bolt; based on the nut surface prediction box information and the nut surface prediction box information of at least one bolt, determine the measured length of at least one bolt, and determine the bolt length increment of at least one bolt based on the measured length and the preset length; based on the connector stiffness, the bolt stiffness, the nut surface loosening angle and the nut surface loosening angle of at least one bolt; or, based on the connector stiffness, the bolt stiffness, the nut surface loosening angle and the bolt length increment of at least one bolt, determine the bolt pre-tightening force of at least one bolt.

[0096] Optionally, the first determination module 201 is further configured to obtain an initial detection model and an initial bolt image dataset; use a pre-set dataset expansion scheme to supplement the initial bolt image dataset to obtain a target bolt image dataset, and the dataset expansion scheme includes expanding the dataset using a basic method and expanding the dataset using a deep learning method; use the target bolt image dataset to train the initial detection model, and when the initial detection model meets the preset model convergence condition, determine the trained initial detection model as the target detection model.

[0097] The bolt state parameter determination device provided in this embodiment can execute the bolt state parameter determination method provided in any embodiment of the present invention, and has the corresponding functional modules and beneficial effects for executing the method.

[0098] Embodiment 3:

[0099] Figure 19 This is a schematic structural diagram of an electronic device provided by the present invention. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smart phones, wearable devices (such as helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present invention described and / or claimed herein.

[0100] As Figure 19 shown, the electronic device 10 includes at least one processor 11, and a memory communicatively connected to the at least one processor 11, such as a read-only memory (ROM) 12, a random access memory (also known as a random access memory, RandomAccess Memory, RAM) 13, etc. Among them, the memory stores a computer program executable by the at least one processor. The processor 11 can perform various appropriate actions and processes according to the computer program stored in the read-only memory (Read Only Memory, ROM) 12 or the computer program loaded from the storage unit 18 into the random access memory (RAM) 13. In the RAM 13, various programs and data required for the operation of the electronic device 10 can also be stored. The processor 11, the ROM 12, and the RAM 13 are connected to each other through a bus 14. The input / output (I / O) interface 15 is also connected to the bus 14.

[0101] Multiple components in the electronic device 10 are connected to the I / O interface 15, including: an input unit 16, such as a keyboard, a mouse, etc.; an output unit 17, such as various types of displays, speakers, etc.; a storage unit 18, such as a disk, an optical disc, etc.; and a communication unit 19, such as a network card, a modem, a wireless communication transceiver, etc. The communication unit 19 allows the electronic device 10 to exchange information / data with other devices through a computer network such as the Internet and / or various telecommunication networks.

[0102] The processor 11 can be various general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any appropriate processor, controller, microcontroller, etc. The processor 11 executes the various methods and processes described above, such as the method for determining bolt state parameters.

[0103] In some embodiments, the method for determining the bolt state parameters can be implemented as a computer program tangibly embodied in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed onto the electronic device 10 via the ROM 12 and / or the communication unit 19. When the computer program is loaded into the RAM 13 and executed by the processor 11, one or more steps of the method for determining the bolt state parameters described above can be executed. Alternatively, in other embodiments, the processor 11 can be configured to execute the method for determining the bolt state parameters by any other suitable means (e.g., by means of firmware).

[0104] The various embodiments of the systems and techniques described above in this document can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), systems on a chip (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include: being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which can be a dedicated or general-purpose programmable processor that receives data and instructions from a storage system, at least one input device, and at least one output device, and transmits the data and instructions to the storage system, the at least one input device, and the at least one output device.

[0105] The computer programs for implementing the methods of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a dedicated computer, or other programmable data processing device, such that when the computer programs are executed by the processor, the functions / operations specified in the flowcharts and / or block diagrams are implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a stand-alone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0106] In the context of the present invention, a computer-readable storage medium can be a tangible medium that can contain or store a computer program for use by or in connection with an instruction execution system, apparatus, or device. The computer-readable storage medium can include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, the computer-readable storage medium can be a machine-readable signal medium. More specific examples of the machine-readable storage medium would include an electrical connection based on one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or Flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0107] To provide for interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and a pointing device (e.g., a mouse or a trackball) by which the user can provide input to the electronic device. Other kinds of devices can also be used to provide for interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0108] The systems and techniques described herein can be implemented in a computing system that includes backend components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes frontend components (e.g., a user computer having a graphical user interface or a web browser through which the user can interact with an implementation of the systems and techniques described herein), or a computing system that includes any combination of such backend components, middleware components, or frontend components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: local area network (LAN), wide area network (WAN), blockchain network, and the Internet.

[0109] A computing system may include clients and servers. The clients and servers are generally far from each other and usually interact via a communication network. The relationship between the clients and servers is generated by computer programs running on respective computers and having a client-server relationship with each other. A server may be a cloud server, also known as a cloud computing server or cloud host, which is a host product in a cloud computing service system, solving the defects of difficult management and weak business scalability existing in traditional physical hosts and VPS services.

[0110] In one embodiment, the present invention further includes a computer program product. The computer program product includes a computer program which, when executed by a processor, implements the method for determining bolt state parameters according to any embodiment of the present invention.

[0111] In the process of implementing the computer program product, computer program code for performing the operations of the present invention can be written in one or more programming languages or combinations thereof. The programming languages include object-oriented programming languages and also conventional procedural programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, executed as an independent software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer can be connected to the user's computer through any type of network - including a local area network (LAN) or a wide area network (WAN) - or can be connected to an external computer (for example, by connecting through an Internet service provider using the Internet).

[0112] It should be understood that various forms of the processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in the present invention can be executed in parallel, sequentially, or in different orders, as long as the desired results of the technical solution of the present invention can be achieved, and no limitation is made herein.

[0113] The above specific embodiments do not constitute a limitation on the protection scope of the present invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent replacements, and improvements made within the spirit and principles of the present invention shall be included within the protection scope of the present invention.

[0114] Although the present invention has been described with reference to preferred embodiments, various modifications can be made thereto and components thereof can be replaced with equivalents without departing from the scope of the present invention. In particular, as long as there is no structural conflict, the technical features mentioned in each embodiment can be combined in any way. The present invention is not limited to the specific embodiments disclosed herein, but includes all technical solutions falling within the scope of the claims.

Claims

1. A method for determining bolt state parameters, characterized in that, Including: Based on the detection requirements of the bolt to be detected, determine the target detection model, and input the bolt information of the bolt to be detected into the target detection model to obtain the first bolt image information of the bolt to be detected; wherein, the bolt information of the bolt to be detected is the image data of the bolt to be detected determined by means of computer vision, and the bolt to be detected includes at least one bolt. Perform image calibration processing on the first bolt image information to obtain second bolt image information; wherein, the orientation of the marking line of the bolt in the second bolt image information is a preset orientation. Based on the second bolt image information, determine the state parameters of the bolt to be detected, where the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The bolt loosening angle is determined based on the included angle between the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle between the marking lines of each bolt, the bolt loosening angle, and the bolt length increment are determined according to the orientations of the bolts in the second bolt image information. Among them, based on the second bolt image information, determining the bolt pre-tightening force of the bolt to be detected includes: based on the second bolt image information, determining the loosening angle of the nut surface and the loosening angle of the nut face of the at least one bolt; processing the second bolt image information by the target detection model to obtain the prediction box information of the nut surface and the prediction box information of the nut face of the at least one bolt; based on the prediction box information of the nut surface and the prediction box information of the nut face of the at least one bolt, determining the measured length of the at least one bolt, and determining the bolt length increment of the at least one bolt based on the measured length and the preset length; based on the connector stiffness, the bolt stiffness, the loosening angle of the nut surface and the loosening angle of the nut face of the at least one bolt; or, based on the connector stiffness, the bolt stiffness, the loosening angle of the nut surface and the bolt length increment of the at least one bolt, determining the bolt pre-tightening force of the at least one bolt; among them, the calculation expression of the bolt pre-tightening force is: , , ; , ; Among them, is the bolt pre-tightening force in the current state, is the pre-tightening force at the initial state of bolt tightening, is the change in bolt pre-tightening force, is the loosening angle of the nut face, is the loosening angle of the nut surface, is the bolt loosening angle increment, is the bolt length increment, is the bolt stiffness; is the connector stiffness, is the pitch of the bolt, is the initial length of the bolt in the tightened state, is the bolt length converted into the true value, is the calculated bolt length, is the conversion coefficient between the pixel length and the true length, is the error correction coefficient, is the friction coefficient, reflecting the inhibitory effect of friction on the change in pre-tightening force.

2. The method for determining the bolt state parameters according to claim 1, characterized in that, The target detection model includes a single-stage algorithm model and a two-stage algorithm model. The detection requirements include detection speed requirements and detection accuracy requirements. Based on the detection requirements of the bolt to be detected, determining the target detection model includes: When the detection requirement is the detection speed requirement, determine the target detection model as the single-stage algorithm model, wherein the single-stage algorithm model is used to directly analyze the input image to obtain the positions and categories of each target in the input image. When the detection requirement is the detection accuracy requirement, determine the target detection model as the two-stage algorithm model, wherein the two-stage algorithm model is used to analyze the input image to obtain multiple candidate regions, and classify and locate each candidate region respectively to determine the positions and categories of each target in the input image.

3. The method for determining the bolt state parameters according to claim 1, wherein The performing image calibration processing on the first bolt image information to obtain second bolt image information includes: Input the first bolt image information into the target detection model for processing to obtain bolt prediction box position information, the bolt prediction box position information includes at least one bolt prediction box, and the bolt prediction box corresponds to the bolt one by one. Based on the at least one bolt prediction box, determine the target center point coordinates of the at least one bolt. Based on the target center point coordinates and the initial center point coordinates of the at least one bolt, determine the calibration matrix. The association relationship among the calibration matrix, the target center point coordinates and the initial center point coordinates of the at least one bolt is: ； wherein, the first axis value representing the initial center point coordinates of the at least one bolt, the second axis value representing the initial center point coordinates of the at least one bolt, the first axis value representing the target center point coordinates of the at least one bolt, the second axis value representing the target center point coordinates of the at least one bolt, representing a calibration matrix, 、 、 、 、 、 、 、 、and are the element values of the calibration matrix, ; Based on the calibration matrix, process the first bolt image information to obtain the second bolt image information.

4. The method for determining the bolt state parameters according to claim 1, characterized in that, Based on the second bolt image information, determine the bolt loosening angle of the bolt to be detected, including: Based on the orientation of the marking line of each bolt in the second bolt image information, determine the first marking line and the second marking line of the at least one bolt, wherein the first marking line and the second marking line are two vector lines determined based on the orientation of the marking line of each bolt in the second bolt image information and the bolt center point. Based on the included angle information of the first marking line and the second marking line of the at least one bolt, determine the bolt loosening angle of the at least one bolt.

5. A device for determining bolt state parameters, characterized in that A device for determining the bolt state parameter for implementing the method for determining the bolt state parameter according to any one of claims 1 to 4, the device for determining the bolt state parameter includes: A first determination module, configured to determine a target detection model based on the detection requirements of the bolt to be detected, and input the bolt information of the bolt to be detected into the target detection model to obtain first bolt image information of the bolt to be detected; wherein, the bolt information of the bolt to be detected is image data of the bolt to be detected determined by means of computer vision, and the bolt to be detected includes at least one bolt; A second determination module, configured to perform image calibration processing on the first bolt image information by using a preset image calibration rule to obtain second bolt image information; wherein, the marking line of the bolt in the second bolt image information faces a preset direction; A parameter determination module, configured to determine the state parameters of the bolt to be detected based on the second bolt image information, wherein the state parameters are the bolt loosening angle and / or the bolt pre-tightening force. The bolt loosening angle is determined based on the included angle of the marking lines of each bolt, and the bolt pre-tightening force is determined based on the bolt loosening angle or the bolt length increment of each bolt. The included angle of the marking lines, the bolt loosening angle, and the bolt length increment of each bolt are determined according to the orientation of each bolt in the second bolt image information. Among them, determining the bolt pre-tightening force of the bolt to be detected based on the second bolt image information includes: determining the loosening angle of the nut surface and the loosening angle of the nut face of at least one bolt based on the second bolt image information; processing the second bolt image information by the target detection model to obtain the nut surface prediction box information and the nut face prediction box information of at least one bolt; determining the measured length of at least one bolt based on the nut surface prediction box information and the nut face prediction box information of at least one bolt, and determining the bolt length increment of at least one bolt based on the measured length and the preset length; determining the bolt pre-tightening force of at least one bolt based on the connector stiffness, the bolt stiffness, the loosening angle of the nut surface and the loosening angle of the nut face of at least one bolt; or determining the bolt pre-tightening force of at least one bolt based on the connector stiffness, the bolt stiffness, the loosening angle of the nut surface and the bolt length increment of at least one bolt. Among them, the calculation expression of the bolt pre-tightening force is: , , ; , ; Among them, is the bolt pre-tightening force in the current state, is the pre-tightening force at the initial state of bolt tightening, is the change amount of the bolt pre-tightening force, is the loosening angle of the nut face, is the loosening angle of the nut surface, is the bolt loosening angle increment, is the bolt length increment, is the bolt stiffness; is the connector stiffness, is the pitch of the bolt, is the initial length of the bolt in the tightened state, is the bolt length converted into the true value, is the calculated bolt length, is the conversion coefficient between the pixel length and the true length, is the error correction coefficient, is the friction coefficient, reflecting the inhibitory effect of friction on the change of pre-tightening force.

6. An electronic device, characterized in that, The electronic device includes: At least one processor; and a memory communicatively connected to the at least one processor; Wherein, the memory stores a computer program executable by the at least one processor, and the computer program is executed by the at least one processor so that the at least one processor can execute the method for determining the bolt state parameter according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions for causing a processor to execute the method for determining the bolt state parameter according to any one of claims 1 to 4 when executed.

8. A computer program product, comprising a computer program, characterized in that, The computer program, when executed by a processor, implements the method for determining the bolt state parameter according to any one of claims 1 to 4.

Images