Bolt state detection method and device

Abstract

The invention discloses a bolt state detection method and device, and the method comprises the steps: obtaining a target bolt image, and extracting a moving region, a fixed region and a marking line in the target bolt image; dividing the marking line into a first marking line and a second marking line based on the position relationship between the marking line and the moving area and the fixed area, the first marking line being located in the moving area, the second marking line being located in the fixed area, and the first marking line and the second marking line having a first identification feature; a first deflection angle corresponding to the first marking line and a second deflection angle corresponding to the second marking line are obtained, the degree of the first included angle is determined according to the first deflection angle and the second deflection angle, the state of the target bolt is judged according to the degree of the first included angle, and the state of the target bolt at least comprises the fastening state and the loosening state. According to the technical scheme provided by the invention, the fastening state of the bolt can be quickly monitored at low cost.

Description

Technical Field

[0001] This invention relates to the field of industrial testing technology, and in particular to a method and apparatus for detecting the condition of bolts. Background Technology

[0002] In fields such as wind power generation, rail transportation, and petrochemicals, bolts are critical connecting components in manufacturing, and their tightness directly affects the normal operation of equipment and production safety. For example, if the bolts on a wind turbine become loose, it may cause rotor blades to fall off or even the tower to collapse, resulting in serious property damage or casualties.

[0003] Currently, the detection of bolt tightness mainly relies on manual inspection, but manual inspection has a long inspection cycle, a high rate of missed detection, and is costly.

[0004] Therefore, it is necessary to provide a new method and apparatus for bolt condition detection to address the aforementioned shortcomings. Summary of the Invention

[0005] The purpose of this application is to provide a bolt condition detection method and apparatus that can quickly and cost-effectively monitor the tightness of bolts.

[0006] To achieve the above objectives, this application provides a bolt state detection method, comprising: acquiring a target bolt image and extracting a moving region, a fixed region, and a marking line from the target bolt image; dividing the marking line into a first marking line and a second marking line based on the positional relationship between the marking line and the moving region and the fixed region, wherein the first marking line is located in the moving region, the second marking line is located in the fixed region, and the first marking line and the second marking line have a first identification feature; acquiring a first deflection angle corresponding to the first marking line and a second deflection angle corresponding to the second marking line, and determining the degree of a first included angle based on the first deflection angle and the second deflection angle, so as to determine the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

[0007] To achieve the above objectives, this application also provides a bolt state detection device, comprising: an image acquisition module for acquiring a target bolt image and extracting a moving region, a fixed region, and a marking line from the target bolt image; a marking line division module for dividing the marking line into a first marking line and a second marking line based on the positional relationship between the marking line and the moving region and the fixed region, wherein the first marking line is located in the moving region, the second marking line is located in the fixed region, and the first marking line and the second marking line have a first identification feature; and a state judgment module for acquiring a first deflection angle corresponding to the first marking line and a second deflection angle corresponding to the second marking line, and determining the degree of a first included angle based on the first deflection angle and the second deflection angle, so as to judge the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

[0008] To achieve the above objectives, this application also provides a bolt condition detection device, wherein the data transmission device includes a memory and a processor, the memory being used to store a computer program, and when the computer program is executed by the processor, the above-described bolt condition detection method is implemented.

[0009] Therefore, the technical solution provided in this application first divides the marking lines in the target bolt image into a first marking line in the moving region and a second marking line in the fixed region using image semantic segmentation and image target detection technologies. Then, it determines whether the bolt is loose by calculating the relative deflection angle between the first and second marking lines. Compared with manual inspection, the solution of this application can greatly improve the detection efficiency. At the same time, this application directly obtains the relative displacement of the bolt and the connector by calculating the angle change between the marking lines in the moving and fixed regions, eliminating the interference of the overall rotation of the bolt and the connector. Furthermore, small deflections of the two marking lines will accumulate into larger angle changes. By calculating the angle between the first and second marking lines, this application can capture the subtle displacement of the bolt, resulting in higher sensitivity for detecting minor loosening. The solution of this application only needs to monitor the angle change between the marking lines in the moving and fixed regions, without relying on external coordinates, which can significantly reduce the deployment and maintenance costs of the system. Attached Figure Description

[0010] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0011] Figure 1 This is a flowchart of the bolt condition detection method in the embodiments of this application;

[0012] Figure 2 This is a data flow diagram of the bolt condition detection method in the embodiments of this application;

[0013] Figure 3 This is a schematic diagram of the functional modules of the bolt condition detection device in an embodiment of the present invention;

[0014] Figure 4 This is a schematic diagram of the bolt condition detection device in an embodiment of the present invention. Detailed Implementation

[0015] To make the objectives, technical solutions, and advantages of this application clearer, the embodiments of this application will be described in further detail below with reference to the accompanying drawings.

[0016] In industrial sectors such as wind power, rail transportation, and petrochemicals, bolts are critical mechanical connection components, and the reliability of their fastening directly affects the normal operation of equipment and production safety. Taking wind turbine generator sets as an example, the decay of the preload of the tower flange connection bolts can lead to abnormal dynamic response of the structure, and in severe cases, it can cause tower resonance or even overturning accidents. Loosening of bolts on rail transit bogies can significantly reduce the running stability of the vehicle, while failure of flange bolts on petrochemical pressure vessels can cause major safety accidents such as media leakage.

[0017] Currently, the industry commonly uses manual inspection to monitor the tightness of bolts. However, due to the limited inspection cycle, it is difficult to detect sudden loosening in a timely manner. Secondly, manual torque checks have significant measurement errors, and hazardous working environments such as high altitudes and confined spaces further reduce the reliability of the inspection. With technological advancements, image detection technology has been applied to monitor bolt tightness, which can solve the problems existing in manual inspections. However, current image detection technologies have poor anti-interference capabilities, require high-precision calibration, and have high deployment costs.

[0018] Therefore, how to improve bolt condition detection technology to quickly and cost-effectively monitor the tightness of bolts has become an urgent issue to be addressed in this field.

[0019] The specific application scenarios for this application are as follows:

[0020] At least one bolt is mounted on a bolt mounting platform. An image acquisition device, such as a camera, is positioned to face the bolt mounting platform to capture images of all bolts on it. It should be noted that the user will mark the bolts (e.g., red, blue, and black lines drawn with a marker pen) with marking lines. After acquiring images containing these marking lines, the image acquisition device can extract images of each bolt using image semantic segmentation and / or image object detection techniques for subsequent analysis. The bolt mounting platform refers to the surface of the equipment where the bolts are installed, including but not limited to the surface of the tower base of a wind turbine and the surface of the blade root flange.

[0021] The implementing entity of this application can be an electronic device, including servers, desktop computers, laptops, mobile phones, etc. This electronic device can determine the tightness of the bolt based on acquired image data. It should be noted that when the electronic device determines that the bolt is loose or in another loose state, it can send a prompt message to a terminal device connected to the electronic device, so that the user can tighten the bolt based on the prompt message. Optionally, the prompt message can be an audible and / or visual alarm message, so that the user can tighten the loose bolt after receiving the prompt message. The prompt message can also be a text and / or image alarm message, so that the user can tighten the loose bolt after receiving the information reminder through the terminal.

[0022] Please refer to the following: Figure 1 and Figure 2 , Figure 1 This is a flowchart of the bolt condition detection method in the embodiments of this application. Figure 2 This is a data flow diagram of the bolt condition detection method in the embodiments of this application.

[0023] S101: Obtain the target bolt image and extract the moving area, fixed area and marking line from the target bolt image.

[0024] In one embodiment, at least one bolt is mounted on the bolt mounting platform. The electronic device can acquire images of the bolt mounting platform to obtain pictures or videos containing the bolts. After acquiring the image information containing the bolts, the electronic device can use a target detection algorithm to extract images of each bolt for subsequent bolt tightening status analysis. Regarding the case where multiple bolts exist in a single image, this application uses one bolt as an example for illustration; for ease of description, this bolt is referred to as the target bolt.

[0025] After acquiring an image of the target bolt, the electronic device can use a semantic segmentation model to extract information such as moving regions, fixed regions, and marker lines from the image. Specifically, when the target bolt is a combination structure with a nut and screw separated, the moving region can be any surface on the nut; when the target bolt is a structure with a head and shank fixedly connected, the moving region can be any surface on the head. Fixed regions are surfaces whose position does not change with the bolt's tightness, such as the surface where the bolt is installed. Marker lines are colored lines manually drawn by the user using a marker pen.

[0026] It should be noted that various image processing methods can be used to extract marker lines, and the specific choice depends on the actual application scenario. For example, for marker lines with high color contrast, color space segmentation can be used for fast extraction; for straight marker lines, edge detection and line detection algorithms (such as Hough transform) can be combined for accurate localization; in complex scenarios, convolutional neural networks (CNNs) can be used to build an end-to-end segmentation model to directly extract marker line pixels.

[0027] Taking color space segmentation for marker line extraction as an example, in one feasible implementation, after the user draws the marker line using a high-contrast color (such as red / blue), the electronic device acquires an image containing the marker line through image acquisition. Then, the image is converted from the RGB (Red, Green, Blue) space to a more suitable color space for color segmentation, such as the HSV (Hue, Saturation, Value) space or the Lab color model. Thresholding segmentation is then used to generate a high-contrast binary image where the marker line is white and the background is black. Subsequently, the electronic device performs morphological processing on the obtained binary image. First, it expands the white area using structuring elements to bridge broken marker lines, and then shrinks the white area to eliminate minor noise, thereby enhancing the continuity of the marker line. Finally, contour detection is used to filter regions that match the characteristics of the marker line based on geometric features such as area and aspect ratio (e.g., elongated stripes), thus completing the accurate extraction of the marker line.

[0028] S102: Based on the positional relationship between the marking line and the moving area and the fixed area, the marking line is divided into a first marking line and a second marking line, wherein the first marking line is located in the moving area, the second marking line is located in the fixed area, and the first marking line and the second marking line have a first identification feature.

[0029] In one embodiment, after acquiring information such as the moving area, fixed area, and marking lines in the target bolt image, the electronic device can further identify the positional relationship between the marking lines and the moving area, as well as between the marking lines and the fixed area, and divide the marking lines into a first marking line and a second marking line based on the identification results. For example, for a single marking line, the electronic device can mark the segment of the marking line located in the moving area as the first marking line, and the segment of the marking line located in the fixed area as the second marking line. Since both the first and second marking lines are located on the same marking line, they have the same color and other identifying features (i.e., the first identifying feature).

[0030] In practical applications, marking lines can be hand-drawn by the user, such as after bolts are tightened, the user draws a line using a marker pen, the line passing through the moving and fixed areas. Users can also utilize a combination of drawn marking lines and existing markings. For example, after bolts are tightened, the user selects an existing marking on either side of the moving or fixed area and hand-draws a marking line along the same or parallel direction as the existing marking. The existing markings can be the inherent geometry of the bolt, such as the straight lines at the edges of a hexagonal bolt; they can also be the inherent geometry of the bolt mounting platform, such as the edge line adjacent to the bolt on the bolt mounting platform; or they can be long-term marking lines marked with a special color on the bolt or bolt mounting platform.

[0031] S103: Obtain the first deflection angle corresponding to the first marking line and the second deflection angle corresponding to the second marking line, and determine the degree of the first included angle based on the first deflection angle and the second deflection angle, so as to determine the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

[0032] Specifically, since the first and second marking lines are drawn when the bolt is in a tightened state, if the electronic device detects that the first and second marking lines are always on the same extension line, it indicates that the bolt has not rotated after tightening, thus indicating that the bolt is still in a tightened state and meets the normal working conditions of the bolt. If the electronic device detects that the extension lines of the first and second marking lines intersect, it indicates that the bolt has rotated after tightening. In this case, the electronic device can further analyze the rotation angle of the bolt to determine whether the bolt is still in a tightened state.

[0033] In one embodiment, the electronic device can perform numerical processing on the first and second marking lines to obtain a first deflection angle corresponding to the first marking line and a second deflection angle corresponding to the second marking line. Then, it determines the degree of the first included angle based on the first and second deflection angles. In practical applications, if the extensions of the first and second marking lines intersect, the angle formed by the two extensions radiating outwards from the intersection point is the first included angle. The degree of the first included angle can be understood as the difference between the first and second deflection angles. This first included angle is mainly formed by the first marking line on the moving area as the bolt rotates. Therefore, the degree of the first included angle can indicate the bolt's rotation angle, and thus the bolt's tightening state. Specifically, the user can preset a bolt rotation angle threshold (denoted as the first threshold) based on empirical values ​​or the torque angle method, and then compare the degree of the first included angle with the first threshold. If the degree of the first included angle is greater than or equal to the first threshold, it indicates that the bolt's rotation angle is large, and the bolt is in a loose state. If the degree of the first included angle is less than the first threshold mentioned above, it means that the rotation angle of the bolt is within the controllable range and the bolt is still in a tightened state.

[0034] In practical applications, different equipment has different requirements for the tightness of bolt connections. For example, some equipment requires bolts to be fully tightened to operate normally, while others only require the bolts to still function as a connector between two parts. Therefore, there is a certain tolerance range for determining whether a bolt is tight or loose. Since the degree of the first included angle reflects the rotation angle of the bolt after tightening, the degree of the first included angle can be set as a range that satisfies the normal operation of the equipment. The upper limit of this range is set as the first threshold. When the degree of the first included angle does not exceed the first threshold, the target bolt is considered to be tight, and when the degree of the first included angle exceeds the first threshold, the target bolt is considered to be loose. For example, if the first threshold is 5°, then when the degree of the first included angle is in the range of 0°-5°, the target bolt can be considered to be tight, and when the degree of the first included angle exceeds 5°, the target bolt can be determined to be loose.

[0035] In one embodiment, obtaining the first deflection angle corresponding to the first marker line and the second deflection angle corresponding to the second marker line can be achieved through the following steps:

[0036] First, obtain the first set of coordinates corresponding to the first marker line and the second set of coordinates corresponding to the second marker line;

[0037] Then, the first marker line function is determined based on the first coordinate set, and the second marker line function is determined based on the second coordinate set;

[0038] Finally, the first deflection angle is determined based on the first marker line function, and the second deflection angle is determined based on the second marker line function.

[0039] Taking the first marker line as an example, when obtaining the first coordinate set corresponding to the first marker line, the electronic device can input the image region containing the first marker line into a pre-trained semantic segmentation model to output a corresponding binary mask. In practical applications, due to external factors such as reflection, the marker line mask output by the semantic segmentation model is usually discontinuous. To improve the mask quality, the electronic device can perform dilation and erosion operations on the binary mask. Specifically, the electronic device can perform directional dilation on the binary mask corresponding to the first marker line along the extension direction of the first marker line using a thin kernel to bridge the breakpoints in the mask, and then perform erosion to remove isolated noise points in the mask. Afterwards, the electronic device can simplify the target object in the binary mask into a single-pixel-width line using algorithms such as median transformation or iterative thinning. Finally, all points on the single-pixel-width line are converted into coordinate data to obtain the first coordinate set corresponding to the first marker line.

[0040] The method for obtaining the second coordinate set corresponding to the second marker line is the same as the method for obtaining the first coordinate set, and will not be repeated here.

[0041] Since the processed lines with a single pixel width may contain discrete points, to reduce the adverse impact of these discrete points on data accuracy, after obtaining the first and second coordinate sets, an initial fitting can be performed on the first and second coordinate sets to obtain the first initial marker line function corresponding to the first marker line and the second initial marker line function corresponding to the second marker line. Then, based on the aforementioned first and second initial marker line functions, discrete points in the first and second coordinate sets are removed, and a second fitting is performed on the first and second coordinate sets after removing all discrete points to obtain more accurate first and second marker line functions.

[0042] Specifically, taking the first marker line as an example, the first coordinate set is an array of point coordinates, which can be represented as [n, w], where n is the number of points and w is the x and y coordinates of the points.

[0043] When initially fitting the first coordinate set, firstly, the first coordinate set is fitted using the least squares method to obtain the specific values ​​of a and b in the function y = ax + b. y = ax + b is the first initial marker line function. Then, based on the specific values ​​of a and b, the y value corresponding to xi is calculated using the first initial marker line function y = ax + b and the value of xi (where i represents any point in the first coordinate set). The y value is then compared with the yi value. If the y value and the yi value differ significantly, the point corresponding to (xi, yi) in the first coordinate set is deleted. For example, for point A(x2, y2), if the y value of x2 calculated using the first initial marker line function y = ax + b is 10, while the value of y2 is 2, then point A(x2, y2) is a discrete point, and point A can be removed from the first coordinate set. Through the above operations, a new first coordinate set after removing all discrete points can be obtained. Then, by performing least squares fitting on the new first coordinate set again, a new function y = cx + d can be obtained, which is the first mark line function corresponding to the first mark line.

[0044] The second marker line function is generated in the same way as the first marker line function, so it will not be described again here.

[0045] It should be noted that after obtaining the first and second marker line functions, the slope values ​​of the first and second marker line functions are calculated using the arctangent function, and then the results are converted into angles to obtain the first deflection angle corresponding to the first marker line function and the second deflection angle corresponding to the second marker line function.

[0046] In practical applications, if the system does not monitor the bolt tightness for an extended period, it may miss the critical transition point from a tight to a loose state. For example, when the marker line deflects by 5° (tight) and 185° (severely loose), the calculated slope is the same. The system may misjudge severe looseness as normal tightness, leading to significant safety hazards. Therefore, simply calculating the deflection angle based on the slope may not be effective in distinguishing the actual state of the bolt.

[0047] To address the aforementioned issues, in one implementation, the first and second deflection angles can be further determined using a quadrant approach. Specifically, the system can average the coordinates of all pixels within the moving area and use this average as the coordinates of the center point of the moving area. A reference coordinate system is then established with this center point as the origin. Next, the system can use this center point as a reference to calculate the Euclidean distance between it and all points in the first coordinate set (after removing all discrete points), and determine the coordinate point with the largest distance. This coordinate point is the point on the first marking line farthest from the center point (referred to as the first point for ease of description). Similarly, the system can use this center point as a reference to calculate the Euclidean distance between it and all points in the second coordinate set (after removing all discrete points), and determine the coordinate point with the largest distance. This coordinate point is the point on the second marking line farthest from the center point (referred to as the second point for ease of description). The system can then place the coordinates of the first and second points into the reference coordinate system to determine their quadrant positions within the reference coordinate system, and ultimately determine the first and second deflection angles. For example, if the first point is located in the first quadrant of the reference coordinate system, it means that the first deflection angle is an acute angle; if the first point is located in the third quadrant of the reference coordinate system, it means that the first deflection angle is an obtuse angle. Combining the slope calculation results recorded earlier, the system can calculate the true value of the first deflection angle.

[0048] In practical applications, the preload of bolts will decrease after prolonged exposure to vibration, alternating loads, or temperature cycling. To ensure the robustness of bolted connections, secondary torque maintenance is often required. To ensure that secondary torque maintenance meets operational specifications, users typically use multi-color marking lines to verify the condition of the bolts.

[0049] The following example, using a bolt with multicolored marking lines, illustrates how to detect whether a bolt is loose.

[0050] In one embodiment, after extracting information such as the moving area, fixed area, and marking lines from the target bolt image, the electronic device can further process the obtained marking lines using algorithms such as color space conversion and segmentation or deep learning fusion to determine whether there are other marking lines characterized by a second identification feature (for ease of description, this application refers to marking lines characterized by a second identification feature as re-tightening marking lines). For example, the electronic device can mark all red marking lines as first-type marking lines, all blue marking lines as second-type marking lines, all black marking lines as third-type marking lines, and so on.

[0051] If the electronic device determines that a re-tightening mark line exists, it can divide the re-tightening mark line into a third mark line and a fourth mark line based on the positional relationship between the re-tightening mark line and the moving area and the fixed area. The third mark line is located in the moving area, and the fourth mark line is located in the fixed area. The method for obtaining the third mark line in the moving area and the fourth mark line in the fixed area is similar to the method for obtaining the first mark line in the moving area and the second mark line in the fixed area in the aforementioned embodiment, and will not be repeated here.

[0052] It should be noted that the second identification feature differs from the first identification feature. For example, if the first identification feature is red, the second identification feature can be blue, black, or white; if the first identification feature is a solid line segment, the second identification feature can be a dashed line segment. This application does not limit the specific presentation of the first and second identification features.

[0053] After dividing the re-tightening mark line into a third mark line and a fourth mark line, the degree of the second included angle can be determined based on the third mark line and the fourth mark line. The method for determining the degree of the second included angle is similar to the method for determining the degree of the first included angle in the aforementioned embodiment, and will not be repeated here.

[0054] Once the electronic device obtains the degree of the second included angle, it can determine the state of the target bolt based on the degrees of the first and second included angles. Specifically, the user can preset an angle threshold (denoted as the second threshold) for bolt secondary torque maintenance based on experience or the torque-angle method. If the degree of the second included angle is greater than or equal to the second threshold, the target bolt can be determined to be in a tightened state. If the degree of the second included angle is less than the second threshold, and the degree of the first included angle is less than the first threshold, the target bolt can be determined to have not completed the re-tightening operation. If the degree of the second included angle is less than the second threshold, and the degree of the first included angle is greater than or equal to the first threshold, the target bolt can be determined to be in a loose state.

[0055] In practical applications, if a bolt undergoes secondary torque maintenance, two included angles will exist (i.e., the first included angle and the second included angle). The marking line forming the first included angle is marked before the re-tightening marking line is drawn; that is, the marking line corresponding to the first included angle is marked during the first torque maintenance operation, and the re-tightening marking line corresponding to the second included angle is marked during the second torque maintenance operation. During the second torque maintenance operation, the degree of the first included angle usually increases. Therefore, when judging the tightness of the bolt, it is usually only necessary to determine that the degree of the second included angle meets the tightness condition. If the degree of the first included angle meets the tightness condition, but the degree of the second included angle does not, it indicates that the second torque maintenance operation was missing or the torque was insufficient.

[0056] For example, suppose the first threshold is D1, the second threshold is D2, the first included angle is d1, and the second included angle is d2. If the system detects that d2 ≥ D2, it can determine that the target bolt is in a tightened state; if the system detects that d2 < D2 and d1 < D1, it can determine that the first torque maintenance operation of the target bolt is up to standard, but the second torque maintenance operation is not up to standard, that is, the target bolt has not completed the re-tightening operation; if the system detects that d2 < D2 and d1 ≥ D1, it can determine that the target bolt is in a loose state.

[0057] This implementation method simultaneously monitors bolt tightness and accurately records torque maintenance operations by deploying two sets of visually distinguishable marking lines. The first set of marking lines records the initial tightness, while the second set verifies the completion of secondary torque maintenance, forming an operational closed loop. The judgment model built based on the deflection angles of the two sets of marking lines can not only identify obvious loosening but also effectively monitor the process compliance of secondary torque maintenance, thereby significantly improving the reliability and maintenance standardization of the bolted connection system.

[0058] It should be understood that, in addition to using two included angles to determine the state of the target bolt in the above embodiments, three or even more included angles can also be used to determine the state of the target bolt. The methods used are similar to those in the above embodiments, and will not be listed one by one in this application.

[0059] Please see Figure 3 This application also provides a bolt condition detection device, the bolt condition detection device comprising:

[0060] The image acquisition module is used to acquire an image of the target bolt and extract the moving area, fixed area, and marking lines from the target bolt image.

[0061] The marking line division module is used to divide the marking line into a first marking line and a second marking line based on the positional relationship between the marking line and the moving area and the fixed area, wherein the first marking line is located in the moving area, the second marking line is located in the fixed area, and the first marking line and the second marking line have a first identification feature;

[0062] The state determination module is used to obtain the first deflection angle corresponding to the first marking line and the second deflection angle corresponding to the second marking line, and determine the degree of the first included angle based on the first deflection angle and the second deflection angle, so as to determine the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

[0063] In one implementation, the state determination module obtains the first deflection angle corresponding to the first marker line and the second deflection angle corresponding to the second marker line through the following operations:

[0064] Obtain the first set of coordinates corresponding to the first marker line and the second set of coordinates corresponding to the second marker line;

[0065] The first marker line function is determined based on the first coordinate set, and the second marker line function is determined based on the second coordinate set;

[0066] The first deflection angle is determined based on the first marker line function, and the second deflection angle is determined based on the second marker line function.

[0067] In one implementation, the state determination module determines the first marker line function and the second marker line function through the following operations:

[0068] Initial fitting is performed on the first coordinate set and the second coordinate set to obtain the first initial mark line function corresponding to the first mark line and the second initial mark line function corresponding to the second mark line;

[0069] Based on the first initial marker line function and the second initial marker line function, discrete points in the first coordinate set and the second coordinate set are removed, and a second fitting is performed on the first coordinate set and the second coordinate set after removing the discrete points to obtain the first marker line function and the second marker line function.

[0070] In one implementation, the state determination module determines the first deflection angle and the second deflection angle through the following operations:

[0071] Obtain the coordinates of the center point of the moving area, and establish a reference coordinate system with the center point as the origin;

[0072] Based on the first set of coordinates and the second set of coordinates after removing the discrete points, determine the first point on the first marking line that is farthest from the center point, and the second point on the second marking line that is farthest from the center point;

[0073] Based on the coordinates of the first point and the second point, determine the quadrant positions of the first point and the second point in the reference coordinate system, and thus determine the first deflection angle and the second deflection angle.

[0074] In one embodiment, the marking line division module is further configured to determine whether there is a re-tightening marking line characterized by a second identification feature among the marking lines. If there is, the re-tightening marking line is divided into a third marking line and a fourth marking line based on the positional relationship between the re-tightening marking line and the moving area and the fixed area, wherein the third marking line is located in the moving area and the fourth marking line is located in the fixed area.

[0075] The state determination module is further configured to determine the degree of the second included angle based on the third and fourth marking lines, and to determine the state of the target bolt based on the degree of the first included angle and the degree of the second included angle.

[0076] Please see Figure 4 This application also provides a bolt condition detection device, which includes a memory and a processor. The memory stores a computer program, and when the computer program is executed by the processor, the bolt condition detection method described above can be implemented. Specifically, at the hardware level, the data transmission device may include a processor, an internal bus, and a memory. The memory may include main memory and non-volatile memory. The processor reads the corresponding computer program from the non-volatile memory into main memory and then runs it. Those skilled in the art will understand that... Figure 4 The structure shown is for illustrative purposes only and does not limit the structure of the bolt condition detection device described above. For example, the bolt condition detection device may also include a... Figure 4 The components shown may include more or fewer components, such as other processing hardware like a GPU (Graphics Processing Unit) or external communication ports. Of course, this application does not exclude other implementation methods besides software implementations, such as logic devices or a combination of hardware and software.

[0077] In this embodiment, the processor may include a central processing unit (CPU) or a graphics processing unit (GPU), and may also include other microcontrollers, logic gates, integrated circuits, or appropriate combinations thereof with logic processing capabilities. The memory described in this embodiment can be a storage device for storing information. In digital systems, a device capable of storing binary data can be a memory; in integrated circuits, a circuit without physical form but with storage function can also be a memory, such as RAM or FIFO; in a system, a storage device with physical form can also be called a memory. In implementation, this memory can also be implemented using a cloud storage method; the specific implementation method is not limited in this specification.

[0078] It should be noted that the specific implementation method of the bolt condition detection device in this specification can be referred to the description of the method implementation method, and will not be repeated here.

[0079] Therefore, the technical solution provided in this application first divides the marking lines in the target bolt image into a first marking line in the moving region and a second marking line in the fixed region using image semantic segmentation and image target detection technologies. Then, it determines whether the bolt is loose by calculating the relative deflection angle between the first and second marking lines. Compared with manual inspection, the solution of this application can greatly improve the detection efficiency. At the same time, this application directly obtains the relative displacement of the bolt and the connector by calculating the angle change between the marking lines in the moving and fixed regions, eliminating the interference of the overall rotation of the bolt and the connector. Furthermore, small deflections of the two marking lines will accumulate into larger angle changes. By calculating the angle between the first and second marking lines, this application can capture the subtle displacement of the bolt, resulting in higher sensitivity for detecting minor loosening. The solution of this application only needs to monitor the angle change between the marking lines in the moving and fixed regions, without relying on external coordinates, which can significantly reduce the deployment and maintenance costs of the system.

[0080] Simultaneously, this application utilizes two sets of visually distinguishable marking lines to simultaneously monitor bolt tightness and accurately record torque maintenance operations. The first set of marking lines records the initial tightness, while the second set verifies the completion of secondary torque maintenance, forming an operational closed loop. The judgment model built upon the deflection angles of the two sets of marking lines can not only identify obvious loosening but also effectively monitor the process compliance of secondary torque maintenance, thereby significantly improving the reliability and maintenance standardization of the bolted connection system.

[0081] Through the above description of the embodiments, those skilled in the art can clearly understand that each embodiment can be implemented by means of software plus necessary general-purpose hardware platforms, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solutions, in essence or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes several instructions to cause a computer device (which may be a personal computer, server, or network device, etc.) to execute the methods described in the various embodiments or some parts of the embodiments.

[0082] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A method for detecting the condition of bolts, characterized in that, include: Acquire an image of the target bolt, and extract the moving area, fixed area, and marker lines from the target bolt image; Based on the positional relationship between the marking line and the moving area and the fixed area, the marking line is divided into a first marking line and a second marking line, wherein the first marking line is located in the moving area, the second marking line is located in the fixed area, and the first marking line and the second marking line have a first identification feature; Obtain the first deflection angle corresponding to the first marking line and the second deflection angle corresponding to the second marking line, and determine the degree of the first included angle based on the first deflection angle and the second deflection angle, so as to determine the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

2. The method according to claim 1, characterized in that, Obtaining the first deflection angle corresponding to the first marker line and the second deflection angle corresponding to the second marker line includes: Obtain the first set of coordinates corresponding to the first marker line and the second set of coordinates corresponding to the second marker line; The first marker line function is determined based on the first coordinate set, and the second marker line function is determined based on the second coordinate set; The first deflection angle is determined based on the first marker line function, and the second deflection angle is determined based on the second marker line function.

3. The method according to claim 2, characterized in that, Determining a first marker line function based on the first coordinate set and a second marker line function based on the second coordinate set includes: Initial fitting is performed on the first coordinate set and the second coordinate set to obtain the first initial mark line function corresponding to the first mark line and the second initial mark line function corresponding to the second mark line; Based on the first initial marker line function and the second initial marker line function, discrete points in the first coordinate set and the second coordinate set are removed, and a second fitting is performed on the first coordinate set and the second coordinate set after removing the discrete points to obtain the first marker line function and the second marker line function.

4. The method according to claim 3, characterized in that, Determining the first deflection angle based on the first marker line function, and determining the second deflection angle based on the second marker line function, includes: Obtain the coordinates of the center point of the moving area, and establish a reference coordinate system with the center point as the origin; Based on the first set of coordinates and the second set of coordinates after removing the discrete points, determine the first point on the first marking line that is farthest from the center point, and the second point on the second marking line that is farthest from the center point; Based on the coordinates of the first point and the second point, determine the quadrant positions of the first point and the second point in the reference coordinate system, and thus determine the first deflection angle and the second deflection angle.

5. The method according to claim 4, characterized in that, Determining the state of the target bolt based on the degree of the first included angle includes: Based on a preset first threshold, it is determined whether the degree of the first included angle is greater than or equal to the first threshold. If the degree of the first included angle is greater than or equal to the first threshold, the target bolt is in a loose state. If the degree of the first included angle is less than the first threshold, the target bolt is in a tight state.

6. The method according to claim 5, characterized in that, The method further includes: Determine whether there is a re-tightening mark line characterized by a second identification feature among the mark lines. If there is, then based on the positional relationship between the re-tightening mark line and the moving area and the fixed area, divide the re-tightening mark line into a third mark line and a fourth mark line, wherein the third mark line is located in the moving area and the fourth mark line is located in the fixed area. The degree of the second included angle is determined based on the third and fourth marking lines, and the state of the target bolt is determined based on the degree of the first included angle and the degree of the second included angle.

7. The method according to claim 6, characterized in that, Determining the state of the target bolt based on the degree values ​​of the first included angle and the second included angle includes: If the degree of the second included angle is greater than or equal to the second threshold, then the target bolt is in the tightened state; If the degree of the second included angle is less than the second threshold, and the degree of the first included angle is less than the first threshold, then the target bolt has not completed the re-tightening operation; If the degree of the second included angle is less than the second threshold, and the degree of the first included angle is greater than or equal to the first threshold, then the target bolt is in the loose state.

8. A bolt condition detection device, characterized in that, The bolt condition detection device includes: The image acquisition module is used to acquire an image of the target bolt and extract the moving area, fixed area, and marking lines from the target bolt image. The marking line division module is used to divide the marking line into a first marking line and a second marking line based on the positional relationship between the marking line and the moving area and the fixed area, wherein the first marking line is located in the moving area, the second marking line is located in the fixed area, and the first marking line and the second marking line have a first identification feature; The state determination module is used to obtain the first deflection angle corresponding to the first marking line and the second deflection angle corresponding to the second marking line, and determine the degree of the first included angle based on the first deflection angle and the second deflection angle, so as to determine the state of the target bolt based on the degree of the first included angle, wherein the state of the target bolt includes at least a tight state and a loose state.

9. The bolt condition detection device according to claim 8, characterized in that, The state determination module obtains the first deflection angle corresponding to the first marker line and the second deflection angle corresponding to the second marker line through the following operations: Obtain the first set of coordinates corresponding to the first marker line and the second set of coordinates corresponding to the second marker line; The first marker line function is determined based on the first coordinate set, and the second marker line function is determined based on the second coordinate set; The first deflection angle is determined based on the first marker line function, and the second deflection angle is determined based on the second marker line function.

10. The bolt condition detection device according to claim 9, characterized in that, The state determination module determines the first marker line function and the second marker line function through the following operations: Initial fitting is performed on the first coordinate set and the second coordinate set to obtain the first initial mark line function corresponding to the first mark line and the second initial mark line function corresponding to the second mark line; Based on the first initial marker line function and the second initial marker line function, discrete points in the first coordinate set and the second coordinate set are removed, and a second fitting is performed on the first coordinate set and the second coordinate set after removing the discrete points to obtain the first marker line function and the second marker line function.

11. The bolt condition detection device according to claim 10, characterized in that, The status determination module determines the first deflection angle and the second deflection angle through the following operations: Obtain the coordinates of the center point of the moving area, and establish a reference coordinate system with the center point as the origin; Based on the first set of coordinates and the second set of coordinates after removing the discrete points, determine the first point on the first marking line that is farthest from the center point, and the second point on the second marking line that is farthest from the center point; Based on the coordinates of the first point and the second point, determine the quadrant positions of the first point and the second point in the reference coordinate system, and thus determine the first deflection angle and the second deflection angle.

12. The bolt condition detection device according to claim 8, characterized in that, The marking line division module is further used to determine whether there is a re-tightening marking line characterized by a second identification feature in the marking line. If there is, the re-tightening marking line is divided into a third marking line and a fourth marking line based on the positional relationship between the re-tightening marking line and the moving area and the fixed area. The third marking line is located in the moving area, and the fourth marking line is located in the fixed area. The state determination module is further configured to determine the degree of the second included angle based on the third and fourth marking lines, and to determine the state of the target bolt based on the degree of the first included angle and the degree of the second included angle.

13. A bolt condition detection device, characterized in that, The bolt condition detection device includes a memory and a processor. The memory stores a computer program, which, when executed by the processor, implements the method as described in any one of claims 1 to 7.

Images