Bolt fastening state detection method and electronic device

Abstract

The application discloses a bolt fixing state detection method and electronic equipment. The method comprises the following steps: collecting regional images of a target bolt at multiple time points under the condition that the target bolt is in a vibration state, determining the earliest collection time as an initial image, and determining all other images as target images; segmenting the regional images into foreground images and background images, determining a plurality of target feature points, and determining a plurality of target reference points; determining reference displacement values of the target reference points, determining reference displacement mean values according to the reference displacement values of the plurality of target images, and determining a plurality of feature point displacement values respectively; determining feature point displacement totals according to the feature point displacement values and the reference displacement mean values in each target image, determining a target displacement sequence, determining spectrum information according to the target displacement sequence, determining a fixing state of the target bolt according to the spectrum information, realizing accurate separation and measurement of bolt displacement in a vibration environment, and effectively improving the reliability of loose fastening state discrimination.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a method for detecting the fixed state of bolts and an electronic device. Background Technology

[0002] With the development of machine vision and image processing technology, bolt state recognition based on image analysis has become a research hotspot. However, image stability, feature point reliability, and displacement calculation accuracy under vibration environment remain challenges.

[0003] In related technologies, single-frame image feature matching or global displacement statistics are often used to determine the bolt status. Typically, feature points are extracted directly from the original image and displacement is calculated without optimizing image preprocessing for the vibration environment or separating background displacement from bolt displacement. This results in image blurring or background interference caused by vibration, and feature point extraction is easily affected by noise. Furthermore, the interference of overall background displacement on feature point displacement is not considered, which can easily lead to misjudging background vibration as bolt loosening. The lack of time-series spectrum analysis makes it difficult to distinguish loosening types (such as gradual or sudden), resulting in low accuracy and poor robustness. Summary of the Invention

[0004] This invention provides a bolt fixing state detection method and electronic device to solve the problems in related technologies, such as easy misjudgment of bolt loosening in vibration environment, difficulty in separating background and bolt displacement, and inability to accurately determine fixing state through time series spectrum.

[0005] According to one aspect of the present invention, a method for detecting bolt fastening status is provided, comprising: When the target bolt is under vibration, regional images of the installation area of ​​the target bolt are acquired at multiple times. The regional image acquired earliest is determined as the initial image, and the regional images other than the initial image are determined as the target images. For each of the aforementioned region images, the region image is segmented into a foreground image and a background image. Multiple target feature points are determined in the foreground image, and multiple target reference points are determined in the background image. The foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located. Determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction; determine the average reference displacement value based on the reference displacement values ​​of multiple target images; and determine the multiple feature point displacement values ​​of multiple target feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction. The total displacement of the feature points is determined based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement. The target displacement sequence is determined based on the total displacement of the feature points in each target image. The spectral information is determined based on the target displacement sequence. The fixing state of the target bolt is determined based on the spectral information, wherein the fixing state includes a loose state and a tight state.

[0006] According to another aspect of the present invention, a method for detecting bolt fastening status is provided, comprising: The image determination module is used to acquire regional images of the installation area of ​​the target bolt at multiple times when the target bolt is in a state of vibration, and to determine the regional image acquired earliest as the initial image, and the regional images other than the initial image as the target image. The point determination module is used to segment each region image into a foreground image and a background image, determine multiple target feature points in the foreground image, and determine multiple target reference points in the background image, wherein the foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located; The displacement value determination module is used to determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction, determine the average reference displacement value based on the reference displacement values ​​of multiple target images, and determine the displacement values ​​of multiple feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction. A fixed state determination module is used to determine the total displacement of feature points based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement, determine the target displacement sequence based on the total displacement of feature points in each target image, determine the spectral information based on the target displacement sequence, and determine the fixed state of the target bolt based on the spectral information, wherein the fixed state includes a loose state and a tight state.

[0007] According to another aspect of the present invention, an electronic device is provided, the electronic device comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, which enables the at least one processor to perform the bolt fixing state detection method according to any embodiment of the present invention.

[0008] According to another aspect of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the bolt fixing state detection method according to any embodiment of the present invention.

[0009] The technical solution of this invention involves acquiring regional images of the installation area of ​​the target bolt at multiple times while the target bolt is vibrating. The earliest acquired regional image is determined as the initial image, and the regional images other than the initial image are determined as target images. Acquiring images at multiple times during vibration, with the earliest acquisition time as the benchmark, eliminates accumulated errors and improves displacement measurement accuracy. For each regional image, it is segmented into a foreground image and a background image. Multiple target feature points are determined in the foreground image, and multiple target reference points are determined in the background image. The foreground image includes the image area where the target bolt is located, and the background image includes the image area excluding the image area where the target bolt is located. Segmenting the foreground and background and extracting feature points and reference points respectively accurately separates bolt displacement interference from the background. The reference displacement value of the target reference point in each target image and the corresponding target reference point in the initial image in a preset direction is determined. Based on the reference displacement values ​​of multiple target images... The method involves determining the reference displacement mean, and separately determining the displacement values ​​of multiple target feature points in each target image and the corresponding target feature points in the initial image in a preset direction. This allows for the calculation of the reference point mean and extraction of feature point displacements, effectively eliminating background interference to obtain the true bolt displacement. The method also involves determining the total feature point displacement based on the multiple feature point displacement values ​​in each target image and the reference displacement mean, determining the target displacement sequence based on the total feature point displacement in each target image, determining the spectral information based on the target displacement sequence, and determining the fixed state of the target bolt based on the spectral information. The fixed state includes both loose and tight states. The method integrates features and reference displacements to obtain the total displacement and constructs a time series. Spectral analysis accurately determines the bolt's looseness or tightness, solving the problems in related technologies where bolt loosening in vibration environments is easily misjudged, background and bolt displacement are difficult to separate, and the fixed state cannot be accurately determined through time-series spectral analysis. This method achieves accurate separation and measurement of bolt displacement in vibration environments, constructs a time-series spectrum, and effectively improves the reliability of loosening and tightening state discrimination.

[0010] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of a bolt fixing status detection method provided in Embodiment 1 of the present invention; Figure 2 This is a flowchart of a bolt fixing status detection method according to Embodiment 2 of the present invention; Figure 3 This is a flowchart of a bolt fixing status detection method provided in Embodiment 3 of the present invention; Figure 4 This is a schematic diagram of a bolt fixing status detection device according to Embodiment 4 of the present invention; Figure 5 This is a schematic diagram of the adaptive region motion amplification principle provided in an embodiment of the present invention; Figure 6 This is a schematic diagram of the coupled differential optical flow tracing principle provided in an embodiment of the present invention; Figure 7 This is a comparison diagram of displacement tracking effects under loose and tight bolt conditions according to an embodiment of the present invention; Figure 8 This is a schematic diagram of the structure of an electronic device that implements the bolt fixing state detection method of the present invention. Detailed Implementation

[0013] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0014] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover a non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0015] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0016] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0017] It is understood that before using the technical solutions disclosed in the various embodiments of this disclosure, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in this disclosure in an appropriate manner in accordance with relevant laws and regulations, and user authorization should be obtained.

[0018] For example, upon receiving a user's active request, a prompt message is sent to the user to explicitly inform them that the requested operation will require the acquisition and use of the user's personal information. This allows the user to independently choose whether to provide personal information to the software or hardware, such as the electronic device, application, server, or storage medium performing the operations of this disclosed technical solution, based on the prompt message.

[0019] As an optional but non-limiting implementation, in response to a user's active request, sending a prompt message to the user can be done via a pop-up window, where the prompt message can be presented in text format. Furthermore, the pop-up window can also include a selection control allowing the user to choose "agree" or "disagree" to provide personal information to the electronic device.

[0020] It is understood that the above notification and user authorization process are merely illustrative and do not constitute a limitation on the implementation of this disclosure. Other methods that comply with relevant laws and regulations may also be applied to the implementation of this disclosure.

[0021] It is understood that the data involved in this technical solution (including but not limited to the data itself, the acquisition or use of the data) shall comply with the requirements of relevant laws, regulations and related provisions.

[0022] Example 1 Figure 1 The flowchart of a bolt fixing state detection method is provided in Embodiment 1 of the present invention. This embodiment is applicable to industrial equipment in a vibration environment that requires non-contact and accurate monitoring of bolt tightness. The method can be executed by a bolt fixing state detection device, which can be implemented in hardware and / or software. Optionally, it can be implemented through electronic devices, such as mobile terminals, PCs, or servers.

[0023] like Figure 1 As shown, the method may specifically include: S110. When the target bolt is under vibration, regional images of the installation area of ​​the target bolt are acquired at multiple times. The regional image acquired earliest is determined as the initial image, and the regional images other than the initial image are determined as the target images.

[0024] The target bolt can be understood as a bolt in a fixed state to be detected, located in a vibrating environment (such as mechanical equipment, bridge structures, etc., which can be vibrated by applying external force). Image analysis is needed to determine whether it is loose or remains tight. As the object to be detected, its fixed state (loose / tight) is the core indicator of concern. By tracking the image changes of its installation area, the displacement or deformation of the bolt caused by vibration can be indirectly reflected, thereby determining whether it is loose. The vibration state can be understood as the environmental state of the target bolt, referring to the periodic or non-periodic reciprocating motion of the bolt caused by external excitation (such as mechanical impact on the structure). Vibration is the main cause of bolt loosening. Acquiring images under vibration can capture the minute displacements of the bolt caused by loosening or the relative changes between the background and foreground, providing a dynamic data basis for subsequent analysis. The area image can be understood as an image containing the installation area of ​​the target bolt acquired at multiple times, covering the complete visual range of the bolt and its surrounding background. As the raw data carrier, it records the visual information of the bolt installation area at different time points. By comparing the area images at different times, the displacement of the bolt relative to points in the background can be extracted, thereby analyzing the fixed state. The initial image can be understood as the earliest acquired regional image, serving as a benchmark for subsequent analysis and providing a visual reference for the initial state. The displacements of feature points and reference points in all subsequent target images are compared to this initial image to ensure the relativity and consistency of displacement calculations. The target image can be understood as any regional image acquired at times other than the initial image, used for comparative analysis of displacement changes with the initial image.

[0025] In one optional implementation, the test object was a high-strength connecting bolt of a standard section of a tower crane at a construction site. A full-frame mirrorless camera equipped with a 24-70mm zoom lens was used. The camera was fixed on a stable tripod to ensure no movement throughout the acquisition process. The camera was set to 1080p resolution (1920×1080 pixels), a frame rate of 50 fps, a shutter speed of 1 / 500 second to ensure image clarity, and automatic ISO. The excitation method was manual, using a rubber mallet to apply instantaneous hammering excitation to the standard section of the tower crane, stimulating the bolt's vibration response. The same bolt was tightened to 300 N·m and 50 N·m respectively for comparative experiments.

[0026] Adjust the camera's focus and angle to ensure the target bolt and surrounding background structures (such as the tower crane's angle steel flange) within at least 10cm are clearly visible in the frame. After starting video recording, the operator uses a rubber mallet to tap a standard section of the tower crane, recording a vibration video sequence lasting approximately 10 seconds. The video is then transferred to a computer for processing.

[0027] S120. For each of the aforementioned region images, the region image is segmented into a foreground image and a background image. Multiple target feature points in the foreground image are determined, and multiple target reference points in the background image are determined. The foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located.

[0028] The foreground image can be understood as a sub-image segmented from the region image, containing the specific area where the target bolt is located (such as the bolt itself), focusing on the visual features of the bolt itself (such as edges, corners, etc.). By extracting the displacement of its feature points, it directly reflects the position or deformation changes of the bolt, and is the core area for detecting loosening. The background image can be understood as the remaining part of the region image excluding the foreground image, containing the fixed structure or environment around the bolt (such as equipment casing, brackets, etc.). The background can be assumed to be fixed (or its displacement characteristics are known). By extracting the displacement of the background's reference points, the "displacement of the background itself" and the "displacement caused by bolt loosening" can be separated, avoiding misjudgment of bolt displacement due to background interference. The target feature points can be understood as key feature points extracted from the foreground image (such as corners, edge intersections, high gradient points, etc.), serving as markers for the bolt's position. Their displacement directly reflects the bolt's positional changes in space. By analyzing the displacement trends of the feature points, the degree of bolt loosening can be quantified. The target reference point can be understood as a key reference point extracted from the background image (such as fixed corner points of the background structure, stable texture points, etc.). Assuming that its displacement over time is mainly caused by non-bolt factors (such as overall vibration, camera shake), a reference system is provided. By calculating the average displacement of the reference point (reference displacement average), the interference of the overall background displacement on the feature point displacement can be corrected, and the accuracy of bolt displacement measurement can be improved.

[0029] Based on the above scheme, optionally, determining multiple target feature points in the foreground image and multiple target reference points in the background image includes: determining the target bolt corner region in the foreground image; enlarging the foreground image according to the texture entropy value of the target bolt corner region to obtain a first enlarged image; determining multiple target feature points according to the first enlarged image, wherein the magnification factor is positively correlated with the texture entropy value; enlarging the background image according to the first enlarged image to obtain a second enlarged image; and determining multiple target reference points according to the second enlarged image.

[0030] The target bolt corner region can be understood as an area with sharp geometric shapes and obvious contour transitions on the surface or edge of the target bolt (such as the hexagonal edges of the bolt head, the joint edges of the screw and nut, etc.). Due to their significant geometric features (such as abrupt edge changes and large grayscale gradients), corner regions typically contain rich local texture information (such as edge direction and corner details), making them key areas for extracting stable feature points. The texture entropy value can be understood as a numerical value used to quantify the complexity or information richness of the texture within the target bolt corner region. The more complex the texture (the richer the details, the more random the distribution), the higher the texture entropy value. The texture entropy value reflects the "discriminability" of the corner region. Regions with high entropy values ​​(such as rough edges and multi-directional textures) contain more unique features and require higher magnification to retain details; regions with low entropy values ​​(such as smooth edges and single textures) have fewer details, and lower magnification is sufficient. The first magnified image can be understood as the image obtained by magnifying the foreground image (including the target bolt corner region) by a magnification factor positively correlated with the texture entropy value. By selectively magnifying angular regions, the clarity of local details (such as edge lines and corners) is enhanced, providing more refined input for feature point extraction. A high-resolution first magnified image avoids feature point localization errors caused by insufficient resolution of the original image. The magnification factor can be understood as a coefficient that proportionally increases the image size, positively correlated with the texture entropy value (i.e., the higher the texture entropy value, the greater the magnification factor). The second magnified image can be understood as an image obtained by magnifying the background image based on the magnification parameters (such as magnification factor) of the first magnified image. The magnification logic of the background image is basically the same as that of the first magnified image (such as the magnification factor determined based on the texture entropy value of the foreground angular region), ensuring that the spatial scale of the foreground and background is consistent. This avoids coordinate misalignment problems caused by mismatched magnification factors between the foreground and background (such as foreground feature points not corresponding correctly to background reference points, and overlap or missing elements in the magnified image), thereby ensuring the accuracy of subsequent reference point displacement calculations.

[0031] This technical solution, by locating the angular region of the target bolt and dynamically magnifying the foreground image based on the texture entropy value, can improve the accuracy of feature point extraction while preserving highly complex angular details, avoiding edge blurring and missed detections. Simultaneously magnifying the background image using the same magnification parameters ensures consistent spatial scale between the foreground and background, preventing coordinate misalignment. This accurately obtains stable target feature points and reference points, effectively suppressing vibration and imaging noise interference, laying a reliable foundation for subsequent displacement separation and loosening detection.

[0032] Based on the above scheme, optionally, the step of magnifying the foreground image according to the texture entropy value of the target bolt corner region to obtain a first magnified image includes: magnifying the target bolt corner region according to the texture entropy value of the target bolt corner region and a preset region magnification factor to obtain a bolt corner magnified image; determining a bolt center image according to the foreground image and the target bolt corner region; magnifying the bolt center image according to the bolt corner magnified image to obtain a bolt center magnified image; and determining the first magnified image according to the bolt corner magnified image and the bolt center magnified image.

[0033] The region magnification factor can be understood as a preset magnification adjustment parameter used to calculate the actual magnification factor in conjunction with the texture entropy value of the target bolt corner region. The region magnification factor transforms the abstract texture complexity (entropy value) into a specific image magnification operation parameter. By adjusting this factor, the sensitivity of the magnification factor to the texture entropy value can be controlled (e.g., a high factor magnifies details in high-entropy regions more significantly, while a low factor weakens the differences), ensuring that the magnified image retains key details while avoiding noise amplification or computational redundancy caused by excessive magnification. The bolt corner magnified image can be understood as a sub-image magnified separately for the target bolt corner region (such as the edge of a hexagonal bolt head, the joint edge of the bolt and nut, etc.), based on its texture entropy value and the region magnification factor. Focusing on the most core geometric feature region of the bolt (the corner is the most easily identifiable marker of the bolt's position), local magnification solves the problem of edge blurring caused by insufficient resolution in the image's corners. When high-texture-entropy angular regions (such as rough edges) are magnified, details such as edge direction and corner inflection points become clearer, providing a high-resolution "microscopic perspective" for feature point extraction and avoiding missed feature point detection or positioning errors caused by blurred edges. The bolt center image can be understood as a sub-image of the central region cropped from the foreground image with the geometric center of the target bolt as the reference. It is an image obtained by subtracting the angular region of the target bolt from the foreground image, supplementing the main bolt information outside the angular region, and avoiding the problem of "local over-magnification and overall defocus" caused by only magnifying the angular region. The magnified bolt center image can be understood as a sub-image obtained by magnifying the bolt center image according to the magnification parameters (such as magnification factor) of the magnified bolt angular image.

[0034] One alternative implementation involves using an image segmentation algorithm to roughly select the bolt head area within the video frame. The algorithm automatically and accurately segments the bolt target region (Region of Bolt), while the remaining portion is considered the background region (Region of Background). Figure 5 As shown.

[0035] A magnification factor of α=25 is applied to the bolt area, and a magnification factor of α=5 is applied to the background area. At the same time, the texture entropy value of each bolt corner area within the bolt area is calculated. An additional 5% magnification factor is added to the bolt corner areas with rich texture, while α=25 is kept for the central area of ​​the bolt top surface with sparse texture.

[0036] The Euler video magnification algorithm (video motion magnification) is employed, with its bandpass filter frequency set to 5-20 Hz (covering the main response frequency band of the tower crane structure) based on pre-analysis and automatic identification. Motion magnification processing is applied to the video sequence (which can be obtained by extracting frames from the video sequence to capture regional images of the installation area of ​​the target bolt at multiple moments) to generate the magnified video. One frame can be extracted every one second to obtain 10 regional images.

[0037] In the magnified video sequence, perform the following operations: Feature point selection: such as Figure 6 As shown, the program automatically selects 20 high-quality feature points (Pt1-Pt20) within the bolt target area using the Shi-Tomasi corner detection algorithm. Figure 6 Only five were marked. At the same time, five points were selected as target reference points (Pr1-Pr5) within the adjacent background area that were believed to be fixed.

[0038] Synchronous tracking: The KLT optical flow algorithm based on image pyramids is used to synchronously track the positions of these 25 points in all frames.

[0039] This technical solution employs a layered magnification strategy. First, it locally magnifies angular areas with high texture entropy values, preserving key edges, corners, and other details. Then, it simultaneously magnifies the central area of ​​the bolt, ensuring that the local and overall scales of the foreground image are consistent, avoiding splicing misalignment and loss of detail. The magnified images of the edges and center are then merged to form the first magnified image, balancing the clarity of microscopic features with the integrity of the overall structure. This significantly improves the accuracy and stability of feature point localization, providing a high-quality image foundation for subsequent displacement analysis.

[0040] S130. Determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction; determine the average reference displacement value based on the reference displacement values ​​of multiple target images; and determine the multiple feature point displacement values ​​of multiple target feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction.

[0041] The preset direction can be understood as a predefined direction (such as horizontal, vertical or specific coordinate axis direction) used to unify the calculation dimension of displacement value, constrain the directionality of displacement analysis, avoid information confusion caused by the superposition of multiple displacement directions, and make displacement statistics more targeted (e.g., focusing on the axial loosening of bolts).

[0042] The reference displacement value can be understood as the displacement of a target reference point in a single target image relative to its corresponding reference point in the initial image in a preset direction. It reflects the overall displacement of the background in the preset direction (such as camera shake or vibration of the equipment mounting platform) and serves as the statistical basis for the reference displacements of multiple target images. The average reference displacement can be understood as the statistical average of the reference displacement values ​​of multiple target images, representing the average displacement level of the background in the preset direction. It is used to subsequently correct the feature point displacement (e.g., subtracting the average background displacement from the total feature point displacement to obtain the actual loosening displacement of the bolt). The feature point displacement value can be understood as the displacement of each target feature point in a single target image relative to its corresponding feature point in the initial image in a preset direction (each feature point corresponds to one displacement value), directly reflecting the dynamic changes in the local position of the bolt.

[0043] S140. Determine the total displacement of feature points based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement; determine the target displacement sequence based on the total displacement of feature points in each target image; determine the spectral information based on the target displacement sequence; and determine the fixing state of the target bolt based on the spectral information, wherein the fixing state includes a loose state and a tight state.

[0044] The total displacement of feature points can be understood as the comprehensive displacement obtained by processing the displacement values ​​of all target feature points in a single target image and subtracting the average reference displacement. This represents the true net displacement of the bolt relative to the fixed background at that moment, eliminating external interference and more accurately reflecting the bolt's state. The target displacement sequence can be understood as a sequence composed of the total displacements of feature points in multiple target images arranged in chronological order (such as time series data). It describes the change law of bolt displacement over time and is the core data for analyzing the loosening dynamic process. The spectral information can be understood as the frequency domain features obtained by transforming the target displacement sequence using methods such as Fourier transform. It reveals the frequency components of the target displacement sequence and helps determine the stability of the fixed state. The fixed state can be understood as the fixed condition of the target bolt, divided into "loose state" (the bolt has undergone significant displacement due to vibration or other reasons and cannot remain tight) and "tight state" (the bolt displacement is small and the connection remains stable). This can be determined through image analysis and displacement features and used to guide maintenance decisions (such as tightening loose bolts in time to avoid accidents).

[0045] The technical solution of this invention involves acquiring regional images of the installation area of ​​the target bolt at multiple times while the target bolt is vibrating. The earliest acquired regional image is determined as the initial image, and the regional images other than the initial image are determined as target images. Acquiring images at multiple times during vibration, with the earliest acquisition time as the benchmark, eliminates accumulated errors and improves displacement measurement accuracy. For each regional image, it is segmented into a foreground image and a background image. Multiple target feature points are determined in the foreground image, and multiple target reference points are determined in the background image. The foreground image includes the image area where the target bolt is located, and the background image includes the image area excluding the image area where the target bolt is located. Segmenting the foreground and background and extracting feature points and reference points respectively accurately separates bolt displacement interference from the background. The reference displacement value of the target reference point in each target image and the corresponding target reference point in the initial image in a preset direction is determined. Based on the reference displacement values ​​of multiple target images... The method involves determining the reference displacement mean, and separately determining the displacement values ​​of multiple target feature points in each target image and the corresponding target feature points in the initial image in a preset direction. This allows for the calculation of the reference point mean and extraction of feature point displacements, effectively eliminating background interference to obtain the true bolt displacement. The method also involves determining the total feature point displacement based on the multiple feature point displacement values ​​in each target image and the reference displacement mean, determining the target displacement sequence based on the total feature point displacement in each target image, determining the spectral information based on the target displacement sequence, and determining the fixed state of the target bolt based on the spectral information. The fixed state includes both loose and tight states. The method integrates features and reference displacements to obtain the total displacement and constructs a time series. Spectral analysis accurately determines the bolt's looseness or tightness, solving the problems in related technologies where bolt loosening in vibration environments is easily misjudged, background and bolt displacement are difficult to separate, and the fixed state cannot be accurately determined through time-series spectral analysis. This method achieves accurate separation and measurement of bolt displacement in vibration environments, constructs a time-series spectrum, and effectively improves the reliability of loosening and tightening state discrimination.

[0046] Example 2 Figure 2 This is a flowchart of a bolt fixing state detection method provided in Embodiment 2 of the present invention. This embodiment is a further refinement of the determination of the target displacement sequence based on the total displacement of feature points in each target image, building upon the previous embodiments. Optionally, determining the target displacement sequence based on the total displacement of feature points in each target image includes: determining the average displacement of feature points based on the total displacement of feature points in each target image and the number of target feature points; and determining the target displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images. For detailed implementation, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the previous embodiments will not be repeated here.

[0047] like Figure 2 As shown, the method may specifically include: S210. When the target bolt is under vibration, regional images of the installation area of ​​the target bolt are acquired at multiple times. The regional image acquired earliest is determined as the initial image, and the regional images other than the initial image are determined as the target images.

[0048] S220. For each of the aforementioned region images, the region image is segmented into a foreground image and a background image. Multiple target feature points in the foreground image are determined, and multiple target reference points in the background image are determined. The foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located.

[0049] S230. Determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction; determine the average reference displacement value based on the reference displacement values ​​of multiple target images; and determine the multiple feature point displacement values ​​of multiple target feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction.

[0050] S240. Determine the total displacement of feature points based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement; determine the average displacement of feature points based on the total displacement of feature points in each target image and the number of target feature points; determine the target displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images; determine the spectral information based on the target displacement sequence; and determine the fixing state of the target bolt based on the spectral information, wherein the fixing state includes a loose state and a tight state.

[0051] The mean displacement of feature points can be understood as the arithmetic mean of the displacement values ​​of all target feature points in a single target image (i.e., the total displacement of feature points in the image divided by the number of target feature points). This transforms the multiple dispersed feature point displacement values ​​in a single image (which may fluctuate due to local noise, uneven distribution of feature points, etc.) into a comprehensive index that reflects the overall average displacement level of the bolt at that moment. By averaging, the interference of abnormal displacements of individual feature points (such as outliers caused by noise) on the results can be reduced, improving the stability and representativeness of the displacement data and making subsequent time series analysis more reliable. The number can be understood as the total number of valid target feature points extracted from a single target image, used as the normalization denominator to convert the total feature point displacement (the cumulative value of all feature point displacements) into the average displacement (mean displacement of feature points).

[0052] Based on the above scheme, optionally, determining the target displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images further includes: determining an initial displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images; determining the correlation coefficient of every two average displacements of feature points in the initial displacement sequence; determining outliers based on the correlation coefficient and a preset coefficient threshold; removing the average displacement of feature points at the corresponding positions of the outliers in the initial displacement sequence to obtain a preliminary screening displacement sequence; interpolating the average displacement of feature points at the corresponding positions of the outliers based on the preliminary screening displacement sequence; and determining the target displacement sequence based on the interpolated result and the preliminary screening displacement sequence.

[0053] The initial displacement sequence can be understood as a time series directly generated based on the acquisition time of all target images and their corresponding mean displacements of feature points (the original sequence sorted by acquisition time without outlier processing). It serves as the "raw data carrier" for subsequent anomaly detection, recording the initial observation results of bolt displacement changes over time. Its core value is providing a complete time-series data foundation for anomaly identification, but it may contain outliers caused by noise, acquisition errors, or sudden interference (such as camera shake or sudden changes in lighting), requiring further cleaning to ensure the reliability of the analysis. The correlation coefficient can be understood as a statistic measuring the degree of linear correlation between the mean displacements of feature points at any two times in the initial displacement sequence. By quantifying the correlation of the mean displacements at different times, "outliers" deviating from the overall trend are identified. The coefficient threshold can be understood as a pre-set critical value for the correlation coefficient, used to determine whether the mean displacements of feature points at two times are significantly correlated. If the correlation coefficient is lower than this threshold, it is considered that the mean displacements at two times are not significantly correlated, and the mean displacements of these two feature points may be outliers. The outliers can be understood as data points in the initial displacement sequence that are "abnormal moments" caused by noise, acquisition errors, or sudden interference, and have no significant correlation with the average displacement values ​​at other times (i.e., the average displacement value of the feature points at that moment deviates from the overall trend). Outliers are the "noise sources" of interference analysis results; if they are not removed, subsequent spectral analysis (such as misjudgment of frequency components) or state discrimination (such as misjudging a loosening abrupt change) will be incorrect. The initial screening displacement sequence can be understood as the "clean" sequence remaining after removing the average displacement values ​​of the feature points corresponding to the outliers from the initial displacement sequence (containing only normal data points that pass the correlation test). As the "basic data" for interpolation repair, obvious interference terms are removed, and the true trend of bolt displacement changes is preserved. The interpolation can be understood as the process of estimating the missing average displacement values ​​at the positions corresponding to the outliers using mathematical methods (such as linear interpolation, spline interpolation, polynomial interpolation, etc.) based on the temporal continuity of the initial screening displacement sequence. Restoring the average displacement values ​​of the feature points at the time of the outliers fills the data gaps caused by the removal of outliers, making the displacement sequence continuous and complete in the time dimension.

[0054] In one optional implementation, for each target feature point Pt_i, the difference between its displacement and the average displacement of all reference points is calculated to obtain the net displacement sequence of that point: ; in, Indicates the first Zhang Image No. The displacement values ​​of the feature points of each target feature point Indicates the reference displacement mean. Indicates the first Zhang Image No. The net displacement of each target feature point relative to the average reference displacement is calculated. Finally, the feature point displacement values ​​of all target feature points are summed and divided by the number of target feature points in each region image to obtain the initial displacement sequence D_true(t) representing the overall relative motion of the bolt.

[0055] The correlation coefficients between each pair of the nine target net displacement sequences were calculated. Two points (Pt_3, Pt_6) were found to have correlation coefficients with the cluster average pattern below the threshold of 0.6; therefore, they were marked as outliers and removed. Replacement data for Pt_3 and Pt_6 were generated using data from the remaining seven valid points through interpolation.

[0056] Spectral analysis: The valid and verified target net displacement sequence D_true(t) is subjected to Fast Fourier Transform (FFT) to obtain its spectrum.

[0057] This technical solution employs correlation analysis on the initial displacement sequence to identify and eliminate outliers caused by noise or interference, effectively suppressing the contamination of time-series data by outliers. Interpolation repair using the initially screened sequence ensures the continuity and integrity of the displacement sequence in the time dimension. This results in a more realistic and smoother target displacement sequence, significantly improving the accuracy and robustness of subsequent spectral analysis and fixed-state discrimination, and avoiding misjudgments.

[0058] The technical solution of this invention normalizes the total displacement of feature points to the mean, eliminating the influence of differences in the number of feature points in different images and local noise, making the displacement data more stable and comparable; by constructing a target displacement sequence in combination with the acquisition time, the dispersed static measurement results are transformed into a continuous time change trajectory, providing a reliable and smooth time series basis for subsequent spectrum analysis, effectively improving the accuracy and anti-interference ability of bolt fixing status judgment.

[0059] Example 3 Figure 3 This is a flowchart of a bolt fixing state detection method provided in Embodiment 3 of the present invention. This embodiment is a further refinement of the above embodiments, which involves determining spectral information based on the target displacement sequence and determining the fixing state of the target bolt based on the spectral information. Optionally, determining spectral information based on the target displacement sequence and determining the fixing state of the target bolt based on the spectral information includes: performing a Fast Fourier Transform (FFT) on the target displacement sequence, determining the frequency corresponding to the point of maximum amplitude in the FFT result as the dominant displacement frequency, and determining the fixing state of the target bolt based on the dominant displacement frequency and a preset frequency reference value. For detailed implementation, please refer to the description of this embodiment. Technical features that are the same as or similar to those in the foregoing embodiments will not be repeated here.

[0060] like Figure 3 As shown, the method may specifically include: S310. When the target bolt is under vibration, regional images of the installation area of ​​the target bolt are acquired at multiple times. The regional image acquired earliest is determined as the initial image, and the regional images other than the initial image are determined as the target images.

[0061] S320. For each of the aforementioned region images, the region image is segmented into a foreground image and a background image. Multiple target feature points in the foreground image are determined, and multiple target reference points in the background image are determined. The foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located.

[0062] S330. Determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction; determine the average reference displacement value based on the reference displacement values ​​of multiple target images; and determine the multiple feature point displacement values ​​of multiple target feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction.

[0063] S340. Determine the total displacement of feature points based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement; determine the target displacement sequence based on the total displacement of feature points in each target image; perform a fast Fourier transform on the target displacement sequence; determine the frequency corresponding to the point with the maximum amplitude in the result of the fast Fourier transform as the main displacement frequency; determine the fixing state of the target bolt based on the main displacement frequency and a preset frequency reference value, wherein the fixing state includes a loose state and a tight state.

[0064] The Fast Fourier Transform (FFT) result can be understood as an array of FFT outputs, visually demonstrating the contribution of different frequency components to the displacement sequence—the larger the amplitude, the more significant the influence of that frequency component on displacement fluctuations. The point of maximum amplitude represents the most significant periodic component in the displacement sequence, a key identifier for distinguishing different displacement causes. The dominant displacement frequency can be understood as the frequency value corresponding to the point of maximum amplitude, i.e., the most important periodic frequency change in the displacement sequence. By analyzing its relationship with the vibration source frequency and the structure's natural frequency, the nature of the loosening can be determined. The frequency reference value can be understood as a pre-set frequency threshold or frequency range, used in conjunction with the dominant displacement frequency to determine the bolt fixing status.

[0065] Based on the above scheme, optionally, determining the fixing state of the target bolt according to the displacement principal frequency and a preset frequency reference value includes: when the displacement principal frequency is less than the frequency reference value, determining the root mean square average value according to the amplitude in the result of the fast Fourier transform, and determining the fixing state of the target bolt according to the root mean square average value and a preset displacement amplitude threshold.

[0066] The root mean square (RMS) average can be understood as the average value obtained by performing a root mean square operation on the FFT amplitude sequence. The displacement amplitude threshold can be understood as a pre-set RMS amplitude critical value used to distinguish between "normal displacement" (tight) and "abnormal displacement" (loose).

[0067] This technical solution introduces root mean square amplitude evaluation when the main frequency is low, and uses both frequency and energy indicators to determine the status, avoiding misjudgment based on a single frequency. It effectively distinguishes between low-frequency background vibration and actual loosening, and improves the accuracy and robustness of bolt fixing status determination under complex vibration environments.

[0068] Based on the above scheme, optionally, determining the fixing state of the target bolt according to the root mean square average value and a preset displacement amplitude threshold includes: If the root mean square average value is greater than the displacement amplitude threshold, the target bolt is determined to be in a loose state.

[0069] This technical solution directly determines loosening by comparing the root mean square value with a threshold, which can quickly identify displacement energy anomalies, enhance the sensitivity and real-time performance of the judgment, and avoid missing potential loosening risks.

[0070] Based on the above scheme, optionally, determining the fixing state of the target bolt according to the displacement main frequency and the preset frequency reference value includes: determining that the target bolt is in a tightened state when the displacement main frequency is greater than the frequency reference value and the root mean square average value is less than the displacement amplitude threshold.

[0071] One alternative implementation involves determining the root mean square value based on the amplitude in the result of the fast Fourier transform: ; in, For the first The amplitude at each frequency point This represents the number of frequency points.

[0072] At 300 N·m, the extracted main frequency is f_tight = 12.5 Hz, and the displacement amplitude (RMS) is A_tight = 0.3 pixels. At 50 N·m, the extracted main frequency is f_loose = 8.1 Hz, and the displacement amplitude (RMS) is A_loose = 1.8 pixels. By comparing with the preset displacement amplitude threshold and the preset frequency reference value, the 300 N·m state is determined to be a tight state, and the 50 N·m state is a loose state.

[0073] Through adaptive region magnification, the minute vibrations of the bolt are effectively highlighted, while background noise is suppressed (comparison). Figure 6 (Before and after magnification). Through coupled differential optical flow tracing, the pure relative displacement signal after eliminating environmental interference was successfully extracted (comparison). Figure 6 The original displacement and the net displacement after differentiation between the target point and the reference point. Finally, significantly different frequency and displacement amplitude results were obtained in the two states. Figure 6 ).

[0074] like Figure 7 As shown, the displacement time history curve amplitude in the loose state is much greater than that in the tight state. Based on the criterion of "frequency decreases and amplitude increases", the system accurately determines that the bolt is in a loose state.

[0075] By employing this technical solution, a high main frequency and a small root mean square average indicate that the bolt displacement is synchronized with external vibration and the amplitude is normal. This allows for reliable determination of tightness, avoids false alarms, and improves the accuracy and stability of condition assessment. The technical solution of this invention extracts the main frequency of displacement by FFT and compares it with the reference value, which can objectively identify whether the bolt moves synchronously with the vibration, effectively distinguish between tightness and looseness, and improve the timeliness and reliability of state judgment.

[0076] Example 4 Figure 4 This is a structural schematic diagram of a bolt fixing status detection device provided in Embodiment 4 of the present invention. Figure 4 As shown, the device includes: an image determination module 410, a point determination module 420, a displacement value determination module 430, and a fixed state determination module 440. Among them, Image determination module 410 is used to acquire regional images of the installation area of ​​the target bolt at multiple times when the target bolt is in a vibrating state, and determine the regional image acquired earliest as the initial image, and the regional images other than the initial image as target images; point determination module 420 is used to segment the regional image into a foreground image and a background image for each regional image, determine multiple target feature points in the foreground image, and determine multiple target reference points in the background image, wherein the foreground image includes the image area where the target bolt is located, and the background image includes the image area other than the image area where the target bolt is located; displacement value determination module 430 is used to determine the target value of each target image. The reference displacement value of the target reference point and the corresponding target reference point in the initial image in a preset direction is determined; the average reference displacement value is determined based on the reference displacement values ​​of multiple target images; and the displacement values ​​of multiple target feature points in each target image and the target feature points in the initial image in a preset direction are determined respectively. The fixed state determination module 440 is used to determine the total displacement of feature points based on the displacement values ​​of multiple feature points in each target image and the average reference displacement value; determine the target displacement sequence based on the total displacement of feature points in each target image; determine the spectral information based on the target displacement sequence; and determine the fixed state of the target bolt based on the spectral information, wherein the fixed state includes a loose state and a tight state.

[0077] The technical solution of this invention involves an image determination module that, when the target bolt is vibrating, acquires regional images of the installation area of ​​the target bolt at multiple times. The earliest acquired regional image is determined as the initial image, and all other regional images are determined as target images. Acquiring images at multiple times during vibration, with the earliest acquisition time as the benchmark, eliminates accumulated errors and improves displacement measurement accuracy. A point determination module segments each regional image into a foreground image and a background image, determining multiple target feature points in the foreground image and multiple target reference points in the background image. The foreground image includes the image area where the target bolt is located, and the background image includes the image area excluding the image area where the target bolt is located. Segmenting the foreground and background and extracting feature points and reference points respectively accurately separates bolt and background displacement interference. A displacement value determination module determines the reference displacement value in a preset direction between the target reference point in each target image and the corresponding target reference point in the initial image. Based on multiple target images... The reference displacement value is used to determine the average reference displacement. Furthermore, the displacement values ​​of multiple target feature points in each target image and their corresponding positions in the initial image are determined in a preset direction. The average reference point value can be calculated and the feature point displacement extracted, effectively eliminating background interference and obtaining the true bolt displacement. The fixed state determination module determines the total feature point displacement based on the multiple feature point displacement values ​​in each target image and the average reference displacement. The total feature point displacement of each target image is used to determine the target displacement sequence. The target displacement sequence is used to determine the spectral information. The spectral information is used to determine the fixed state of the target bolt, where the fixed state includes a loose state and a tight state. The total displacement can be constructed by fusing features and reference displacement to establish a time series. Spectral analysis accurately determines the bolt's looseness or tightness, solving the problems in related technologies where bolt loosening in vibration environments is easily misjudged, background and bolt displacement are difficult to separate, and the fixed state cannot be accurately determined through time-series spectral analysis. This achieves accurate separation and measurement of bolt displacement in vibration environments, constructs a time-series spectrum, and effectively improves the reliability of loosening and tightening state determination.

[0078] Based on the above scheme, optionally, the point determination module includes: a feature point determination submodule and a reference point determination submodule. The feature point determination submodule is used to determine the target bolt corner region in the foreground image, enlarge the foreground image based on the texture entropy value of the target bolt corner region to obtain a first enlarged image, and determine multiple target feature points based on the first enlarged image, wherein the magnification factor is positively correlated with the texture entropy value; the reference point determination submodule is used to enlarge the background image based on the first enlarged image to obtain a second enlarged image, and determine multiple target reference points based on the second enlarged image.

[0079] Optionally, based on the above scheme, the feature point determination submodule includes a first image magnification unit. The first image magnification unit is used to magnify the target bolt corner region based on the texture entropy value of the target bolt corner region and a preset region magnification factor to obtain a magnified bolt corner image; determine a bolt center image based on the foreground image and the target bolt corner region; magnify the bolt center image based on the magnified bolt corner image to obtain a magnified bolt center image; and determine a first magnified image based on the magnified bolt corner image and the magnified bolt center image.

[0080] Based on the above scheme, optionally, the fixed state determination module includes a target displacement sequence determination submodule. The target displacement sequence determination submodule is used to determine the average displacement of feature points based on the total displacement of feature points in each target image and the number of target feature points, and to determine the target displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images.

[0081] Optionally, based on the above scheme, the target displacement sequence determination submodule includes a target displacement sequence determination unit. The target displacement sequence determination unit is configured to: determine an initial displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images; determine the correlation coefficient between the average displacements of every two feature points in the initial displacement sequence; determine outliers based on the correlation coefficient and a preset coefficient threshold; remove the average displacement of feature points corresponding to the outliers in the initial displacement sequence to obtain a preliminary screening displacement sequence; interpolate the average displacement of feature points corresponding to the outliers based on the preliminary screening displacement sequence; and determine the target displacement sequence based on the interpolated result and the preliminary screening displacement sequence.

[0082] Based on the above scheme, optionally, the fixed state determination module includes a fixed state determination submodule. The fixed state determination submodule is used to perform a Fast Fourier Transform (FFT) on the target displacement sequence, determine the frequency corresponding to the point of maximum amplitude in the FFT result as the dominant displacement frequency, and determine the fixed state of the target bolt based on the dominant displacement frequency and a preset frequency reference value.

[0083] Optionally, based on the above scheme, the fixed state determination submodule includes a first state determination unit. The first state determination unit is used to determine the root mean square (RMS) value based on the amplitude in the result of the fast Fourier transform when the dominant displacement frequency is less than the frequency reference value, and to determine the fixed state of the target bolt based on the RMS value and a preset displacement amplitude threshold.

[0084] Optionally, based on the above scheme, the first state determination unit includes a loosening state determination subunit. The loosening state determination subunit is used to determine that the target bolt is in a loosening state when the root mean square average value is greater than the displacement amplitude threshold.

[0085] Optionally, based on the above scheme, the fixed state determination submodule includes a second state determination unit. The second state determination unit is used to determine that the target bolt is in a tightened state when the dominant displacement frequency is greater than the frequency reference value and the root mean square average value is less than the displacement amplitude threshold.

[0086] The bolt fixing status detection device provided in this embodiment of the invention can execute the bolt fixing status detection method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the method.

[0087] Example 5 Figure 8 A schematic diagram of an electronic device 10, which can be used to implement embodiments of the present invention, is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital processors, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0088] like Figure 8 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0089] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0090] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, a digital signal processor (DSP), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as a bolt fixing status detection method.

[0091] In particular, according to embodiments of the present invention, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of the present invention include a computer program product comprising a computer program carried on a non-transitory computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication unit 19, or installed from storage unit 18, or installed from ROM 12. When the computer program is executed by processor 11, it performs the functions defined in the methods of the embodiments of the present invention.

[0092] In some embodiments, a bolt fastening state detection method may be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program may be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the bolt fastening state detection method described above may be performed. Alternatively, in other embodiments, processor 11 may be configured to perform a bolt fastening state detection method by any other suitable means (e.g., by means of firmware).

[0093] Various embodiments of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, field-programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard products (ASSPs), systems-on-a-chip (SoCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0094] Computer programs used to implement the methods of the present invention may be written in any combination of one or more programming languages. These computer programs may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be performed. The computer programs may be executed entirely on a machine, partially on a machine, or as a standalone software package, partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0095] In the context of this invention, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium may include, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination thereof. Alternatively, a computer-readable storage medium may be a machine-readable signal medium. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0096] To provide interaction with a user, the systems and techniques described herein can be implemented on an electronic device having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the electronic device. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0097] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0098] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system to address the shortcomings of traditional physical hosts and VPS services, such as high management difficulty and weak business scalability.

[0099] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0100] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. A method for detecting the fixed state of bolts, characterized in that, include: When the target bolt is under vibration, regional images of the installation area of ​​the target bolt are acquired at multiple times. The regional image acquired earliest is determined as the initial image, and the regional images other than the initial image are determined as the target images. For each of the aforementioned region images, the region image is segmented into a foreground image and a background image. Multiple target feature points are determined in the foreground image, and multiple target reference points are determined in the background image. The foreground image includes the image region where the target bolt is located, and the background image includes the image region other than the image region where the target bolt is located. Determine the reference displacement value of the target reference point in each target image and the target reference point at the corresponding position in the initial image in a preset direction; determine the average reference displacement value based on the reference displacement values ​​of multiple target images; and determine the multiple feature point displacement values ​​of multiple target feature points in each target image and the target feature points at the corresponding positions in the initial image in a preset direction. The total displacement of the feature points is determined based on the displacement values ​​of multiple feature points in each target image and the average value of the reference displacement. The target displacement sequence is determined based on the total displacement of the feature points in each target image. The spectral information is determined based on the target displacement sequence. The fixing state of the target bolt is determined based on the spectral information, wherein the fixing state includes a loose state and a tight state.

2. The method according to claim 1, wherein determining a plurality of target feature points in the foreground image and determining a plurality of target reference points in the background image comprises: The target bolt corner region in the foreground image is determined, and the foreground image is magnified according to the texture entropy value of the target bolt corner region to obtain a first magnified image. Multiple target feature points are determined according to the first magnified image, wherein the magnification factor is positively correlated with the texture entropy value. The background image is magnified based on the first magnified image to obtain a second magnified image, and multiple target reference points are determined based on the second magnified image.

3. The method according to claim 2, characterized in that, The step of magnifying the foreground image based on the texture entropy value of the target bolt's corner region to obtain a first magnified image includes: Based on the texture entropy value of the target bolt corner region and the preset region magnification factor, the target bolt corner region is magnified to obtain a magnified bolt corner image; The bolt center image is determined based on the foreground image and the target bolt corner area. The bolt center image is then magnified based on the bolt corner magnification image to obtain the bolt center magnification image. The first magnified image is determined based on the magnified image of the bolt's edges and the magnified image of the bolt's center.

4. The method according to claim 1, characterized in that, The step of determining the target displacement sequence based on the total displacement of feature points in each target image includes: The average displacement of feature points is determined based on the total displacement of feature points in each target image and the number of target feature points. The target displacement sequence is determined based on the average displacement of feature points in multiple target images and the acquisition time of the target images.

5. The method according to claim 4, characterized in that, The step of determining the target displacement sequence based on the average displacement of feature points in multiple target images and the acquisition time of the target images further includes: An initial displacement sequence is determined based on the average displacement of feature points in multiple target images and the acquisition time of the target images. The correlation coefficient between the average displacements of every two feature points in the initial displacement sequence is determined. Anomalies are determined based on the correlation coefficient and a preset coefficient threshold. Remove the mean displacement of the feature points corresponding to the abnormal points in the initial displacement sequence to obtain the initial screening displacement sequence; The mean displacement of the feature points at the corresponding positions of the anomaly points is interpolated based on the initial screening displacement sequence, and the target displacement sequence is determined based on the interpolated result and the initial screening displacement sequence.

6. The method according to claim 1, characterized in that, The step of determining spectral information based on the target displacement sequence and determining the fixing state of the target bolt based on the spectral information includes: Perform a Fast Fourier Transform (FFT) on the target displacement sequence, determine the frequency corresponding to the point with the maximum amplitude in the FFT result as the main displacement frequency, and determine the fixing state of the target bolt based on the main displacement frequency and a preset frequency reference value.

7. The method according to claim 6, characterized in that, Determining the fixing state of the target bolt based on the displacement principal frequency and a preset frequency reference value includes: When the dominant displacement frequency is less than the frequency reference value, the root mean square average is determined based on the amplitude in the result of the fast Fourier transform, and the fixing state of the target bolt is determined based on the root mean square average and the preset displacement amplitude threshold.

8. The method according to claim 7, characterized in that, Determining the fixing state of the target bolt based on the root mean square average value and a preset displacement amplitude threshold includes: If the root mean square average value is greater than the displacement amplitude threshold, the target bolt is determined to be in a loose state.

9. The method according to claim 7, characterized in that, Determining the fixing state of the target bolt based on the displacement principal frequency and a preset frequency reference value includes: If the dominant displacement frequency is greater than the frequency reference value and the root mean square average value is less than the displacement amplitude threshold, the target bolt is determined to be in a tightened state.

10. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the bolt fastening state detection method according to any one of claims 1-9.

Images