Screw looseness detection method and system, terminal and medium

Abstract

The invention relates to the field of image processing, and particularly discloses a screw looseness detection method and system, a terminal and a medium, and the method comprises the steps: receiving a target region image which is collected by a camera and comprises a screw and a nut; a pre-processed image is firstly obtained; performing color space conversion on the preprocessed image by adopting an HSV-HIS mixed color space conversion model, extracting two mark line areas, performing straight line fitting to obtain two color reference lines, and calculating to obtain a first angle deviation and a first displacement deviation between the two color reference lines; performing gray processing on the preprocessed image to obtain a gray-scale image, performing edge extraction on two marking lines in the gray-scale image through an edge detection algorithm, performing straight line fitting to obtain two gray-scale reference lines, and calculating to obtain a second angle deviation and a second displacement deviation between the two gray-scale reference lines; and judging whether the screw is loosened according to the angle deviation and the displacement deviation. According to the invention, environmental noise interference is reduced, the detection precision is improved, and the generalization ability is strong.

Description

Technical Field

[0001] The present invention relates to the field of image processing, and in particular to a method, system, terminal and medium for detecting loose screws. Background Art

[0002] With the advancement of industrial automation and intelligent manufacturing, the structural complexity and operational intensity of mechanical equipment are constantly increasing. Screws, key components in mechanical connections, can cause loosening, potentially leading to increased vibration, structural instability, and even serious safety accidents. Traditional methods for detecting loose screws rely primarily on manual inspections or threshold judgment based on vibration signals. However, these methods suffer from low efficiency, high false alarm rates, and difficulty adapting to complex operating conditions.

[0003] Deep learning models based on vibration signals or image data can automatically extract features and identify the looseness of screws. However, existing methods still face several challenges, including environmental noise interference. The large amount of background noise present in industrial sites hinders effective feature extraction from vibration signals. They also suffer from insufficient generalization capabilities, significantly degrading the performance of existing models when applied across multiple devices and working conditions. Summary of the Invention

[0004] To solve the above problems, the present invention provides a screw loosening detection method, system, terminal and medium. Image processing is performed based on the marking line to obtain the reference line deviation of the marking line, and then the screw loosening is checked according to the deviation. During the process, the two branches of color space conversion and edge extraction are fused to reduce environmental noise interference, improve detection accuracy, and have strong generalization ability.

[0005] In a first aspect, the technical solution of the present invention provides a method for detecting screw loosening, comprising the following steps: A marking line is set on each of the screw and nut, and an image of a target area including the screw and nut captured by a camera is received; Preprocessing the target area image to obtain a preprocessing image; The HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line regions. Linear fitting is performed on the two marker line regions to obtain two color reference lines. The deviation between the two color reference lines is calculated as the first angle deviation and the first displacement deviation. The pre-processed image is gray-scale processed to obtain a gray-scale image. The edges of the two marking lines in the gray-scale image are extracted using an edge detection algorithm. Then, linear fitting is performed based on the two extracted edges to obtain two gray-scale reference lines. The deviation between the two gray-scale reference lines is calculated to obtain the second angle deviation and the second displacement deviation, respectively. Whether the screw is loose is determined according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0006] In an optional embodiment, the HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line areas, specifically including: The preprocessed image is an RGB image. The preprocessed image is converted into the HSV color space to obtain the initial HSV features of each pixel, including hue, saturation and brightness; Use the HIS color space enhancement algorithm to enhance the saturation and brightness of each pixel to obtain enhanced saturation and enhanced brightness; Combine the hue, enhanced saturation, and enhanced brightness in the initial HSV feature to obtain the mixed HSV feature of each pixel; Determine the feature base threshold boundary of the marking line in advance through a machine learning algorithm, then determine an adjustment amplitude based on the average brightness of the current preprocessed image, and adjust the feature base threshold boundary according to the adjustment amplitude to obtain the current feature threshold boundary; The mixed HSV feature of each pixel is compared with the current feature threshold boundary. If the mixed HSV feature of the pixel is within the current feature threshold range, the pixel is a pixel in the marker line area. All pixels are traversed to extract two marker line areas.

[0007] In an optional embodiment, after the pre-processed image is converted into a color space using the HSV-HIS hybrid color space conversion model and two marker line regions are extracted, the method further includes a step of processing the marker line regions using a morphological processing chain to remove noise, fill holes and smooth edges.

[0008] In an optional embodiment, performing linear fitting on the two marking line regions to obtain two color reference lines specifically includes: Get the spatial coordinates of each pixel in the marking line area; The spatial coordinates of each pixel are used to obtain two color reference lines by linear fitting using the least squares method.

[0009] In an optional embodiment, edge extraction is performed on the two marking lines in the grayscale image using an edge detection algorithm, specifically including: Calculate the horizontal gradient and vertical gradient of each pixel using a 5×5 extended Sobel operator, calculate the pixel gradient magnitude and gradient direction based on the horizontal gradient and vertical gradient, and obtain a gradient magnitude map and a gradient direction map; Non-local maximum suppression based on quadratic interpolation traverses each pixel in the gradient magnitude map and compares the gradient value of the current pixel with the gradient value of the two adjacent pixels in the gradient direction. If the current pixel gradient value is not the local maximum, the pixel is suppressed and only the local maximum point in the gradient direction is retained to obtain a preliminary edge map. Analyze the gradient histogram features of the local area of ​​the initial edge map, calculate the high and low thresholds of the gradient amplitude, traverse the preliminary edge map, determine the points above the high threshold as strong edges, and directly suppress the points below the low threshold. The points between the high and low thresholds are retained or suppressed in combination with the edge tracking algorithm to obtain a complete edge map; According to the spatial position prior of the screws and nuts in the image, the edge pixel sets of the two marking lines are separated from the complete edge map, that is, the marking line edges are extracted.

[0010] In an optional embodiment, the target area image is preprocessed, including performing white balance correction and gamma correction on the target area image.

[0011] In an optional embodiment, determining whether the screw is loose according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation specifically includes: Performing a weighted summation of the first angle deviation and the second angle deviation to obtain a fusion angle deviation; Performing a weighted summation of the first displacement deviation and the second displacement deviation to obtain a fused displacement deviation; When the fusion angle deviation is less than the first angle threshold and the fusion displacement deviation is less than the first displacement threshold, it is determined to be a tightened state; When the fusion angle deviation is greater than or equal to the first angle threshold and less than the second angle threshold, or the fusion displacement deviation is greater than or equal to the first displacement threshold and less than the second displacement threshold, it is determined to be a warning state; When the fusion angle deviation is greater than the second angle threshold, or the fusion displacement deviation is greater than the second displacement threshold, it is determined to be a loose state.

[0012] In a second aspect, the technical solution of the present invention provides a screw loosening detection system, comprising: The area image acquisition module is used to set a marking line on the screw and nut respectively, and receive the target area image containing the screw and nut captured by the camera; An image preprocessing module is used to preprocess the target area image to obtain a preprocessed image; A first deviation calculation module is used to perform color space conversion on the preprocessed image using the HSV-HIS hybrid color space conversion model and extract two marked line areas, perform straight line fitting on the two marked line areas to obtain two color reference lines, and calculate the deviation between the two color reference lines, which are respectively a first angle deviation and a first displacement deviation; A second deviation calculation module is used to grayscale the preprocessed image to obtain a grayscale image, extract the edges of the two marking lines in the grayscale image using an edge detection algorithm, and then perform straight line fitting based on the two extracted edges to obtain two grayscale reference lines, and calculate the deviation between the two grayscale reference lines, which are the second angle deviation and the second displacement deviation respectively; The screw state judgment module is used to judge whether the screw is loose according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0013] In a third aspect, the technical solution of the present invention provides a terminal, including: A memory, used for storing a screw loosening detection program; A processor is used to implement the steps of any of the above-mentioned screw loosening detection methods when executing the screw loosening detection program.

[0014] In a fourth aspect, the technical solution of the present invention provides a computer-readable storage medium, on which a loose screw detection program is stored. When the loose screw detection program is executed by a processor, the steps of the loose screw detection method as described in any one of the above items are implemented.

[0015] As can be seen from the above technical solution, this application offers the following advantages: Marking lines are pre-set on the screw and nut. During operation, real-time image acquisition is performed. Two branches are then used to extract reference lines for the two marking lines. The screw looseness status is determined based on the displacement and angle deviation between the reference lines. The present invention extracts marking line information from two branches, namely color features and edge contours, and fuses and verifies the deviation results. This reduces single-dimensional misjudgments caused by noise, illumination variations, and other factors, thereby improving the accuracy of screw loosening detection. Detection is performed through image processing, allowing for individual image processing of each pair of screws and nuts to avoid interference from multiple screw coupling. The HSV-HIS hybrid color space is more robust to industrial site illumination variations and background noise (such as equipment oil stains and metal reflections). Color conversion is used to highlight the marking line area. Grayscale edge detection, combined with an improved algorithm, suppresses environmental noise interference from edge extraction. The two branches operate in parallel, covering complex operating conditions such as illumination, noise, and stains, making it adaptable to diverse industrial environments. Marking lines are physical features, and processing based on marking lines can adapt to various equipment types and operating conditions. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] In order to more clearly illustrate the technical solution of the present application, the following is a brief introduction to the drawings required for the description. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without any creative work.

[0017] Figure 1 A schematic flow chart of a method for detecting loose screws provided in an embodiment of the present invention.

[0018] Figure 2 A schematic block diagram of the structure of a screw loosening detection system provided by an embodiment of the present invention.

[0019] Figure 3 A schematic diagram of the structure of a terminal provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the application objectives, features, and advantages of this application more obvious and easy to understand, the technical solutions protected by this application will be clearly and completely described below using specific embodiments and drawings. Obviously, the embodiments described below are only part of the embodiments of this application, not all of them. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0021] Unless otherwise defined, all technical and scientific terms used in this application have the same meaning as those commonly understood by those skilled in the art to which the present invention pertains. The terms used in this application and in the specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention.

[0022] Figure 1 The following is a flow chart of a method for detecting loose screws provided by an embodiment of the present invention. Figure 1 The execution subject may be a loose screw detection system. The loose screw detection method provided in the embodiment of the present invention is executed by a computer device, and accordingly, the loose screw detection system runs in the computer device. Depending on different needs, the order of the steps in the flowchart can be changed, and some steps can be omitted.

[0023] like Figure 1 As shown, the method includes the following steps.

[0024] S1, set a marking line on the screw and nut respectively, and receive the target area image containing the screw and nut captured by the camera.

[0025] Mark a marking line on the screw and nut in advance, which can be red or white, or other suitable colors. After the screw is tightened, the marking line on the screw and the marking line on the nut are aligned.

[0026] During operation, the camera periodically captures images of screws and nuts. Specifically, an industrial camera is used in conjunction with a ring light source and a diffuse reflector to ensure uniform lighting in the imaging environment. A polarizing filter is also used on the front end of the camera to effectively suppress interference from specular reflections on the metal surface. This stage collects a large amount of screw data, and removes overexposed, blurred, and out-of-focus images.

[0027] S2, preprocessing the target area image to obtain a preprocessed image.

[0028] Specifically, automatic white balance and gamma correction are performed on the target area image. Automatic white balance adjusts the image's color channels (R, G, B) to eliminate color deviations caused by differences in light source color temperature, such as a yellowish cast under warm light and a bluish cast under cold light. This ensures the color of the marker line is faithfully reproduced and maintains color consistency across images captured at different times or using different devices, preventing algorithm misjudgments due to ambient light variations. Gamma correction enhances low-contrast details and makes the marker line edge more visible through nonlinear brightness adjustment (typically, dark areas are brightened when gamma < 1, and highlights are suppressed when gamma > 1).

[0029] S3, uses the HSV-HIS hybrid color space conversion model to convert the preprocessed image into a color space and extract two marked line areas. Straight line fitting is performed based on the two marked line areas to obtain two color reference lines. The deviation between the two color reference lines is calculated, which is the first angle deviation and the first displacement deviation, respectively.

[0030] Steps S3 and S4 are two branches that extract marker line information from color and edge contour dimensions, respectively. The color space focuses on distinguishing the marker line's color from the background, while the grayscale space enhances edge geometry. The fusion of these two deviations reduces misjudgments caused by noise, lighting variations, and other factors in a single dimension, effectively improving the accuracy of calculating screw loosening angles and displacement deviations.

[0031] This step, designed for color marker recognition, uses the HSV-HIS hybrid color space conversion model. By establishing a machine learning-based adaptive color threshold algorithm (with a dynamically adjustable ±15%), combined with a morphological processing chain (including two iterative erosions of a 3×3 circular kernel, dilation of a 5×5 cross kernel, and post-processing with non-local means filtering), we obtain reference lines for the markers. The steps involved are as follows.

[0032] S31, using the HSV-HIS hybrid color space conversion model to convert the preprocessed image into a color space and extract two marker line areas.

[0033] S311, the pre-processed image is an RGB image, and the pre-processed image is converted into an HSV color space to obtain the initial HSV features of each pixel, including hue, saturation, and brightness.

[0034] The conversion formula is,

[0035] in, is the pixel hue, indicating the color phase, It is the pixel saturation, indicating the color purity. The larger the value, the less susceptible it is to light. For pixel brightness, highlight areas need to be suppressed.

[0036] S312: Use the HIS color space enhancement algorithm to enhance the saturation and brightness of each pixel to obtain enhanced saturation and enhanced brightness.

[0037] Specifically, the saturation and brightness are nonlinearly stretched to enhance the low-contrast areas, which is expressed as,

[0038] S313: Combine the hue, enhanced saturation, and enhanced brightness in the initial HSV feature to obtain a mixed HSV feature for each pixel.

[0039] Combine HSV hue ( ) and the enhanced channel of HIS ( , ), forming a mixed HSV feature .

[0040] S314, pre-determine the feature basic threshold boundary of the marking line through a machine learning algorithm, then determine the adjustment amplitude according to the average brightness of the current preprocessing image, and adjust the feature basic threshold boundary according to the adjustment amplitude to obtain the current feature threshold boundary.

[0041] Collect marker line samples under different lighting conditions and extract mixed HSV features Then, we use SVM or random forest classifier to learn the color threshold boundary and obtain the feature-based threshold boundary of the marker line. It should be noted that marker lines of different colors need to be trained separately.

[0042] During real-time detection, the adjustment range is determined based on the average brightness of the current pre-processed image. Different average brightnesses correspond to different adjustment ranges. For example, if the adjustment range is determined to be ±15%, the threshold boundary after adjustment is,

[0043] Where, 、 is the baseline threshold for the classifier output.

[0044] S315, compare the mixed HSV feature of each pixel with the current feature threshold boundary. If the mixed HSV feature of the pixel is within the current feature threshold range, the pixel is a pixel in the marking line area, and all pixels are traversed to extract two marking line areas.

[0045] For each pixel, if If it is within the dynamic threshold range, it is determined to be the marked line area.

[0046] In some optional implementations, after the marker line region is extracted in step S35 , a morphological processing chain is further used to process the marker line region to remove noise, fill holes, and smooth edges.

[0047] S32, performing straight line fitting according to the two marking line areas to obtain two color reference lines.

[0048] S321, obtaining the spatial coordinates of each pixel in the marking line area.

[0049] S322 , using the least squares method to perform straight line fitting based on the spatial coordinates of each pixel to obtain two color reference lines.

[0050] For the connected areas in the binary image, directly use the least squares method to fit the straight line equation:

[0051] Where, is the slope, is the intercept, are pixel space coordinates.

[0052] Among them, the slope The formula is,

[0053] Where, is the number of pixels involved in the calculation.

[0054] S33 , calculating and obtaining the deviation between the two color reference lines, which are a first angle deviation and a first displacement deviation.

[0055] The slope difference between the two colored reference lines, i.e. the first angle deviation, is calculated by the following formula: ,

[0056] Where, The slope of the colored reference line marked on the screw, The slope of the colored reference line marking the nut.

[0057] The center point distance between the two color reference lines, i.e. the first displacement deviation, is calculated by the following formula:

[0058] Where, The center point coordinates of the colored reference line marking the screw. The coordinates of the center point of the colored reference line marking the nut.

[0059] S4, grayscale processing is performed on the preprocessed image to obtain a grayscale image, and edges of the two marked lines in the grayscale image are extracted using an edge detection algorithm. Then, linear fitting is performed based on the two extracted edges to obtain two grayscale reference lines, and the deviation between the two grayscale reference lines is calculated, which are the second angle deviation and the second displacement deviation, respectively.

[0060] An improved multi-scale Canny edge detection algorithm is employed. First, a 5×5 dilated convolution kernel is constructed based on the Sobel operator. Gradients are calculated based on this convolution kernel, including first-order partial derivatives in the x and y directions. Floating-point operations are used to accurately determine the gradient magnitude (with an accuracy of 0.01 grayscale level) and orientation angle (quantized to four principal directions: 0°, 45°, 90°, and 135°) at each pixel. A non-local maximum suppression technique based on quadratic interpolation is then introduced. This technique eliminates points with small gradient variability, allowing for clearer edge delineation and avoiding interference. Sub-pixel-level extrema are located in the gradient direction, effectively suppressing false edges while preserving true edge details (edge ​​localization accuracy better than 0.3 pixels). The algorithm also incorporates an adaptive dual-threshold mechanism. By analyzing the gradient histogram features of local regions in the image, high and low thresholds are dynamically calculated (maintaining a threshold ratio of 1:2.5 to 1:3). An edge tracing algorithm is then used to connect broken edges, ensuring the integrity and continuity of reference line detection. The algorithm includes the following steps.

[0061] S41, performing grayscale processing on the preprocessed image to obtain a grayscale image.

[0062] First, the pre-processed image is converted to a grayscale image for subsequent edge extraction. After being processed into a grayscale image, Gaussian filtering can also be performed.

[0063] S42, extracting edges of the two marking lines in the grayscale image using an edge detection algorithm.

[0064] S421 , using a 5×5 extended Sobel operator to calculate the horizontal gradient and vertical gradient of each pixel, and calculating the pixel gradient magnitude and gradient direction based on the horizontal gradient and vertical gradient to obtain a gradient magnitude map and a gradient direction map.

[0065] The 5×5 extended Sobel operator is used to calculate the horizontal and vertical gradients of each pixel, expressed as,

[0066] Where, is the horizontal gradient, is the vertical gradient, is the grayscale value.

[0067] The gradient amplitude is , the gradient direction is , quantize the gradient direction to four main directions of 0°, 45°, 90°, and 135°.

[0068] S422, based on non-local maximum suppression using quadratic interpolation, traverses each pixel in the gradient magnitude map and compares the gradient value of the current pixel with the gradient value of the two adjacent pixels in the gradient direction. If the gradient value of the current pixel is not the local maximum, the pixel is suppressed and only the local maximum points in the gradient direction are retained to obtain a preliminary edge map.

[0069] It should be noted that suppressing the pixel means setting the gradient amplitude of the pixel to 0.

[0070] S423, analyze the gradient histogram features of the local area of ​​the initial edge map, calculate the high and low thresholds of the gradient amplitude, traverse the preliminary edge map, determine the points above the high threshold as strong edges, directly suppress the points below the low threshold, and retain or suppress the points between the high and low thresholds in combination with the edge tracking algorithm to obtain a complete edge map.

[0071] Specifically, the initial edge map is divided into multiple local regions. The gradient histogram of each local region is then analyzed, and high and low thresholds are proportionally assigned. For example, the top 15% quantile of the gradient value is used as the high threshold, and 1 / 3 of the high threshold is used as the low threshold. Strong edges are retained, and weak edges are only retained if they are connected to strong edges (edge ​​tracking algorithm).

[0072] S424 , based on the spatial position priors of the screw and nut in the image, separate the edge pixel sets of the two marking lines from the complete edge map, that is, extract the marking line edges.

[0073] In some optional embodiments, after isolating the edge pixel set of the marking line, a binary edge image can be obtained. This image can then be used to extract the marking lines on the screws and nuts through line segment detection based on a Hough transform. Specifically, the edge points are mapped to a polar coordinate parameter space, the votes are counted in an accumulator, the line parameters corresponding to the peaks are found, the top N polar coordinate parameter spaces with the highest votes are extracted, and the resulting image is converted into a line equation. Short line segments (length < 1 / 5 of the screw diameter) and overlapping line segments are filtered out to obtain a new set of marking lines.

[0074] S43, performing straight line fitting according to the two extracted edges to obtain two grayscale reference lines.

[0075] S44 , calculating and obtaining the deviation between the two grayscale reference lines, which are a second angle deviation and a second displacement deviation.

[0076] Step S43 and step S44 are similar to the above-mentioned steps S32 and S33 respectively, and will not be repeated here.

[0077] S5, judging whether the screw is loose according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0078] S51: Perform weighted summation on the first angle deviation and the second angle deviation to obtain a fusion angle deviation.

[0079] S52: Perform weighted summation on the first displacement deviation and the second displacement deviation to obtain a fused displacement deviation.

[0080] S53: When the fusion angle deviation is smaller than the first angle threshold and the fusion displacement deviation is smaller than the first displacement threshold, it is determined to be a tightened state.

[0081] S54: When the fusion angle deviation is greater than or equal to the first angle threshold and less than the second angle threshold, or the fusion displacement deviation is greater than or equal to the first displacement threshold and less than the second displacement threshold, it is determined to be a warning state.

[0082] S55: When the fusion angle deviation is greater than the second angle threshold, or the fusion displacement deviation is greater than the second displacement threshold, it is determined to be a loose state.

[0083] The weights of the two branches are set in advance as needed, and the displacement and angle deviation are fused according to the weight coefficients.

[0084] For example, a three-level judgment threshold is set: when the angle deviation is less than 2° and the position offset is less than 1mm, it is judged as a "tightened state" (confidence level > 99%); when the angle deviation is between 2°-5° or the position offset is between 1mm-2mm, it is judged as a "warning state" (confidence level 85%-95%); when the angle deviation exceeds 5° or the position offset exceeds 2mm, it is judged as a "loose state" (confidence level > 97%).

[0085] In particular, the dynamic compensation mechanism is integrated to automatically correct system errors by analyzing historical detection data, and a temperature compensation algorithm is used to eliminate the influence of thermal deformation on measurement results.

[0086] An embodiment of a method for detecting loose screws has been described in detail above. Based on the method for detecting loose screws described in the above embodiment, an embodiment of the present invention further provides a system for detecting loose screws corresponding to the method.

[0087] Figure 2 This is a schematic block diagram of the structure of a loose screw detection system provided in an embodiment of the present invention. In this embodiment, the loose screw detection system 200 can be divided into multiple functional modules according to the functions they perform. A module, as referred to in this invention, is a series of computer program segments that can be executed by at least one processor and can perform a fixed function, and is stored in a memory.

[0088] The area image acquisition module 210 is used to set a marking line on each of the screw and nut, and receive the target area image including the screw and nut captured by the camera.

[0089] The image preprocessing module 220 is used to preprocess the target area image to obtain a preprocessed image.

[0090] The first deviation calculation module 220 is used to use the HSV-HIS mixed color space conversion model to convert the preprocessed image into a color space and extract two marked line areas, perform straight line fitting based on the two marked line areas to obtain two color reference lines, and calculate the deviation between the two color reference lines, which are the first angle deviation and the first displacement deviation, respectively.

[0091] The second deviation calculation module 240 is used to grayscale the preprocessed image to obtain a grayscale image, extract the edges of the two marked lines in the grayscale image through an edge detection algorithm, and then perform straight line fitting based on the two extracted edges to obtain two grayscale reference lines, and calculate the deviation between the two grayscale reference lines, which are the second angle deviation and the second displacement deviation, respectively.

[0092] The screw state judgment module 250 is used to judge whether the screw is loose according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0093] The screw loosening detection system of this embodiment is used to implement the aforementioned screw loosening detection method. Therefore, the specific implementation method of the system can be seen in the embodiment part of the screw loosening detection method in the previous text. Therefore, its specific implementation method can refer to the description of the corresponding embodiments of each part and will not be elaborated here.

[0094] In addition, since the screw loosening detection system of this embodiment is used to implement the aforementioned screw loosening detection method, its function corresponds to that of the aforementioned method and will not be described in detail here.

[0095] Figure 3 The present invention provides a schematic diagram of a terminal 300, which includes a processor 310, a memory 320, and a communication unit 330. The processor 310 is configured to implement the following steps when executing the screw loosening detection program stored in the memory 320: A marking line is set on each of the screw and nut, and an image of a target area including the screw and nut captured by a camera is received; Preprocessing the target area image to obtain a preprocessing image; The HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line regions. Linear fitting is performed on the two marker line regions to obtain two color reference lines. The deviation between the two color reference lines is calculated as the first angle deviation and the first displacement deviation. The pre-processed image is gray-scale processed to obtain a gray-scale image. The edges of the two marking lines in the gray-scale image are extracted using an edge detection algorithm. Then, linear fitting is performed based on the two extracted edges to obtain two gray-scale reference lines. The deviation between the two gray-scale reference lines is calculated to obtain the second angle deviation and the second displacement deviation, respectively. Whether the screw is loose is determined according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0096] The terminal 300 includes a processor 310, a memory 320, and a communication unit 330. These components communicate via one or more buses. Those skilled in the art will appreciate that the server structure shown in the figure does not limit the present invention; it may be a bus structure or a star structure, and may include more or fewer components than shown, or combine certain components, or arrange the components differently.

[0097] Memory 320 can be used to store execution instructions of processor 310. Memory 320 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk. When the execution instructions in memory 320 are executed by processor 310, terminal 300 can perform some or all of the steps in the above-described method embodiments.

[0098] The processor 310 is the control center of the storage terminal. It uses various interfaces and lines to connect various parts of the entire electronic terminal. It executes various functions of the electronic terminal and / or processes data by running or executing software programs and / or modules stored in the memory 320, and calling data stored in the memory. The processor can be composed of an integrated circuit (IC), for example, it can be composed of a single packaged IC, or it can be composed of multiple packaged ICs with the same or different functions. For example, the processor 310 can only include a central processing unit (CPU). In the embodiment of the present invention, the CPU can be a single computing core or multiple computing cores.

[0099] The communication unit 330 is configured to establish a communication channel so that the storage terminal can communicate with other terminals, receive user data sent by other terminals, or send user data to other terminals.

[0100] The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0101] The present invention also provides a computer storage medium, wherein the storage medium may be a magnetic disk, an optical disk, a read-only memory (ROM) or a random access memory (RAM).

[0102] The computer storage medium stores a screw loosening detection program. When the screw loosening detection program is executed by the processor, the following steps are implemented: A marking line is set on each of the screw and nut, and an image of a target area including the screw and nut captured by a camera is received; Preprocessing the target area image to obtain a preprocessing image; The HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line regions. Linear fitting is performed on the two marker line regions to obtain two color reference lines. The deviation between the two color reference lines is calculated as the first angle deviation and the first displacement deviation. The pre-processed image is gray-scale processed to obtain a gray-scale image. The edges of the two marking lines in the gray-scale image are extracted using an edge detection algorithm. Then, linear fitting is performed based on the two extracted edges to obtain two gray-scale reference lines. The deviation between the two gray-scale reference lines is calculated to obtain the second angle deviation and the second displacement deviation, respectively. Whether the screw is loose is determined according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

[0103] Those skilled in the art will clearly understand that the techniques in the embodiments of the present invention can be implemented using software plus a necessary general-purpose hardware platform. Based on this understanding, the technical solutions in the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium such as a USB flash drive, a mobile hard drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk, among other media capable of storing program code, and includes instructions for causing a computer terminal (which can be a personal computer, a server, or a second terminal, a network terminal, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention.

[0104] In the several embodiments provided by the present invention, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0105] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0106] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0107] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein, but is intended to be construed in the widest manner consistent with the principles and novel features disclosed herein.

Claims

1. A method for detecting loose screws, characterized in that: The following steps are involved: A marking line is set on each of the screw and nut, and an image of a target area including the screw and nut captured by a camera is received; Preprocessing the target area image to obtain a preprocessing image; The HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line regions. Linear fitting is performed on the two marker line regions to obtain two color reference lines. The deviation between the two color reference lines is calculated as the first angle deviation and the first displacement deviation. The pre-processed image is gray-scale processed to obtain a gray-scale image. The edges of the two marking lines in the gray-scale image are extracted using an edge detection algorithm. Then, linear fitting is performed based on the two extracted edges to obtain two gray-scale reference lines. The deviation between the two gray-scale reference lines is calculated to obtain the second angle deviation and the second displacement deviation, respectively. Whether the screw is loose is determined according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

2. The method for detecting screw loosening according to claim 1, wherein: The HSV-HIS hybrid color space conversion model is used to convert the preprocessed image into a color space and extract two marker line areas, including: The preprocessed image is an RGB image. The preprocessed image is converted into the HSV color space to obtain the initial HSV features of each pixel, including hue, saturation and brightness; Use the HIS color space enhancement algorithm to enhance the saturation and brightness of each pixel to obtain enhanced saturation and enhanced brightness; Combine the hue, enhanced saturation, and enhanced brightness in the initial HSV feature to obtain the mixed HSV feature of each pixel; Determine the feature base threshold boundary of the marking line in advance through a machine learning algorithm, then determine an adjustment amplitude based on the average brightness of the current preprocessed image, and adjust the feature base threshold boundary according to the adjustment amplitude to obtain the current feature threshold boundary; The mixed HSV feature of each pixel is compared with the current feature threshold boundary. If the mixed HSV feature of the pixel is within the current feature threshold range, the pixel is a pixel in the marker line area. All pixels are traversed to extract two marker line areas.

3. The method for detecting screw loosening according to claim 1, wherein: After the preprocessed image is converted into color space using the HSV-HIS hybrid color space conversion model and two marker line regions are extracted, the method also includes a step of processing the marker line regions using a morphological processing chain to remove noise, fill holes and smooth edges.

4. The method for detecting screw loosening according to claim 1, wherein: Two color reference lines are obtained by performing straight line fitting on the two marked line areas, specifically including: Get the spatial coordinates of each pixel in the marking line area; The spatial coordinates of each pixel are used to obtain two color reference lines by linear fitting using the least squares method.

5. The method for detecting screw loosening according to claim 1, wherein: The edge detection algorithm is used to extract the edges of the two marking lines in the grayscale image, specifically including: Calculate the horizontal gradient and vertical gradient of each pixel using a 5×5 extended Sobel operator, calculate the pixel gradient magnitude and gradient direction based on the horizontal gradient and vertical gradient, and obtain a gradient magnitude map and a gradient direction map; Non-local maximum suppression based on quadratic interpolation traverses each pixel in the gradient magnitude map and compares the gradient value of the current pixel with the gradient value of the two adjacent pixels in the gradient direction. If the current pixel gradient value is not the local maximum, the pixel is suppressed and only the local maximum point in the gradient direction is retained to obtain a preliminary edge map. Analyze the gradient histogram features of the local area of ​​the initial edge map, calculate the high and low thresholds of the gradient amplitude, traverse the preliminary edge map, determine the points above the high threshold as strong edges, and directly suppress the points below the low threshold. The points between the high and low thresholds are retained or suppressed in combination with the edge tracking algorithm to obtain a complete edge map; According to the spatial position prior of the screws and nuts in the image, the edge pixel sets of the two marking lines are separated from the complete edge map, that is, the marking line edges are extracted.

6. The method for detecting screw loosening according to claim 1, wherein: The target area image is preprocessed, including white balance correction and gamma correction.

7. The method for detecting screw loosening according to claim 1, wherein: Determining whether the screw is loose based on the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation specifically includes: Performing a weighted summation of the first angle deviation and the second angle deviation to obtain a fusion angle deviation; Performing a weighted summation of the first displacement deviation and the second displacement deviation to obtain a fused displacement deviation; When the fusion angle deviation is less than the first angle threshold and the fusion displacement deviation is less than the first displacement threshold, it is determined to be a tightened state; When the fusion angle deviation is greater than or equal to the first angle threshold and less than the second angle threshold, or the fusion displacement deviation is greater than or equal to the first displacement threshold and less than the second displacement threshold, it is determined to be a warning state; When the fusion angle deviation is greater than the second angle threshold, or the fusion displacement deviation is greater than the second displacement threshold, it is determined to be a loose state.

8. A screw loosening detection system, characterized in that: include: The area image acquisition module is used to set a marking line on the screw and nut respectively, and receive the target area image containing the screw and nut captured by the camera; An image preprocessing module is used to preprocess the target area image to obtain a preprocessed image; A first deviation calculation module is used to perform color space conversion on the preprocessed image using the HSV-HIS hybrid color space conversion model and extract two marked line areas, perform straight line fitting on the two marked line areas to obtain two color reference lines, and calculate the deviation between the two color reference lines, which are respectively a first angle deviation and a first displacement deviation; A second deviation calculation module is used to grayscale the preprocessed image to obtain a grayscale image, extract the edges of the two marking lines in the grayscale image using an edge detection algorithm, and then perform straight line fitting based on the two extracted edges to obtain two grayscale reference lines, and calculate the deviation between the two grayscale reference lines, which are the second angle deviation and the second displacement deviation respectively; The screw state judgment module is used to judge whether the screw is loose according to the first angle deviation, the first displacement deviation, the second angle deviation, and the second displacement deviation.

9. A terminal, characterized in that: include: A memory, used for storing a screw loosening detection program; A processor is configured to implement the steps of the screw loosening detection method according to any one of claims 1 to 7 when executing the screw loosening detection program.

10. A computer-readable storage medium, characterized in that The readable storage medium stores a screw loosening detection program, and when the screw loosening detection program is executed by the processor, the steps of the screw loosening detection method according to any one of claims 1 to 7 are implemented.

Images