A screw locking quality detection method and system based on machine vision

Abstract

The present application provides a method and system for detecting screw locking quality based on machine vision, which extracts the image edge information of the target screw from the screw image set, determines the image pixel line of the target screw edge contour according to the image edge information and the pixel point features of the screw image set; determines the edge fluctuation cost of the target screw through the image pixel line and the light features around the target screw, and determines the screw defect domain of the target screw for the screw image set according to the edge fluctuation cost; identifies the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, determines the detection confidence index of the target screw after locking according to the thread engagement state and the screw defect domain; and calibrates the quality of the target screw after locking through the detection confidence index. The above scheme can eliminate the influence of tiny defects on the screw surface on the screw locking quality of machine vision detection, thereby improving the detection quality of screw locking.

Description

Technical Field

[0001] The present application relates to the technical field of screw locking, and more specifically, to a method and system for detecting screw locking quality based on machine vision. Background Art

[0002] Screw locking is a device or method used to ensure a firm screw connection. It is commonly used in mechanical engineering and construction. Its main function is to prevent the screws from loosening due to vibration, impact or other external forces, ensuring the stability and safety of the connection.

[0003] The screw locking quality inspection based on machine vision uses computer vision technology to monitor and inspect the screw installation process in real time to ensure the quality of screw tightening. This method has the advantages of high precision, high efficiency and non-contact, and is widely used in the field of industrial automation. In the existing screw locking quality inspection process based on machine vision, the image after the screw is locked is collected by machine vision equipment, and the collected image is analyzed to determine whether the screw after locking meets the application requirements. However, if there are tiny defects on the surface of the screw or the detection light fluctuates greatly, the image taken by the machine vision will not be able to fully show the current condition of the screw, resulting in false detection and missed detection of the detected screw results. Therefore, how to eliminate the influence of tiny defects on the surface of the screw on the quality of screw locking detected by machine vision, thereby improving the detection quality of screw locking has become a difficult problem faced by the industry. Summary of the invention

[0004] The present application provides a method and system for detecting screw fastening quality based on machine vision, which can eliminate the influence of tiny defects on the surface of the screw on the screw fastening quality detected by machine vision, thereby improving the detection quality of screw fastening.

[0005] In a first aspect, the present application provides a method for detecting screw fastening quality based on machine vision, comprising the following steps:

[0006] Collecting screw images during the tightening process of the target screws to obtain a screw image set;

[0007] Extracting image edge information of a target screw from the screw image set, performing edge recognition on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, and thereby obtaining an image pixel line of an edge contour of the target screw;

[0008] Determine the edge fluctuation cost of the target screw in the screw image set by using the image pixel line and the light features around the target screw, perform quality identification on the pixel value data in the screw image set according to the edge fluctuation cost, and obtain the screw defect domain of the target screw;

[0009] Acquire the locking information of the target screw, identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain;

[0010] The quality of the target screw after locking is calibrated by the detection confidence index.

[0011] In some embodiments, extracting the image edge information of the target screw from the screw image set specifically includes:

[0012] Extracting image contour features of each screw image in the screw image set;

[0013] Determine the image edge features of the target screw according to all the image contour features;

[0014] The image edge information of the target screw is determined by using the image edge features and the screw image set.

[0015] In some embodiments, edge recognition of a target screw in the screw image set is performed according to the image edge information and the pixel point features of the screw image set, thereby obtaining an image pixel line of the target screw edge contour specifically comprising:

[0016] Obtaining pixel features of the screw image set;

[0017] Determine edge recognition information of a target screw according to the image edge information and pixel features of the screw image set;

[0018] determining a plurality of image contour lines according to the edge recognition information and the screw image set;

[0019] The image pixel line of the target screw edge contour is determined through all image contour lines.

[0020] In some embodiments, determining the edge fluctuation cost of the target screw in the screw image set by using the image pixel line and the light features around the target screw specifically includes:

[0021] Get the light characteristics around the target screw;

[0022] determining a light fluctuation entropy of the light characteristic;

[0023] determining confidence fluctuation information according to the light fluctuation entropy and the image pixel line;

[0024] The edge fluctuation cost of the target screw in the screw image set is determined according to the confidence fluctuation information.

[0025] In some embodiments, quality identification is performed on the pixel value data in the screw image set according to the edge fluctuation cost to obtain the screw defect domain of the target screw specifically including:

[0026] Acquire pixel value data and image pixel lines in the screw image set;

[0027] Determine the surface defect information of the target screw by using the pixel value data and the image pixel line;

[0028] The screw defect area of ​​the target screw is identified by the edge fluctuation cost and the surface defect information.

[0029] In some embodiments, identifying the thread engagement state of the target screw in the screw image set in the tightened state by using the locking information specifically includes:

[0030] Determine an angle characteristic quantity of a target screw according to the locking information;

[0031] determining a tightening state of a target screw according to the screw image set;

[0032] The thread engagement state of the target screw in the tightened state in the screw image set is determined according to the angle feature amount and the tightened state.

[0033] In some embodiments, determining the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect area specifically includes:

[0034] Determining the engagement confidence of the target screw by using the identified thread engagement state;

[0035] Determining the defect ratio of the target screw after tightening according to the screw defect domain;

[0036] The detection confidence index of the target screw after locking is determined according to the occlusion confidence and the defect proportion.

[0037] In a second aspect, the present application provides a screw fastening quality inspection system based on machine vision, comprising:

[0038] An acquisition module is used to collect screw images during the tightening process of the target screw to obtain a screw image set;

[0039] A processing module, used to extract image edge information of a target screw from the screw image set, perform edge recognition on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, and thereby obtain an image pixel line of an edge contour of the target screw;

[0040] The processing module is further used to determine the edge fluctuation cost of the target screw in the screw image set through the image pixel line and the light characteristics around the target screw, and perform quality identification on the pixel value data in the screw image set according to the edge fluctuation cost to obtain the screw defect domain of the target screw;

[0041] The processing module is further used to obtain the locking information of the target screw, identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain;

[0042] The execution module is used to calibrate the quality of the target screw after locking through the detection confidence index.

[0043] In a third aspect, the present application provides a computer device, comprising a memory and a processor, wherein the memory stores a code, and the processor is configured to obtain the code and execute the above-mentioned machine vision-based screw fastening quality inspection method.

[0044] In a fourth aspect, the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the above-mentioned machine vision-based screw fastening quality inspection method is implemented.

[0045] The technical solution provided by the embodiments disclosed in this application has the following beneficial effects:

[0046] In the machine vision-based screw locking quality detection method and system provided by the present application, firstly, a screw image of a target screw in the locking process is collected to obtain a screw image set; image edge information of the target screw is extracted from the screw image set, and edge recognition is performed on the target screw in the screw image set according to the image edge information and the pixel point features of the screw image set, thereby obtaining an image pixel line of the edge contour of the target screw; the edge fluctuation cost of the target screw in the screw image set is determined according to the image pixel line and the light features around the target screw, and quality recognition is performed on the pixel value data in the screw image set according to the edge fluctuation cost to obtain a screw defect domain of the target screw; the locking information of the target screw is obtained, and the thread engagement state of the target screw in the screw image set in the tightened state is identified according to the locking information, and the detection confidence index of the target screw after locking is determined according to the identified thread engagement state and the screw defect domain; and the quality of the target screw after locking is calibrated according to the detection confidence index.

[0047] It can be seen that in the process of screw locking quality inspection, the present application first analyzes the screw image set through the light characteristics around the target screw, determines the degree of fluctuation of the edge identification line in the picture taken when the target screw is under light fluctuation, and then obtains the edge fluctuation cost, so as to determine the edge in the picture taken when the target screw is locked, thereby suppressing the influence of light fluctuation on the detection of target screw locking; secondly, the pixel value data in the screw image set is quality identified through the edge fluctuation cost, and the screw defect domain of the target screw after locking is obtained, so as to locate the defects of the target screw after locking, find out the small defects and the like that appear in the locking process of the target screw, thereby improving the detection quality of the target screw locking; further, by analyzing the screw defect domain in combination with the occlusion of the thread of the target screw after tightening, and then obtaining the detection confidence index, the state of the target screw after locking can be judged more accurately, thereby eliminating the influence of the presence of small defects on the screw surface or the large fluctuation of the detection light on the quality of the target screw after locking, and finally, the quality of the target screw after locking is calibrated by the detection confidence index. In summary, the technical solution improved by the present application eliminates the influence of tiny defects on the surface of the screw on the quality of screw locking detected by machine vision, thereby improving the detection quality of screw locking. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] Figure 1 is an exemplary flow chart of a method for inspecting screw fastening quality based on machine vision according to some embodiments of the present application;

[0049] Figure 2 is an exemplary flow chart for determining image pixel lines according to some embodiments of the present application;

[0050] Figure 3 is a schematic diagram of a confidence fluctuation region according to some embodiments of the present application;

[0051] Figure 4 is an exemplary flow chart of determining the thread engagement state according to some embodiments of the present application;

[0052] Figure 5 is a schematic diagram of exemplary hardware and / or software of a screw fastening quality inspection system based on machine vision according to some embodiments of the present application;

[0053] Figure 6 It is a structural schematic diagram of a computer device for implementing a screw fastening quality inspection method based on machine vision according to some embodiments of the present application. DETAILED DESCRIPTION

[0054] The core of this application is to collect the screw image during the locking process of the target screw to obtain a screw image set; extract the image edge information of the target screw from the screw image set, and perform edge recognition on the target screw in the screw image set according to the image edge information and the pixel point features of the screw image set, thereby obtaining the image pixel line of the edge contour of the target screw; determine the edge fluctuation cost of the target screw in the screw image set through the image pixel line and the light characteristics around the target screw, and perform quality recognition on the pixel value data in the screw image set according to the edge fluctuation cost to obtain the screw defect domain of the target screw; obtain the locking information of the target screw, and identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain; and perform quality calibration on the target screw after locking through the detection confidence index. The above scheme can eliminate the influence of tiny defects on the surface of the screw on the quality of screw locking by machine vision detection, thereby improving the detection quality of screw locking.

[0055] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods. Figure 1 , which is an exemplary flow chart of a screw fastening quality detection method based on machine vision according to some embodiments of the present application. The screw fastening quality detection method 100 based on machine vision mainly includes the following steps:

[0056] In step 101, a screw image of a target screw during a tightening process is collected to obtain a screw image set.

[0057] In a specific implementation, an industrial camera in the prior art is used to collect screw images during the target screw fastening process to obtain a screw image set, wherein the screw image set represents a collection of all screw images.

[0058] It should be noted that the screw images in the screw image set in the present application represent pictures taken by an industrial camera during the locking process of the target screw, which record the running status of the target screw during the locking process.

[0059] In step 102, image edge information of a target screw is extracted from the screw image set, and edge recognition is performed on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, thereby obtaining an image pixel line of the target screw edge contour.

[0060] In some embodiments, extracting the image edge information of the target screw from the screw image set may be achieved by using the following steps:

[0061] Extracting image contour features of each screw image in the screw image set;

[0062] Determine the image edge features of the target screw according to all the image contour features;

[0063] The image edge information of the target screw is determined by using the image edge features and the screw image set.

[0064] In specific implementation, the image contour features of each screw image in the screw image set can be extracted in the following manner, namely: a screw image in the screw image set is selected as the selected screw image, the contour size of the target screw in the selected screw image is extracted by a contour detection algorithm in the prior art, and the contour size is used as the image contour feature of the selected screw image, and the image contour features of the remaining screw images in the screw image set are further determined, wherein the contour detection algorithm first performs binarization processing on the image to obtain a binary image, and uses the cv2.findContours function to search for the contour of the target screw in the binary image, and calculates the size of the contour through the contour, and the image contour feature represents the contour size feature of the target screw in the screw image, which is used to analyze the state of the target screw during the tightening process; according to all the image contour features Determining the image edge feature of the target screw can be achieved in the following manner, namely: taking the average size of the contour sizes in all image edge features as the image edge size of the target screw during the tightening process, and taking the image edge size as the image edge feature, wherein the image edge feature represents the edge size of the target screw during the tightening process, and is used to analyze the situation of the target screw during the tightening process; determining the image edge information of the target screw through the image edge feature and the screw image set can be achieved in the following manner, namely: determining the image edge information of the target screw through a shape matching algorithm in the prior art in combination with the image edge feature and the screw image set, wherein the shape matching algorithm in this embodiment is a Hu moment algorithm, and the shape of the target screw is matched by calculating the seven invariant moments of the image. In other embodiments, other methods can also be used for determination, which are not limited here.

[0065] It should be noted that the image edge information in this application is information that reflects the contour, shape, boundary and other features of the target screw. The image edge information is used to detect areas in the image where pixel values ​​change significantly. These changes usually correspond to the boundaries of the target screw and external texture features.

[0066] In some embodiments, reference Figure 2As shown, this figure is a schematic diagram of the process of determining image pixel lines in some embodiments of the present application. In this embodiment, edge recognition of the target screw in the screw image set is performed based on the image edge information and the pixel point features of the screw image set, and then the image pixel line of the target screw edge contour is obtained, which can be achieved by the following steps:

[0067] First, in step 1021, the pixel features of the screw image set are obtained;

[0068] Next, in step 1022, edge recognition information of the target screw is determined based on the image edge information and pixel features of the screw image set;

[0069] Then, in step 1023, a plurality of image contour lines are determined according to the edge recognition information and the screw image set;

[0070] Finally, in step 1024, the image pixel lines of the target screw edge contour are determined through all the image contour lines.

[0071] In specific implementation, the pixel point features in the screw image set can be obtained in the following manner, namely: the total number of pixels of each screw image in the screw image set is read through the Python OpenCV library in the prior art, and the total number of all pixels is used as the pixel point feature of the screw image set, wherein the pixel point feature represents the feature of the total number of pixels in the screw image set.

[0072] In a specific implementation, the edge recognition information of the target screw is determined according to the image edge information and the pixel point features of the screw image set, which can be achieved in the following manner, namely: the shape in the image edge information is used as the segmentation frame of the target screw, and the area enclosed by the segmentation frame on each screw image in the screw image set is used as the screw area to be detected of each screw image, a screw image in the screw image set is selected as the selected screw image, the number of pixels in the screw area to be detected corresponding to the selected screw image is divided by the total number of pixels corresponding to the pixel point features of the selected screw image, and the value obtained by the division is used as the screw ratio of the selected screw image, and the screw ratio of the remaining screw images in the screw image set is continued to be determined, wherein the screw ratio represents a parameter value of the ratio of the pixel points corresponding to the target screw in the screw image, which is used to detect the current captured image of the target screw. The situation at the moment is judged, a screw ratio is selected as the selected screw ratio, and the selected screw ratio is compared with the average ratio of all screw ratios. If the selected screw ratio is greater than the average ratio of all screw ratios, the screw area to be detected corresponding to the selected screw ratio is equally reduced, and the reduction value of the selected screw ratio is calculated until the reduction value of the selected screw ratio is the same as the average ratio of all screw ratios, and the screw area to be detected after the same reduction is used as the screw area of ​​the target screw in the screw image corresponding to the selected screw ratio, and the screw area of ​​the target screw in the screw image corresponding to the remaining screw ratio is continued to be determined, wherein the screw area represents the area of ​​the target screw in the screw image, and the set of all screw areas is used as edge recognition information. In other embodiments, other methods can also be used for determination, which is not limited here.

[0073] It should be noted that the edge recognition information in the present application represents the edge information obtained by segmenting the target screw from the picture taken during the locking process, and is used to segment the target screw in the picture taken to facilitate the analysis of the condition of the target screw during the locking process.

[0074] In specific implementation, determining multiple image contour lines according to the edge recognition information and the screw image set can be achieved in the following manner, namely: selecting a screw area in the edge recognition information as the selected screw area, and using the boundary line of the area enclosed by the selected screw area on the corresponding screw image in the screw image set as the image contour line of the selected screw area, and continuing to determine the image contour lines of the remaining screw areas in the edge recognition information, wherein the image contour line represents the contour line of the target screw in the screw image; determining the image pixel line of the edge contour of the target screw through all the image contour lines can be achieved in the following manner, namely: fitting all the image contour lines through the linear regression method in the prior art, and using the fitted line as the image pixel line of the edge contour of the target screw in the screw image set. In other embodiments, other methods can also be used for determination, which are not limited here.

[0075] It should be noted that in the present application, edge recognition is performed on the target screw in the screw image set according to the image edge information and the pixel point features of the screw image set, that is: the edge recognition information of the target screw is determined according to the image edge information and the pixel point features of the screw image set, and multiple image contour lines are determined according to the edge recognition information and the screw image set, wherein the edge recognition represents the segmentation of the edge of the target screw in the screw image; the image pixel line is a pixel line that reliably reflects the edge contour of the target screw, and is used to segment the target screw in the screw image set, and then analyze the locking condition of the target screw.

[0076] In step 103, the edge fluctuation cost of the target screw in the screw image set is determined by the image pixel line and the light characteristics around the target screw, and the quality of the pixel value data in the screw image set is identified according to the edge fluctuation cost to obtain the screw defect domain of the target screw.

[0077] In some embodiments, determining the edge fluctuation cost of the target screw in the screw image set by using the image pixel line and the light characteristics around the target screw can be achieved by the following steps:

[0078] Get the light characteristics around the target screw;

[0079] determining a light fluctuation entropy of the light characteristic;

[0080] determining confidence fluctuation information according to the light fluctuation entropy and the image pixel line;

[0081] The edge fluctuation cost of the target screw in the screw image set is determined according to the confidence fluctuation information.

[0082] In specific implementation, obtaining the light characteristics around the target screw can be achieved in the following manner, namely: using the light intensity acquisition device in the prior art to collect all the light intensities of the target screw during the tightening process, and using all the light intensities as the light characteristics around the target screw, wherein the light characteristics represent the characteristics of the light intensity of the target screw during the tightening process; determining the light fluctuation entropy of the light characteristics can be achieved in the following manner, namely: using the entropy of all light intensities in the light characteristics as the light fluctuation entropy of the light characteristics, wherein the light fluctuation entropy represents the parameter value of the fluctuation degree of the light intensity around the target screw, which is used to analyze the picture of the target screw during the tightening process; determining the confidence fluctuation information based on the light fluctuation entropy and the image pixel line can be achieved in the following manner, namely: multiplying the image pixel line with the light fluctuation entropy, using the area between the multiplied line and the image pixel line as the confidence fluctuation area, and using the confidence fluctuation area as the confidence fluctuation information, for example: the confidence fluctuation area is such as Figure 3 As shown, the confidence fluctuation information represents the information of the fluctuation interval of the image pixel line under the interference of light, which is used to analyze the state of the target screw under the light change during the tightening process; the edge fluctuation cost of the target screw in the screw image set according to the confidence fluctuation information can be determined in the following manner, namely: all light intensities of the target screw in the screw image set are obtained, the maximum light intensity of all light intensities is divided by the minimum light intensity, and the value obtained by the division is subjected to a logarithmic operation with a base of 2, the value obtained by the logarithmic operation is multiplied by the confidence fluctuation area in the confidence fluctuation information, and the multiplied area is used as the edge fluctuation cost of the target screw in the screw image set. In other embodiments, other methods can also be used for determination, which are not limited here.

[0083] It should be noted that the edge fluctuation cost in the present application is a cost that reflects the degree of fluctuation of the edge identification line of the target screw in the picture taken under light fluctuation conditions. It is used to judge the fluctuation of the edge of the target screw in the picture taken, suppresses the influence of light fluctuation on the detection of target screw locking, and improves the quality of detecting target screw locking.

[0084] In some embodiments, quality identification of the pixel value data in the screw image set is performed according to the edge fluctuation cost to obtain the screw defect region of the target screw by using the following steps:

[0085] Acquire pixel value data and image pixel lines in the screw image set;

[0086] Determine the surface defect information of the target screw by using the pixel value data and the image pixel line;

[0087] The screw defect area of ​​the target screw is identified by the edge fluctuation cost and the surface defect information.

[0088] In specific implementation, obtaining the pixel value data in the screw image set can be achieved in the following manner, namely: obtaining the pixel value set of each screw image in the screw image set through the OpenCV library of Python software in the prior art, wherein the pixel value set represents the set of all pixel values, and the set of all pixel value sets is used as the pixel value data in the screw image set; determining the surface defect information of the target screw through the pixel value data and the image pixel line can be achieved in the following manner, namely: selecting a screw image in the screw image set as the selected screw image, extracting the area surrounded by the image pixel line in the selected screw image, extracting all pixel values ​​of the surrounded area from the pixel value data, and evenly dividing the surrounded area into 9 sub-areas, calculating the standard deviation, mean, and histogram of the pixel values ​​in each sub-area based on all the extracted pixel values, and using the set of the standard deviation, mean, and histogram as the surface quality identification set of the selected screw image, continuing to determine the surface quality identification set of the remaining screw images in the screw image set, and using all the surface quality identification sets of the remaining screw images in the screw image set. The quality recognition set comprises surface defect information of the target screw, wherein the surface defect information represents information of the surface defect of the target screw during the rotation process when tightened, and is used to judge the surface texture of the target screw in the screw image set; the screw defect domain of the target screw can be identified by the edge fluctuation cost and the surface defect information, which can be implemented in the following manner, namely: initializing a screw quality recognition model, taking the edge fluctuation cost as a constraint parameter of the screw quality recognition model, taking the surface defect information as an initialization parameter of the screw quality recognition model, and then obtaining the screw defect domain of the target screw through the screw quality recognition model, wherein the screw quality recognition model is a detection model of the screw defect domain established by a machine learning algorithm (such as a regression algorithm, a neural network, etc.), and the detection model is, for example: screw defect domain = surface defect information * A + edge fluctuation cost * B, wherein A and B are weight coefficients, A and B can be determined according to a large number of screw defect domains, and can also be determined in other ways in other embodiments, which are not limited here.

[0089] It should be noted that, in the present application, quality identification is performed on the pixel value data in the screw image set according to the edge fluctuation cost, that is: the surface defect information of the target screw is determined by the pixel value data and the image pixel line, and the screw defect domain of the target screw is identified by the edge fluctuation cost and the surface defect information, wherein the quality identification represents the detection of surface defects of the target screw after locking; the screw defect domain is an area reflecting defects in the target screw during the locking process (surface texture has defects such as unevenness and pits), which is used to judge the locking process of the target screw, eliminates small defects in the target screw during the locking process, and improves the detection quality of the locking of the target screw.

[0090] In step 104, the locking information of the target screw is obtained, and the thread engagement state of the target screw in the screw image set in the tightened state is identified through the locking information, and the detection confidence index of the target screw after locking is determined according to the identified thread engagement state and the screw defect domain.

[0091] It should be noted that the locking information in the present application includes the tightening force, tightening angle, etc. of the target screw, which is used to predict the tightening condition of the target screw and then analyze the tightening state of the target screw. As a preferred embodiment, the locking information of the target screw can be obtained from the screw database of the target screw, wherein the historical locking information of all screws of the same type in history is determined by the type of the target screw, and the average value of the tightening force in all the historical locking information is used as the tightening force in the locking information of the target screw, and the average value of the tightening angle in all the historical locking information is used as the tightening angle in the locking information of the target screw, and the obtained locking information is stored in the screw database of the target screw. In other embodiments, other methods can also be used to obtain it, which is not limited here.

[0092] In some embodiments, reference Figure 4 As shown, this figure is a schematic diagram of the process of determining the thread engagement state in some embodiments of the present application. In this embodiment, the thread engagement state of the target screw in the screw image set in the tightened state can be identified by the locking information in the screw image set by the following steps:

[0093] First, in step 1041, the angle characteristic quantity of the target screw is determined according to the locking information;

[0094] Secondly, in step 1042, the tightening state of the target screw is determined according to the screw image set;

[0095] Finally, in step 1043, the thread engagement state of the target screw in the screw image set in the tightened state is determined according to the angle feature value and the tightened state.

[0096] In specific implementation, determining the angle feature of the target screw according to the locking information can be achieved in the following manner, namely: performing a natural exponential operation on the opposite number of the tightening force in the locking information, multiplying the value obtained by the natural exponential operation by the tightening angle in the locking information, and using the multiplied value as the angle feature of the target screw, wherein the angle feature represents the parameter value of the angle condition of the target screw after tightening. When the angle feature is too large, it indicates that the deviation of the target screw after tightening is large, otherwise it indicates that the deviation of the target screw after tightening is small, which is used to analyze the locking condition of the target screw; determining the tightening state of the target screw according to the screw image set can be achieved in the following manner, namely: extracting the screw image corresponding to the last acquisition time in the screw image set, performing feature analysis on the screw image (analyzing the visibility and shape of the screw head and thread), and using the result of the feature analysis as the tightening state of the target screw, wherein the tightening state of the target screw represents the visibility and shape deformation of the screw head and thread of the target screw after tightening, and the tightened screw usually has a uniform uniform shape and non-deformed head, while loose screws have deformation or gaps; determining the thread engagement state of the target screw in the screw image set in the tightened state according to the angle feature and the tightening state can be achieved in the following manner, namely: obtaining the tightening state reference information of the target screw from the screw database of the target screw, wherein the tightening state reference information represents the visibility of the screw head and thread and the maximum deformation degree of the shape of the target screw after tightening; if the tightening state is within the range of the tightening state reference information, the angle feature is used as the characterization of the thread engagement state; if the tightening state is not within the range of the tightening state reference information, the angle feature is multiplied by the deformation amount of the target screw, and the multiplied value is used as the characterization of the thread engagement state, wherein in this embodiment, the deformation amount of the target screw can be obtained by the digital image correlation method in the prior art, and the deformation amount of the target screw represents the parameter value of the deformation degree of the target screw after tightening, which is used to analyze the situation of the target screw after tightening; in other embodiments, other methods can also be used to determine, which are not limited here.

[0097] It should be noted that the thread engagement state in the present application reflects the engagement of the thread of the target screw after tightening, and is used to judge the condition of the target screw after locking. It can be used to identify whether the locking of the target screw meets the requirements. The larger the characterization value of the thread engagement state, the worse the quality of the target screw after locking, and vice versa.

[0098] In some embodiments, determining the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect area can be achieved by the following steps:

[0099] Determining the engagement confidence of the target screw by using the identified thread engagement state;

[0100] Determining the defect ratio of the target screw after tightening according to the screw defect domain;

[0101] The detection confidence index of the target screw after locking is determined according to the occlusion confidence and the defect proportion.

[0102] In specific implementation, the bite confidence of the target screw can be determined by the identified thread bite state, which can be achieved in the following manner, namely: the inverse of the characterization quantity of the thread bite state is subjected to a natural exponential operation, and the value obtained by the natural exponential operation is used as the bite confidence of the target screw, wherein the bite confidence represents the parameter value of the confidence degree of the thread bite of the target screw after tightening, and the larger the bite confidence, the better the thread bite of the target screw after tightening, and vice versa, the worse the thread bite of the target screw after tightening; determining the defect ratio of the target screw after locking according to the screw defect domain can be achieved in the following manner, namely: obtaining the overall area of ​​the target thread, wherein the overall area represents the sum of the areas obtained by photographing the target thread at various angles during the tightening process, dividing the size of the overall area by the size of the screw defect domain, performing a logarithmic operation with a base of 2 on the value obtained by the division, and using the value obtained by the logarithmic operation as the defect ratio of the target screw after locking, wherein the The defect ratio represents a parameter value of the ratio of surface defects of the target screw after locking, and is used to analyze the surface condition of the target screw after locking; determining the detection confidence index of the target screw after locking according to the bite confidence and the defect ratio can be implemented in the following manner, namely: initializing a detection confidence index model, taking the bite confidence as a constraint parameter of the detection confidence index model, taking the defect ratio as an initial parameter of the detection confidence index model, and obtaining the detection confidence index of the target screw after locking through the detection confidence index model, wherein the detection confidence index model is a confidence index model that uses a machine learning algorithm (such as a regression algorithm, a neural network, etc.) to establish a detection confidence index, and the confidence index model is, for example: detection confidence index = defect ratio * C + bite confidence * D, wherein C and D are weight coefficients, C and D can be determined based on a large number of detection confidence indices, and can also be determined in other ways in other embodiments, which are not limited here.

[0103] It should be noted that the detection confidence index in the present application is a parameter value that reflects the confidence level of the detection of the target screw after locking, and is used to judge the condition of the target screw after locking. The larger the detection confidence index, the better the condition of the target screw after locking, and vice versa.

[0104] In step 105, the quality of the locked target screw is calibrated using the detection confidence index.

[0105] In some embodiments, the quality calibration of the target screw after locking by using the detection confidence index can be achieved by the following steps:

[0106] Determine the detection confidence threshold of the target screw

[0107] comparing the detection confidence index with the detection confidence threshold;

[0108] If the detection confidence index is greater than or equal to the detection confidence threshold, the quality of the target screw after locking is marked as qualified;

[0109] If the detection confidence index is less than the detection confidence threshold, the quality of the target screw after locking is marked as unqualified.

[0110] During specific implementation, the marked target screw is stored in a database of target screws.

[0111] It should be noted that in the present application, the detection confidence threshold can be set according to the specific needs of the target screw. For example, if the target screw is used in equipment in a military factory, the detection confidence threshold can be set in a high range. If the target screw is used in civilian equipment, the detection confidence threshold can be set in a low range. In other embodiments, for example, when the target screw is in a high-precision device, the detection confidence threshold can be set in a high range, thereby improving the tightening effect of the target screw.

[0112] In addition, in another aspect of the present application, in some embodiments, the present application provides a screw fastening quality detection system based on machine vision, referring to Figure 5 , which is a schematic diagram of exemplary hardware and / or software of a screw fastening quality detection system based on machine vision according to some embodiments of the present application. The screw fastening quality detection system based on machine vision 400 includes: an acquisition module 401, a processing module 402 and an execution module 403, which are described as follows:

[0113] Acquisition module 401, in this application, acquisition module 401 is mainly used to collect screw images during the tightening process of the target screw to obtain a screw image set;

[0114] Processing module 402, in the present application, is used to extract image edge information of a target screw from the screw image set, perform edge recognition on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, and thereby obtain an image pixel line of an edge contour of the target screw;

[0115] It should be noted that the processing module 402 in the present application is also used to determine the edge fluctuation cost of the target screw in the screw image set through the image pixel line and the light characteristics around the target screw, and perform quality identification on the pixel value data in the screw image set according to the edge fluctuation cost to obtain the screw defect domain of the target screw;

[0116] In addition, it should be noted that the processing module 402 in the present application is also used to obtain the locking information of the target screw, identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain;

[0117] Execution module 403, in the present application, execution module 403 is mainly used to calibrate the quality of the target screw after locking through the detection confidence index.

[0118] In addition, the present application also provides a computer device, which includes a memory and a processor, wherein the memory stores a code, and the processor is configured to obtain the code and execute the above-mentioned machine vision-based screw fastening quality detection method.

[0119] In some embodiments, reference Figure 6 , which is a schematic diagram of the structure of a computer device for implementing a screw tightening quality detection method based on machine vision according to some embodiments of the present application. The screw tightening quality detection method based on machine vision in the above embodiment can be performed by Figure 6 The computer device 500 shown in the figure is implemented, and the computer device 500 includes at least one processor 501, a communication bus 502, a memory 503 and at least one communication interface 504.

[0120] The processor 501 may be a general-purpose central processing unit (CPU) or an application specific integrated circuit (ASIC).

[0121] The communication bus 502 may be used to transmit information between the above-mentioned components.

[0122] The memory 503 may be a read only memory (ROM) or other types of static storage devices that can store static information and instructions, a random access memory (RAM) or other types of dynamic storage devices that can store information and instructions, or an electrically erasable programmable read only memory (EEPROM), a compact disc read only memory (CD ROM) or other optical disc storage, an optical disc storage (including a compressed optical disc, a laser disc, an optical disc, a digital versatile disc, a Blu-ray disc, etc.), a magnetic disk or other magnetic storage device, or any other medium that can be used to carry or store the desired program code in the form of an instruction or data structure and can be accessed by a computer, but is not limited thereto. The memory 503 may exist independently and be connected to the processor 501 via the communication bus 502. The memory 503 may also be integrated with the processor 501.

[0123] The memory 503 is used to store the program code for executing the solution of the present application, and the execution is controlled by the processor 501. The processor 501 is used to execute the program code stored in the memory 503. The program code may include one or more software modules. The method used in the above embodiment can be implemented by the processor 501 and one or more software modules in the program code in the memory 503.

[0124] The communication interface 504 uses any transceiver or other device for communicating with other devices or communication networks, such as Ethernet, radio access network (RAN), wireless local area networks (WLAN), etc.

[0125] In a specific implementation, as an embodiment, a computer device may include multiple processors, each of which may be a single-core (single CPU) processor or a multi-core (multi CPU) processor. The processor here may refer to one or more devices, circuits, and / or processing cores for processing data (such as computer program instructions).

[0126] The above-mentioned computer device may be a general-purpose computer device or a special-purpose computer device. In a specific implementation, the computer device may be a desktop computer, a portable computer, a network server, a personal digital assistant (PDA), a mobile phone, a tablet computer, a wireless terminal device, a communication device or an embedded device. The embodiment of the present application does not limit the type of computer device.

[0127] In addition, the present application also provides a computer-readable storage medium, which stores a computer program. When the computer program is executed by a processor, the above-mentioned machine vision-based screw fastening quality detection method is implemented.

[0128] Although the preferred embodiments of the present application have been described, those skilled in the art may make other changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications falling within the scope of the present application.

[0129] Obviously, those skilled in the art can make various changes and modifications to the present application without departing from the spirit and scope of the present application. Thus, if these modifications and variations of the present application fall within the scope of the claims of the present application and their equivalents, the present application is also intended to include these modifications and variations.

Claims

1. A screw fastening quality inspection method based on machine vision, characterized in that: The steps include: Collecting screw images during the tightening process of the target screws to obtain a screw image set; Extracting image edge information of a target screw from the screw image set, performing edge recognition on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, and thereby obtaining an image pixel line of an edge contour of the target screw; Determine the edge fluctuation cost of the target screw in the screw image set by using the image pixel line and the light features around the target screw, perform quality identification on the pixel value data in the screw image set according to the edge fluctuation cost, and obtain the screw defect domain of the target screw; Acquire the locking information of the target screw, identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain; The quality of the target screw after locking is calibrated by the detection confidence index.

2. The method according to claim 1, characterized in that Extracting the image edge information of the target screw from the screw image set specifically includes: Extracting image contour features of each screw image in the screw image set; Determine the image edge features of the target screw according to all image contour features; The image edge information of the target screw is determined by using the image edge features and the screw image set.

3. The method according to claim 1, characterized in that Performing edge recognition on a target screw in the screw image set according to the image edge information and the pixel point features of the screw image set, and then obtaining an image pixel line of the target screw edge contour specifically includes: Obtaining pixel features of the screw image set; Determine edge recognition information of a target screw according to the image edge information and pixel features of the screw image set; determining a plurality of image contour lines according to the edge recognition information and the screw image set; The image pixel line of the target screw edge contour is determined through all image contour lines.

4. The method according to claim 1, characterized in that Determining the edge fluctuation cost of the target screw in the screw image set by using the image pixel line and the light features around the target screw specifically includes: Get the light characteristics around the target screw; determining a light fluctuation entropy of the light characteristic; determining confidence fluctuation information according to the light fluctuation entropy and the image pixel line; The edge fluctuation cost of the target screw in the screw image set is determined according to the confidence fluctuation information.

5. The method according to claim 1, characterized in that The quality of the pixel value data in the screw image set is identified according to the edge fluctuation cost, and the screw defect domain of the target screw is obtained, which specifically includes: Acquire pixel value data and image pixel lines in the screw image set; Determine the surface defect information of the target screw by using the pixel value data and the image pixel line; The screw defect area of ​​the target screw is identified by the edge fluctuation cost and the surface defect information.

6. The method according to claim 1, characterized in that Identifying the thread engagement state of the target screw in the screw image set in the tightened state by using the locking information specifically includes: Determine an angle characteristic quantity of a target screw according to the locking information; determining a tightening state of a target screw according to the screw image set; The thread engagement state of the target screw in the tightened state in the screw image set is determined according to the angle feature amount and the tightened state.

7. The method according to claim 1, characterized in that Determining the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect area specifically includes: Determining the engagement confidence of the target screw by using the identified thread engagement state; Determining the defect ratio of the target screw after tightening according to the screw defect domain; The detection confidence index of the target screw after locking is determined according to the occlusion confidence and the defect proportion.

8. A screw tightening quality inspection system based on machine vision, characterized in that: include: An acquisition module is used to collect screw images during the tightening process of the target screw to obtain a screw image set; A processing module, used to extract image edge information of a target screw from the screw image set, perform edge recognition on the target screw in the screw image set according to the image edge information and pixel point features of the screw image set, and thereby obtain an image pixel line of an edge contour of the target screw; The processing module is further used to determine the edge fluctuation cost of the target screw in the screw image set through the image pixel line and the light characteristics around the target screw, and perform quality identification on the pixel value data in the screw image set according to the edge fluctuation cost to obtain the screw defect domain of the target screw; The processing module is further used to obtain the locking information of the target screw, identify the thread engagement state of the target screw in the screw image set in the tightened state through the locking information, and determine the detection confidence index of the target screw after locking according to the identified thread engagement state and the screw defect domain; The execution module is used to calibrate the quality of the target screw after locking through the detection confidence index.

9. A computer device, characterized in that: The computer device includes a memory and a processor, the memory stores codes, and the processor is configured to obtain the codes and execute the machine vision-based screw fastening quality detection method according to any one of claims 1 to 7.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting screw fastening quality based on machine vision according to any one of claims 1 to 7 is implemented.

Images