Door lock mechanism defect detection method and system based on image recognition
By using image recognition-based methods to collect and segment the lock tongue image sequence, and combining it with a machine learning classifier, efficient and accurate non-contact defect detection of smart door lock mechanisms is achieved, solving the problems of low efficiency and poor accuracy in existing technologies.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- SHENZHEN LOCSTAR TECH
- Filing Date
- 2026-04-22
- Publication Date
- 2026-07-21
AI Technical Summary
In existing technologies, the detection efficiency of defects in smart door lock mechanisms is low, highly subjective, and inconsistent. Furthermore, sensor-based detection methods require contact with the product, affecting its movement and leading to inaccurate detection results.
By employing an image recognition-based method, a sequence of lock tongue images is acquired through various preset driving conditions. Lock tongue features are analyzed in segments, and a machine learning classifier is used to identify defects, thus achieving non-contact detection.
It improves detection efficiency, consistency, and accuracy, enabling it to comprehensively capture various defects, accurately locate defect positions, and meet the needs of large-scale production.
Smart Images
Figure CN122435320A_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of defect recognition technology, and more specifically, to a method and system for detecting defects in door lock mechanisms based on image recognition. Background Technology
[0002] With the rapid development of the smart home industry, smart door locks, as the first line of defense for home security, are experiencing continuous market demand growth. Smart door locks integrate technologies such as electronic control, biometrics, and wireless communication to provide users with convenient and secure access control solutions. Internally, smart door locks contain multiple cooperating mechanisms, such as drive mechanisms, transmission mechanisms, guide rail mechanisms, and spring mechanisms, which work together to achieve the extension and retraction of the latch. In the manufacturing process of smart door locks, quality inspection of the lock mechanism is a crucial step in ensuring product qualification rate and reliability. Potential defects in lock mechanisms are diverse, including abnormal overall resistance, localized jamming, directional defects, guide rail abnormalities, excessive assembly gaps, and transmission mechanism abnormalities.
[0003] Currently, defect detection methods for door lock mechanisms include: manual inspection, where inspectors judge defects by observing the movement of the latch, listening to the sound of movement, and manually feeling the resistance. However, manual inspection suffers from low efficiency, high subjectivity, and poor consistency, making it difficult to meet the quality control requirements of large-scale production and prone to missed detections. Sensor-based detection involves installing force sensors and displacement sensors on the door lock mechanism to measure the mechanical and positional parameters of the latch movement. However, this method requires installing sensors on the product under test or physical contact with the product, making the detection process complex. Furthermore, sensor installation may affect the normal movement of the latch, leading to inaccurate test results. Summary of the Invention
[0004] To overcome the aforementioned problems in the prior art, this invention proposes a method and system for detecting defects in door lock mechanisms based on image recognition, which is used to solve the above problems.
[0005] This invention provides the following technical solution: Image recognition-based methods for detecting defects in door lock mechanisms include: Multiple driving conditions are preset, and for each driving condition, the image sequence of the locking tongue during the driving time is collected to obtain the original image sequence; Extract the latch portion image from the original image sequence to obtain the latch image sequence, and segment the latch image sequence based on preset stroke segmentation points to obtain the segmented latch image sequence; Based on each segmented latch image sequence, the corresponding segment features are calculated, and deviation analysis is performed between each segment feature and the pre-stored standard feature set to obtain the deviation features of each segment. The deviation features of each segment under each driving condition are fused to obtain the fused features of each segment. The fused features of each segment are then input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.
[0006] Preferably, the driving conditions include combinations of different driving speeds and different driving directions; The preset multiple driving conditions include at least the following: First driving condition: driving the bolt to move in the extension direction at the rated driving speed; Second driving condition: driving the bolt to move in the extension direction at a slower speed than the rated driving speed; Third driving condition: driving the bolt to move in the retraction direction at the rated driving speed; Fourth driving condition: driving the bolt to move in the retraction direction at a slower speed.
[0007] Preferably, the step of extracting the latch portion image from the original image sequence to obtain the latch image sequence includes: performing binarization processing on the images in the original image sequence, separating the latch region from the background region, and extracting the latch region image to form the latch image sequence.
[0008] Preferably, the preset travel segment points are preset percentage values of the travel; The segmented latch image sequence obtained by segmenting the latch image sequence based on preset stroke segmentation points includes: Edge detection is performed on each frame of the latch image sequence. By comparing the positional changes of each edge of the latch in each frame, edges that remain unchanged in position are identified as fixed edges, and non-fixed edges are identified as moving edges. Calculate the number of pixels between moving edges and fixed edges in each frame image, and use it as the travel pixel value for each frame; Take the maximum value of all frame travel pixel values in the latch image sequence as the full travel reference value; Calculate the ratio of the travel pixel value of each frame to the full travel reference value to obtain the travel percentage for each frame; For each travel segment point, the frame in which the travel percentage first reaches that travel segment point is retrieved in chronological order and used as the boundary frame; The image frames between two adjacent boundary frames are divided into segments to form a segmented latch image sequence.
[0009] Preferably, the segmentation features include segment transit time, intra-segment velocity fluctuation, and intra-segment tremor feature vector; The method for calculating the segment passage time includes: taking the difference between the timestamp of the end boundary frame of the segment and the timestamp of the start boundary frame as the segment passage time; The method for calculating the speed fluctuation within the segment includes: calculating the instantaneous speed of each frame based on the difference between the travel percentage of adjacent frames within the segment and the difference between the timestamps, and taking the difference between the maximum and minimum instantaneous speeds of each frame as the speed fluctuation within the segment. The method for calculating the intra-segment jitter feature vector includes: performing edge detection on each frame of the latch image within the segment, calculating the edge sharpness value and gradient direction consistency value of the edge region of each frame respectively; forming sharpness time-series data based on the edge sharpness value of each frame, and forming direction consistency time-series data based on the gradient direction consistency value of each frame; performing time-series statistical analysis on the sharpness time-series data and the direction consistency time-series data respectively, and combining the obtained statistical features to form the intra-segment jitter feature vector.
[0010] Preferably, the method for calculating the edge sharpness value includes: performing edge detection on the latch image and extracting the latch edge region; calculating the gradient magnitude of each pixel in the edge region, and taking the average of the gradient magnitudes as the edge sharpness value of the frame; The method for calculating the gradient direction consistency value includes: calculating the gradient vector of each pixel in the edge region, normalizing each gradient vector to obtain a unit direction vector; calculating the vector sum of all unit direction vectors, and taking the ratio of the magnitude of the vector sum to the number of edge pixels as the gradient direction consistency value of the frame. The time-series statistical analysis of sharpness time-series data and direction consistency time-series data includes: calculating the sharpness mean, sharpness standard deviation, and sharpness range for the sharpness time-series data; calculating the consistency mean and consistency standard deviation for the direction consistency time-series data; and combining the sharpness mean, sharpness standard deviation, sharpness range, consistency mean, and consistency standard deviation to form an intra-segment jitter feature vector.
[0011] Preferably, the standard feature set is pre-established based on the same travel segment points and includes the standard segment passage time, speed fluctuation within the standard segment, and tremor feature vector within the standard segment for each segment; The deviation analysis based on each segment feature and the pre-stored standard feature set includes: For each segment, the segment passage time deviation is calculated based on the segment passage time and the standard segment passage time; the speed fluctuation deviation is calculated based on the intra-segment speed fluctuation and the standard segment speed fluctuation; and the flutter deviation is calculated based on the intra-segment flutter feature vector and the standard segment flutter feature vector. The deviation characteristics of a segment are formed by combining time deviation, velocity fluctuation deviation, and flutter deviation.
[0012] Preferably, the process of fusing the deviation characteristics of each segment under each driving condition to obtain the fused characteristics of each segment includes: For each segment, the deviation features of that segment are extracted from the deviation features under each driving condition to form a multi-condition deviation set for that segment; Vector concatenation of deviation features from multiple operating conditions is performed to form the fusion feature of that segment.
[0013] Preferably, the training method for the defect classifier includes: Collect bolt image sequences of multiple defective door lock samples under various driving conditions; The image sequences of each door lock sample are processed by extracting and segmenting the latch image, calculating the features of each segment, performing deviation analysis based on the standard feature set to obtain the deviation features, and fusing the deviation features under each driving condition to obtain the fused features of each segment. Based on the actual defects of the door lock samples, the corresponding defect category labels are labeled for each segment of the fused features to form a training sample set. A defect classifier is obtained by training a machine learning classification model based on a training sample set.
[0014] This invention also provides an image recognition-based door lock mechanism defect detection system, used to implement an image recognition-based door lock mechanism defect detection method, including: The image acquisition module is used to preset multiple driving conditions, and for each driving condition, it acquires an image sequence of the locking tongue within the driving time to obtain the original image sequence; The image segmentation module is used to extract the latch portion image from the original image sequence to obtain the latch image sequence, and to segment the latch image sequence based on preset stroke segmentation points to obtain the segmented latch image sequence. The feature analysis module is used to calculate the corresponding segment features based on each segment of the latch image sequence, and to perform deviation analysis based on the features of each segment and the pre-stored standard feature set to obtain the deviation features of each segment. The defect identification module is used to fuse the deviation features of each segment under each driving condition to obtain the fused features of each segment. The fused features of each segment are then input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.
[0015] This invention provides a method and system for detecting defects in door lock mechanisms based on image recognition, which has the following beneficial effects: By analyzing images of the latch's movement, a non-contact testing method is used, eliminating the need for sensors on the product or physical contact. This method does not affect the normal movement of the latch, resulting in more accurate and reliable test results. Since the latch's movement is the ultimate reflection of the coordinated operation of various internal mechanisms, analyzing the latch images can indirectly reflect the working status of multiple internal mechanisms such as the drive mechanism, transmission mechanism, guide rail mechanism, and spring mechanism, enabling comprehensive testing of the lock's mechanisms. Furthermore, the entire testing process is based on image acquisition and automatic analysis, requiring no manual intervention. This results in high testing efficiency and consistency, meeting the quality control requirements of large-scale production.
[0016] Detection is performed under multiple preset driving conditions, and the latch image sequence is segmented based on preset stroke segmentation points. Segmentation features are calculated for each segment, and the deviation features under each driving condition are fused and input into a pre-trained defect classifier for identification. Since different types of defects exhibit different characteristics under different driving conditions, multi-condition detection and feature fusion can comprehensively capture the performance characteristics of various defects, effectively reducing the false negative rate. Because the stroke segmentation points can be set according to the structural characteristics of the door lock mechanism, each segment has a corresponding relationship with the physical structural area of the door lock mechanism, thus accurately locating the stroke position of the defect. By using a machine learning classifier to automatically learn the feature patterns of various defects from training samples, different defect types can be accurately distinguished. Attached Figure Description
[0017] Figure 1 This is a flowchart illustrating the image recognition-based door lock mechanism defect detection method of the present invention. Figure 2 This is a schematic diagram of the module of the image recognition-based door lock mechanism defect detection system of the present invention. Detailed Implementation
[0018] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0019] Please refer to Example 1 Figure 1 In this embodiment, the image recognition-based door lock mechanism defect detection method includes: S1. Preset multiple driving conditions, and for each driving condition, collect the image sequence of the locking tongue during the driving time to obtain the original image sequence; The driving conditions include combinations of different driving speeds and different driving directions; The preset multiple driving conditions include at least the following: First driving condition: driving the bolt to move in the extension direction at the rated driving speed; Second driving condition: driving the bolt to move in the extension direction at a slower speed than the rated driving speed; Third driving condition: driving the bolt to move in the retraction direction at the rated driving speed; Fourth driving condition: driving the bolt to move in the retraction direction at a slower speed.
[0020] In this embodiment, it should be noted that the driving condition refers to the combination of control conditions that drive the bolt movement. Different driving conditions will cause the lock mechanism to operate in different working states, thereby exposing different types of defects. The rated driving speed refers to the standard working speed specified in the lock mechanism's design specifications, and the slow speed is usually set to 50% to 70% of the rated speed. The reason for using different speeds for testing is that: at high speeds, the inertial force is greater, which easily exposes the clearance and jamming problems of the transmission mechanism; at low speeds, the frictional force has a relatively greater impact, which easily exposes poor lubrication and abnormal resistance problems.
[0021] It should be noted that because the locking mechanism experiences different force states in the two directions, the performance of unidirectional components such as springs and ratchet may differ in different directions. Certain direction-related defects, such as spring fatigue or unidirectional bearing wear, can only be detected when tested in a specific direction. Therefore, the extension and retraction directions are tested separately. The original image sequence refers to the set of image frames that can be continuously captured by a fixed-mount industrial camera during a complete lock tongue movement.
[0022] S2. Extract the latch portion image from the original image sequence to obtain the latch image sequence, and segment the latch image sequence based on the preset stroke segmentation points to obtain the segmented latch image sequence. The step of extracting the latch portion image from the original image sequence to obtain the latch image sequence includes: performing binarization processing on the images in the original image sequence, separating the latch region from the background region, and extracting the latch region image to form the latch image sequence.
[0023] The preset travel segment points are preset percentage values of the travel; The segmented latch image sequence obtained by segmenting the latch image sequence based on preset stroke segmentation points includes: Edge detection is performed on each frame of the latch image sequence. By comparing the positional changes of each edge of the latch in each frame, edges that remain unchanged in position are identified as fixed edges, and non-fixed edges are identified as moving edges. Calculate the number of pixels between moving edges and fixed edges in each frame image, and use it as the travel pixel value for each frame; Take the maximum value of all frame travel pixel values in the latch image sequence as the full travel reference value; Calculate the ratio of the travel pixel value of each frame to the full travel reference value to obtain the travel percentage for each frame; For each travel segment point, the frame in which the travel percentage first reaches that travel segment point is retrieved in chronological order and used as the boundary frame; The image frames between two adjacent boundary frames are divided into segments to form a segmented latch image sequence.
[0024] In this embodiment, it should be noted that binarization is the process of converting a grayscale image into an image with only black and white pixel values. Since the latch is typically made of metal, it differs significantly from the background in grayscale or color. Therefore, a suitable grayscale threshold can be set to separate the latch area from the background. The binarization threshold can be automatically determined using the maximum inter-class variance method, or it can be pre-calibrated based on the actual shooting environment. Extracting the latch area image eliminates background interference, making subsequent edge detection and feature calculation more accurate.
[0025] It should be noted that stroke segmentation points are pre-set percentage values used to divide the complete movement of the latch into multiple segments. Because the mechanical structure and force characteristics of the locking mechanism differ in different stroke regions—for example, the initial segment of the latch may be significantly affected by spring preload, the middle segment moves more smoothly, and the final segment may be affected by the limiting mechanism—stroke segmentation points are usually set non-uniformly based on the structural characteristics of the locking mechanism. For example, stroke segmentation points can be set to 1%, 20%, 50%, 80%, and 99%, dividing the entire stroke into four segments: the initial segment (1%–20%), the front-middle segment (20%–50%), the rear-middle segment (50%–80%), and the final segment (80%–99%). This allows for targeted detection based on the characteristics of different regions. The purpose of segmentation is to achieve precise defect location; when an abnormality occurs in a certain segment, the specific area of the latch's stroke can be directly located.
[0026] It should be noted that edge detection is used to identify edge contours in the latch image. Commonly used edge detection algorithms include the Canny operator and the Sobel operator. The detected edges include all boundaries of the latch, and it is necessary to further distinguish between fixed edges and moving edges. Since the side of the latch connected to the lock body remains in a constant position during movement, while the end of the latch changes position with the extension and retraction of the latch, by comparing the edge positions of each frame in the image sequence, edges with position changes less than a preset threshold can be identified as fixed edges, while edges with significant position changes can be identified as moving edges.
[0027] It should be noted that the stroke pixel value refers to the number of pixels along the direction of motion of the moving edge, reflecting the extent to which the latch extends in that frame of the image. This value can be obtained by counting the pixel length occupied by the moving edge along the direction of motion.
[0028] It should be noted that due to differences in absolute pixel values caused by different detection devices and different installation locations, the maximum value of the stroke pixel value of all frames in the latch image sequence is taken as the full stroke reference value. This value is then used as a reference for normalization to make the stroke percentage comparable.
[0029] It should be noted that a boundary frame refers to the image frame corresponding to the point when the latch movement reaches a certain travel segment. Since the latch movement is unidirectional, boundary frames are retrieved sequentially over time, taking the frame where the travel percentage first reaches that segment point to ensure the correct temporal sequence of the segments. All image frames between two adjacent boundary frames constitute a segmented latch image sequence.
[0030] S3. Based on each segmented latch image sequence, calculate the corresponding segment features, and perform deviation analysis between each segment feature and the pre-stored standard feature set to obtain the deviation features of each segment. The segmentation features include segment transit time, intra-segment velocity fluctuation, and intra-segment tremor feature vector; The method for calculating the segment passage time includes: taking the difference between the timestamp of the end boundary frame of the segment and the timestamp of the start boundary frame as the segment passage time; The method for calculating the speed fluctuation within the segment includes: calculating the instantaneous speed of each frame based on the difference between the travel percentage of adjacent frames within the segment and the difference between the timestamps, and taking the difference between the maximum and minimum instantaneous speeds of each frame as the speed fluctuation within the segment. The method for calculating the intra-segment jitter feature vector includes: performing edge detection on each frame of the latch image within the segment, calculating the edge sharpness value and gradient direction consistency value of the edge region of each frame respectively; forming sharpness time-series data based on the edge sharpness value of each frame, and forming direction consistency time-series data based on the gradient direction consistency value of each frame; performing time-series statistical analysis on the sharpness time-series data and the direction consistency time-series data respectively, and combining the obtained statistical features to form the intra-segment jitter feature vector.
[0031] The method for calculating the edge sharpness value includes: performing edge detection on the latch image and extracting the latch edge region; calculating the gradient magnitude of each pixel in the edge region and taking the average of the gradient magnitudes as the edge sharpness value of the frame. The method for calculating the gradient direction consistency value includes: calculating the gradient vector of each pixel in the edge region, normalizing each gradient vector to obtain a unit direction vector; calculating the vector sum of all unit direction vectors, and taking the ratio of the magnitude of the vector sum to the number of edge pixels as the gradient direction consistency value of the frame. The time-series statistical analysis of sharpness time-series data and direction consistency time-series data includes: calculating the sharpness mean, sharpness standard deviation, and sharpness range for the sharpness time-series data; calculating the consistency mean and consistency standard deviation for the direction consistency time-series data; and combining the sharpness mean, sharpness standard deviation, sharpness range, consistency mean, and consistency standard deviation to form an intra-segment jitter feature vector.
[0032] The standard feature set is pre-established based on the same travel segment points and includes the standard segment passage time, speed fluctuation within the standard segment, and tremor feature vector within the standard segment for each segment. The deviation analysis based on each segment feature and the pre-stored standard feature set includes: For each segment, the segment passage time deviation is calculated based on the segment passage time and the standard segment passage time; the speed fluctuation deviation is calculated based on the intra-segment speed fluctuation and the standard segment speed fluctuation; and the flutter deviation is calculated based on the intra-segment flutter feature vector and the standard segment flutter feature vector. The deviation characteristics of a segment are formed by combining time deviation, velocity fluctuation deviation, and flutter deviation.
[0033] In this embodiment, it should be noted that the segmented features are a set of feature values used to quantitatively describe the motion state of the bolt within a certain segment. Since the segment characterizes the bolt's motion state from three dimensions—motion speed, speed stability, and trajectory stability—through time, intra-segment speed fluctuation, and intra-segment vibration feature vectors, respectively, it can comprehensively reflect the working status of the locking mechanism. Therefore, these three types of features are selected as segmented features.
[0034] It should be noted that the passage time of a segment reflects the speed at which the bolt passes through that segment. A shorter passage time indicates a faster movement speed, and a longer passage time indicates a slower movement speed. Since different segments correspond to different mechanical structure areas—for example, the initial segment may have a slower movement speed and a longer passage time due to the spring preload, the middle segment may have a smoother movement and a faster movement speed and a shorter passage time, and the final segment may have a reduced speed and an increased passage time due to the limiting buffer—there are differences in the standard passage time for each segment. If the passage time of a segment deviates significantly from the standard value, it may indicate abnormal resistance or a transmission fault in that area. Speed fluctuation within a segment reflects the stability of the bolt's movement speed within that segment. The instantaneous speed is calculated by dividing the difference in the percentage of travel between two adjacent frames by the difference in timestamps; its physical meaning is the change in the percentage of bolt travel per unit time. Speed fluctuation within a segment is the range of all instantaneous speeds within that segment, i.e., the difference between the maximum and minimum instantaneous speeds. Excessive speed fluctuations may indicate problems such as backlash in the transmission mechanism, gear wear, or unstable motor control. Therefore, speed fluctuations can serve as an important indicator of the stability of the bolt's movement speed. The intra-segment chatter feature vector reflects the stability of the bolt's trajectory within that segment. Since chatter is a small vibration that deviates from the ideal straight trajectory during bolt movement, it is usually caused by factors such as mechanism backlash, bearing wear, or improper installation. Furthermore, chatter can cause blurred edges or inconsistent directions in the bolt image. Therefore, analyzing edge features can indirectly reflect trajectory stability.
[0035] It should be noted that the edge sharpness value reflects the clarity of the latch edge. The calculation method involves calculating the gradient magnitude of each pixel in the edge region and taking the average as the edge sharpness value for that frame. The gradient magnitude can be calculated using the Sobel operator or other differential operators. Since the latch edge should be clear and sharp when moving along a straight trajectory under normal circumstances, and motion blur caused by jitter can lead to blurred edges, the edge sharpness value can serve as an important indicator for trajectory stability detection.
[0036] It should be noted that the gradient direction consistency value reflects the uniformity of the gradient directions of edge pixels. The calculation method first calculates the gradient vector of each edge pixel, normalizes each gradient vector to a unit direction vector, and then calculates the vector sum of all unit direction vectors. The ratio of the magnitude of the vector sum to the number of edge pixels is then taken. This ratio is 1 when all gradient directions are completely consistent, and approaches 0 when the gradient directions are completely randomly distributed. Since the edges should be straight and the gradient directions highly consistent when the latch moves along a straight trajectory under normal circumstances, jagged edges or fluctuations can occur when there is jaggedness, leading to a decrease in gradient direction consistency. Therefore, the gradient direction consistency value can be used as an auxiliary indicator for trajectory stability detection. The sharpness time-series data and the direction consistency time-series data are data sequences formed by arranging the edge sharpness values and gradient direction consistency values of each frame within this segment in chronological order. Since the time-series data contains dynamic changes in jitter, statistical analysis of these two time-series data can extract statistical features describing trajectory stability.
[0037] It should be noted that time-series statistical analysis includes calculating statistics such as mean, standard deviation, and range. The sharpness mean reflects the overall sharpness of the edges within the segment, while the sharpness standard deviation and sharpness range reflect the fluctuation in edge sharpness. The consistency mean reflects the overall consistency of the edge direction, and the consistency standard deviation reflects the fluctuation in the consistency of the edge direction. These five statistics combine to form an intra-segment jitter feature vector, used to quantitatively describe the stability of the latch movement trajectory within the segment.
[0038] It should be noted that the standard feature set is a pre-established reference value for each segment feature based on qualified samples. Since the standard feature set needs to use the same segmentation method as the sample to be tested to be comparable, it is established based on the same travel segmentation points. The establishment method involves selecting multiple door lock samples that have passed quality inspection, acquiring images under the same driving conditions and travel segmentation points, calculating segment features, and taking the average of the corresponding segment features for each sample as the standard value.
[0039] It should be noted that deviation analysis compares the segmented characteristics of the sample to be tested with a standard feature set to calculate the degree of difference. Segment deviation can usually be calculated by ratio of time deviation and velocity fluctuation deviation. The method for calculating jitter deviation is to calculate the ratio of the jitter feature vector within the segment of the sample to the jitter feature vector within the standard segment in each dimension, resulting in a deviation vector containing multiple components (ratio of sharpness mean, ratio of sharpness standard deviation, etc.).
[0040] S4. The deviation features of each segment under each driving condition are fused to obtain the fused features of each segment. The fused features of each segment are input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.
[0041] The process of fusing the deviation characteristics of each segment under each driving condition to obtain the fused characteristics of each segment includes: For each segment, the deviation features of that segment are extracted from the deviation features under each driving condition to form a multi-condition deviation set for that segment; Vector concatenation of deviation features from multiple operating conditions is performed to form the fusion feature of that segment.
[0042] Collect bolt image sequences of multiple defective door lock samples under various driving conditions; The image sequences of each door lock sample are processed by extracting and segmenting the latch image, calculating the features of each segment, performing deviation analysis based on the standard feature set to obtain the deviation features, and fusing the deviation features under each driving condition to obtain the fused features of each segment. Based on the actual defects of the door lock samples, the corresponding defect category labels are labeled for each segment of the fused features to form a training sample set. A defect classifier is obtained by training a machine learning classification model based on a training sample set.
[0043] In this embodiment, it should be noted that the multi-condition deviation set refers to the set of deviation characteristics obtained for the same segment under different driving conditions. For example, for the initial segment (1%-20%), its multi-condition deviation set includes the deviation characteristics obtained for the segment under the first driving condition, the second driving condition, the third driving condition, and the fourth driving condition, respectively.
[0044] It should be noted that vector concatenation is the operation of joining multiple vectors end-to-end to form a longer vector. Because vector concatenation preserves complete information about the deviation features under each driving condition, the classifier can learn the combination patterns of deviation modes under different driving conditions.
[0045] It should be noted that because different types of defects exhibit different behavior patterns under different driving conditions, the fused feature vector contains rich information on multi-condition deviations. For example, defects with overall drag anomalies show consistent deviations in passage time under various conditions, while defects with localized jamming exhibit more significant deviations in speed fluctuations under high-speed conditions, and direction-related defects show significant differences in deviations between the extension and retraction directions. These pattern differences provide the classifier with a basis for distinguishing different defect types.
[0046] It's important to note that the defect classifier is a pre-trained machine learning classification model used to determine the defect type based on the fused feature vector. Optional classification models include Support Vector Machines (SVM), Random Forests, Gradient Boosting Trees (XGBoost), or neural networks. The classifier's input is the fused feature vector for a given segment, and its output is the defect category label corresponding to that segment.
[0047] It should be noted that constructing the training sample set requires collecting multiple door lock samples containing different types of defects. These defective samples can originate from non-conforming products on the production line, man-made defective samples, or samples that have undergone accelerated aging tests. Each defective sample needs to be manually inspected to confirm its defect type and to mark the segment location of the defect.
[0048] It should be noted that, since the training samples and the samples to be detected need to use the same feature extraction process to ensure that the feature distribution is consistent, the same processing process is used when processing the image sequences of each door lock sample: lock tongue image extraction, segmentation, feature calculation, deviation analysis and feature fusion.
[0049] It should be noted that defect category labels are classification identifiers for defect types. Common defect categories include, but are not limited to: normal, abnormal overall resistance, local jamming, direction-related defects, guide rail abnormalities, excessive assembly clearance, and transmission mechanism abnormalities. The labeling system can be expanded or adjusted according to actual production needs.
[0050] It should be noted that the classification model can be trained using supervised learning methods. The training sample set is divided into a training set and a validation set according to a certain ratio. The training set is used to optimize the model parameters, and the validation set is used to evaluate the model performance and adjust hyperparameters. The trained model is then the defect classifier, which can be deployed in a detection system for online defect identification.
[0051] It should be noted that the defect identification results are defect category labels corresponding to each segment. Since the stroke segment points are set based on the structural characteristics of the lock mechanism, each segment corresponds to a physical structural area of the lock mechanism. By summarizing the identification results of each segment, a defect diagnosis report for the entire door lock sample can be obtained, including the defect location (which segments have defects and their corresponding mechanical areas) and the defect type (the defect category corresponding to each abnormal segment). This report can be used to guide quality control and product sorting on the production line.
[0052] Please refer to Example 2 Figure 2 This invention provides a door lock mechanism defect detection system based on image recognition, used to implement a door lock mechanism defect detection method based on image recognition, including: The image acquisition module is used to preset multiple driving conditions, and for each driving condition, it acquires an image sequence of the locking tongue within the driving time to obtain the original image sequence; The image segmentation module is used to extract the latch portion image from the original image sequence to obtain the latch image sequence, and to segment the latch image sequence based on preset stroke segmentation points to obtain the segmented latch image sequence. The feature analysis module is used to calculate the corresponding segment features based on each segment of the latch image sequence, and to perform deviation analysis based on the features of each segment and the pre-stored standard feature set to obtain the deviation features of each segment. The defect identification module is used to fuse the deviation features of each segment under each driving condition to obtain the fused features of each segment. The fused features of each segment are then input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.
[0053] In the several embodiments provided by this invention, it should be understood that the disclosed systems, apparatuses, and methods can be implemented in other ways. For example, the apparatus embodiments described above are merely illustrative; for instance, the division of units is only one method, and in actual implementation, there may be other division methods. For example, multiple units or components may be combined or integrated into another system, or some features may be ignored or not executed. Furthermore, the coupling or direct coupling or communication connection shown or discussed may be through some interfaces; the indirect coupling or communication connection between apparatuses or units may be electrical, mechanical, or other forms.
[0054] The above description is merely a specific embodiment of the present invention, but the scope of protection of the present invention is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention.
[0055] In conclusion, the above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.
Claims
1. A method for detecting defects in door lock mechanisms based on image recognition, characterized in that, include: Multiple driving conditions are preset, and for each driving condition, the image sequence of the locking tongue during the driving time is collected to obtain the original image sequence; Extract the latch portion image from the original image sequence to obtain the latch image sequence, and segment the latch image sequence based on preset stroke segmentation points to obtain the segmented latch image sequence; Based on each segmented latch image sequence, the corresponding segment features are calculated, and deviation analysis is performed between each segment feature and the pre-stored standard feature set to obtain the deviation features of each segment. The deviation features of each segment under each driving condition are fused to obtain the fused features of each segment. The fused features of each segment are then input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.
2. The method for detecting defects in door lock mechanisms based on image recognition according to claim 1, characterized in that, The driving conditions include combinations of different driving speeds and different driving directions; The preset multiple driving conditions include at least the following: First driving condition: driving the bolt to move in the extension direction at the rated driving speed; Second driving condition: driving the bolt to move in the extension direction at a slower speed than the rated driving speed; Third driving condition: driving the bolt to move in the retraction direction at the rated driving speed; Fourth driving condition: driving the bolt to move in the retraction direction at a slower speed.
3. The method for detecting defects in door lock mechanisms based on image recognition according to claim 1, characterized in that, The step of extracting the latch portion image from the original image sequence to obtain the latch image sequence includes: performing binarization processing on the images in the original image sequence, separating the latch region from the background region, and extracting the latch region image to form the latch image sequence.
4. The method for detecting defects in door lock mechanisms based on image recognition according to claim 1, characterized in that, The preset travel segment points are preset percentage values of the travel; The segmented latch image sequence obtained by segmenting the latch image sequence based on preset stroke segmentation points includes: Edge detection is performed on each frame of the latch image sequence. By comparing the positional changes of each edge of the latch in each frame, edges that remain unchanged in position are identified as fixed edges, and non-fixed edges are identified as moving edges. Calculate the number of pixels between moving edges and fixed edges in each frame image, and use it as the travel pixel value for each frame; Take the maximum value of all frame travel pixel values in the latch image sequence as the full travel reference value; Calculate the ratio of the travel pixel value of each frame to the full travel reference value to obtain the travel percentage for each frame; For each travel segment point, the frame in which the travel percentage first reaches that travel segment point is retrieved in chronological order and used as the boundary frame; The image frames between two adjacent boundary frames are divided into segments to form a segmented latch image sequence.
5. The method for detecting defects in door lock mechanisms based on image recognition according to claim 4, characterized in that, The segmentation features include segment transit time, intra-segment velocity fluctuation, and intra-segment tremor feature vector; The method for calculating the segment passage time includes: taking the difference between the timestamp of the end boundary frame of the segment and the timestamp of the start boundary frame as the segment passage time; The method for calculating the speed fluctuation within the segment includes: calculating the instantaneous speed of each frame based on the difference between the travel percentage of adjacent frames within the segment and the difference between the timestamps, and taking the difference between the maximum and minimum instantaneous speeds of each frame as the speed fluctuation within the segment. The method for calculating the intra-segment jitter feature vector includes: performing edge detection on each frame of the latch image within the segment, calculating the edge sharpness value and gradient direction consistency value of the edge region of each frame respectively; forming sharpness time-series data based on the edge sharpness value of each frame, and forming direction consistency time-series data based on the gradient direction consistency value of each frame; performing time-series statistical analysis on the sharpness time-series data and the direction consistency time-series data respectively, and combining the obtained statistical features to form the intra-segment jitter feature vector.
6. The method for detecting defects in door lock mechanisms based on image recognition according to claim 5, characterized in that, The method for calculating the edge sharpness value includes: performing edge detection on the latch image and extracting the latch edge region; calculating the gradient magnitude of each pixel in the edge region and taking the average of the gradient magnitudes as the edge sharpness value of the frame. The method for calculating the gradient direction consistency value includes: calculating the gradient vector of each pixel in the edge region, normalizing each gradient vector to obtain a unit direction vector; calculating the vector sum of all unit direction vectors, and taking the ratio of the magnitude of the vector sum to the number of edge pixels as the gradient direction consistency value of the frame. The time-series statistical analysis of sharpness time-series data and direction consistency time-series data includes: calculating the sharpness mean, sharpness standard deviation, and sharpness range for the sharpness time-series data; calculating the consistency mean and consistency standard deviation for the direction consistency time-series data; and combining the sharpness mean, sharpness standard deviation, sharpness range, consistency mean, and consistency standard deviation to form an intra-segment jitter feature vector.
7. The method for detecting defects in door lock mechanisms based on image recognition according to claim 6, characterized in that, The standard feature set is pre-established based on the same travel segment points and includes the standard segment passage time, speed fluctuation within the standard segment, and tremor feature vector within the standard segment for each segment. The deviation analysis based on each segment feature and the pre-stored standard feature set includes: For each segment, the segment passage time deviation is calculated based on the segment passage time and the standard segment passage time; the speed fluctuation deviation is calculated based on the intra-segment speed fluctuation and the standard segment speed fluctuation; and the flutter deviation is calculated based on the intra-segment flutter feature vector and the standard segment flutter feature vector. The deviation characteristics of a segment are formed by combining time deviation, velocity fluctuation deviation, and flutter deviation.
8. The method for detecting defects in door lock mechanisms based on image recognition according to claim 7, characterized in that, The process of fusing the deviation characteristics of each segment under each driving condition to obtain the fused characteristics of each segment includes: For each segment, the deviation features of that segment are extracted from the deviation features under each driving condition to form a multi-condition deviation set for that segment; Vector concatenation of deviation features from multiple operating conditions is performed to form the fusion feature of that segment.
9. The method for detecting defects in door lock mechanisms based on image recognition according to claim 8, characterized in that, The training method for the defect classifier includes: Collect bolt image sequences of multiple defective door lock samples under various driving conditions; The image sequences of each door lock sample are processed by extracting and segmenting the latch image, calculating the features of each segment, performing deviation analysis based on the standard feature set to obtain the deviation features, and fusing the deviation features under each driving condition to obtain the fused features of each segment. Based on the actual defects of the door lock samples, the corresponding defect category labels are labeled for each segment of the fused features to form a training sample set. A defect classifier is obtained by training a machine learning classification model based on a training sample set.
10. A door lock mechanism defect detection system based on image recognition, used to implement the door lock mechanism defect detection method based on image recognition as described in any one of claims 1-9, characterized in that, include: The image acquisition module is used to preset multiple driving conditions, and for each driving condition, it acquires an image sequence of the locking tongue within the driving time to obtain the original image sequence; The image segmentation module is used to extract the latch portion image from the original image sequence to obtain the latch image sequence, and to segment the latch image sequence based on preset stroke segmentation points to obtain the segmented latch image sequence. The feature analysis module is used to calculate the corresponding segment features based on each segment of the latch image sequence, and to perform deviation analysis based on the features of each segment and the pre-stored standard feature set to obtain the deviation features of each segment. The defect identification module is used to fuse the deviation features of each segment under each driving condition to obtain the fused features of each segment. The fused features of each segment are then input into a pre-trained defect classifier for defect identification, and the defect identification results corresponding to each segment are output.