An image-guided circuit breaker assembly method and system

Abstract

This invention belongs to the field of image processing technology, specifically relating to an image-guided circuit breaker assembly method and system. The method includes: processing video of the assembly process using a neural network segmentation technique to accurately separate the energy storage spring and the trip hook; quantifying instantaneous strain by calculating the component of the optical flow field along the principal direction of the spring during its dynamic compression process, thereby obtaining a spring strain index to evaluate the smoothness of the process; calculating the hook engagement confidence score to evaluate the final accuracy by detecting its key points and analyzing its topological geometric features such as engagement distance and angle after assembly; multiplying and fusing the spring strain index and the engagement confidence score to obtain a joint assembly quality score, and determining whether the assembly is qualified, thus achieving a comprehensive and automated evaluation of assembly quality.

Description

Technical Field

[0001] This invention relates to the field of image processing technology. More specifically, this invention relates to an image-guided circuit breaker assembly method and system. Background Technology

[0002] Circuit breakers are core protective components in power systems, and the assembly quality of their operating mechanisms is crucial. The core aspects of the assembly process include the dynamic compression of the energy storage spring and the final static engagement of the tripping mechanism. The former determines whether the mechanism can smoothly store sufficient energy, while the latter ensures stable locking in standby mode. Therefore, accurate quality assessment of these two aspects is key to ensuring the long-term reliable operation of circuit breakers.

[0003] However, existing quality inspection methods suffer from significant technical bottlenecks. Traditional final inspection methods, such as measuring static dimensions or performing functional tests, focus only on the final result and cannot monitor process defects that may occur during assembly, such as spring jamming or impacts. These defects can introduce internal stress and create potential reliability issues. While some advanced production lines have attempted to introduce contact sensors, such as force and displacement sensors, for process monitoring, these sensors are complex to install, expensive, and their measurement signals are susceptible to interference from factors such as vibration and temperature drift, failing to comprehensively reflect the overall deformation state of the components.

[0004] Therefore, the industry has begun to explore the use of non-contact machine vision technology for quality inspection. For dynamic process analysis, optical flow is typically used to track pixel motion. However, standard dense optical flow algorithms calculate the motion vectors of all pixels within the field of view. In spring compression scenarios, it cannot effectively distinguish between effective motion along the main compression direction and ineffective vibrations in the vertical direction, making it difficult to accurately assess the smoothness of the energy storage process. For static structure inspection, key point detection algorithms can be used to locate the functional points of components, but the output image coordinates are extremely sensitive to the component's installation pose. Even slight translations or rotations can cause significant changes in the coordinate values, making it difficult to reliably determine the intrinsic geometric accuracy of the hook structure based solely on coordinates. Summary of the Invention

[0005] To address the aforementioned technical problem of poor quality inspection during circuit breaker assembly, this invention provides solutions in the following aspects.

[0006] In a first aspect, the present invention provides an image-guided circuit breaker assembly method, comprising:

[0007] A circuit breaker assembly video is acquired, and an instance segmentation neural network is used to process the video frame by frame to obtain a target video containing spring masks and trip hook masks. The spring optical flow field sequence of the target video is obtained. Based on the degree of movement of each pixel in the spring optical flow field sequence relative to the main compression direction, the instantaneous strain degree of the spring optical flow field is calculated. Based on the difference between the instantaneous strain degree of the spring optical flow field sequence and the standard spring strain index, the spring strain index of the target video is obtained. Key points of the trip hook in the target video are obtained. The geometric features of the triangular topology formed by the key points are extracted. Based on the difference between the geometric features and the standard geometric features, the topological difference degree of the trip hook in the target video is obtained. Based on the difference between the topological difference degree of the trip hook in the target video and the maximum topological difference degree, the hook engagement confidence score of the trip hook in the target video is obtained. The spring strain index is multiplied by the hook engagement confidence score to obtain the joint assembly quality of the target video. The circuit breaker assembly is then judged to be qualified based on the joint assembly quality.

[0008] This invention analyzes video of the entire assembly process to monitor and detect process defects such as spring jamming in real time, overcoming the shortcomings of traditional methods that cannot monitor the process. This invention quantitatively evaluates the dynamic smoothness and static accuracy of the two core components, the spring and the trip hook, and finally merges them into a joint assembly quality score. This avoids the problems of not being able to fully reflect component deformation, being susceptible to vibration interference, having difficulty distinguishing effective movement, and being sensitive to changes in component posture. Thus, it achieves non-contact, high-precision, and comprehensive automated inspection of circuit breaker assembly quality.

[0009] Preferably, the instantaneous strain degree satisfies the expression:

[0010] ;

[0011] In the formula, This represents the instantaneous strain degree of the t-th spring optical flow field; Let represent the set of pixels in the t-th spring optical flow field; This represents the optical flow vector of the i-th pixel in the t-th spring optical flow field; The dot product symbol; Represents a unit vector; Represents the absolute value symbol.

[0012] This invention projects the optical flow vectors of each pixel onto the main compression direction of the spring and sums them, which can filter out invalid motion components perpendicular to the main direction caused by factors such as vibration. It can more accurately quantify the effective motion of the spring in the compression direction, thereby more accurately reflecting the dynamic smoothness of the energy storage process and improving the anti-interference ability and accuracy of dynamic evaluation.

[0013] Preferably, the spring strain exponent of the target video satisfies the expression:

[0014] ;

[0015] ;

[0016] In the formula, The spring strain index represents the target video. Indicates the sensitivity coefficient; Indicates the total instantaneous strain of the target video; Indicates the standard spring strain index; Indicates the spring's starting time; Indicates the spring's cutoff time; This represents the instantaneous strain degree of the t-th spring optical flow field; Represents the absolute value function; This represents the natural exponential function.

[0017] This invention compares the total instantaneous strain with a standard value using normalization and maps it using an exponential function. This transforms the smoothness of the entire dynamic assembly process into a single, sensitive quantitative indicator, enabling a more intuitive and objective assessment of the overall quality of the dynamic process. It also effectively amplifies deviations from the standard state, resulting in a higher degree of differentiation between qualified and unqualified products, facilitating automated judgment.

[0018] Preferably, the key points include: the hook tip, the slot bearing point, and the pivot center.

[0019] Preferably, the geometric features include: hooking distance and hooking angle, wherein the hooking distance is the Euclidean distance between the hook tip and the card slot bearing point, and the hooking angle is the included angle between the two line segments connecting the hook tip and the card slot bearing point with the center of the rotating shaft as the vertex.

[0020] This invention defines geometric features consisting of key points, hooking distance, and hooking angle. By calculating these relative, intrinsic geometric features, the influence of huge changes in coordinate values ​​caused by tiny translations or rotations of components during assembly is eliminated, thereby enabling a more stable and accurate evaluation of the geometric accuracy of the hooking structure.

[0021] Preferably, the topological difference of the hook in the target video satisfies the expression:

[0022] ;

[0023] In the formula, Indicates the topological difference of the hook in the target video; , This indicates the hooking distance and hooking angle of the release hook in the target video; , Indicates the standard hook-up distance and standard hook-up angle; , Weighting coefficients representing the hooking distance and hooking angle; This represents the absolute value function.

[0024] This invention performs a weighted summation of the relative deviations of the hooking distance and the hooking angle to form a comprehensive geometric difference measure. This invention can assign different weights to different parameters based on their importance to the hooking function, thereby calculating the geometric difference between the release hook and the standard part more comprehensively and in line with actual functional requirements.

[0025] Preferably, the hook engagement confidence score of the hook in the target video satisfies the expression:

[0026] ;

[0027] In the formula, The hook-on confidence score represents the hook engagement of the target video. Indicates the topological difference of the hook in the target video; Indicates the maximum topological dissimilarity; This represents the maximum value function.

[0028] Preferably, the standard spring strain index is obtained by:

[0029] Acquire assembly videos of several circuit breakers that have been confirmed as qualified; acquire the spring start time and spring stop time of the assembly videos of all qualified circuit breakers; the sum of the instantaneous strain levels of all video frames between the spring start time and spring stop time of the assembly video of any qualified circuit breaker is recorded as the total instantaneous strain level of the qualified circuit breaker; the average of the total instantaneous strain levels of all qualified circuit breakers is recorded as the standard spring strain index.

[0030] Preferably, the acquisition of the standard hooking distance and standard hooking angle includes:

[0031] Extract the hook engagement distance and angle of the release hook of the product corresponding to the target video from the CAD design drawings, and record them as the standard hook engagement distance and standard hook engagement angle.

[0032] Secondly, the present invention provides an image-guided circuit breaker assembly system, including a processor and a memory, wherein the memory stores computer program instructions, and when the computer program instructions are executed by the processor, the above-mentioned image-guided circuit breaker assembly method is implemented.

[0033] By adopting the above technical solution, a computer program is generated from the above image-guided circuit breaker assembly method and stored in a memory so that it can be loaded and executed by a processor. This allows for the creation of a terminal device based on the memory and processor, making it convenient to use.

[0034] The beneficial effects of this invention are as follows:

[0035] (1) This invention evaluates the smoothness of the dynamic process by analyzing assembly videos and combining it with the accuracy of the final static structure;

[0036] (2) The present invention utilizes the projection optical flow method to accurately evaluate the dynamic smoothness of the spring energy storage process and reduce the interference of ineffective vibrations;

[0037] (3) By analyzing the inherent topological geometric features of the key points of the hook, the present invention achieves a stable judgment on the hook engagement accuracy that is not affected by the pose, forms a comprehensive joint quality score, and improves the efficiency and accuracy of automated inspection of assembly quality. Attached Figure Description

[0038] Figure 1 This is a flowchart illustrating an image-guided circuit breaker assembly method according to the present invention.

[0039] Figure 2 This is a schematic diagram illustrating the instantaneous strain of a defective part;

[0040] Figure 3 This is a schematic diagram showing a comparison of the geometric features of the release hook. Detailed Implementation

[0041] This invention discloses an image-guided circuit breaker assembly method, referring to... Figure 1 This includes steps S1-S4:

[0042] S1: Obtain the circuit breaker assembly video. Use an instance segmentation neural network to process the circuit breaker assembly video frame by frame to obtain the target video containing the spring mask and the trip hook mask.

[0043] It should be noted that in the high-speed, automated circuit breaker assembly area, multiple metal components interact and move against a complex background. The original video stream is a set of undifferentiated pixels; accurately locating and separating the energy storage spring and trip hook from the original video stream is a prerequisite for all analysis. Considering that pixel masks can depict the precise outline of the target component in each frame of the image with pixel-level accuracy, thereby completely separating the target from the background and other interference, this invention uses pixel masks as the key data structure carrying component information.

[0044] Specifically, the circuit breaker assembly video is acquired, and an instance segmentation neural network is used to process the circuit breaker assembly video frame by frame to obtain the target video containing the spring mask and the trip hook mask, including:

[0045] An industrial camera is deployed in the circuit breaker assembly area, ensuring its field of view clearly and completely covers the interaction area between the energy storage spring and the tripping mechanism. When the PLC (Programmable Logic Controller) at the workstation issues an assembly start signal, the industrial camera is simultaneously triggered to capture a video stream of the entire dynamic process, which is recorded as the circuit breaker assembly video. For example, the frame rate of the industrial camera is 200fps.

[0046] A pre-trained instance segmentation neural network is used to process the circuit breaker assembly video frame by frame to obtain the target video containing spring masks and trip hook masks. For example, the neural network is a Mask R-CNN model, which requires transfer learning training to accurately identify specific components in the scene. The training dataset contains multiple images with manually labeled pixel-level masks of the energy storage spring and trip hook under different lighting and angles; to ensure training effectiveness, the number of images is one thousand. The training objective is to minimize the total loss of the model's classification, bounding box regression, and mask prediction. The trained model is deployed to the edge computing device on the production line, and for each frame of the circuit breaker assembly video, the model can directly output the pixel mask of the target component.

[0047] At this point, the target video containing the spring mask and the release hook mask has been obtained.

[0048] S2: Obtain the spring optical flow field sequence of the target video; calculate the instantaneous strain of the spring optical flow field based on the degree of movement of each pixel relative to the main compression direction in the spring optical flow field sequence; obtain the spring strain index of the target video based on the difference between the instantaneous strain of the spring optical flow field sequence and the standard spring strain index.

[0049] It should be noted that during assembly, the energy storage spring is compressed, which is a dynamic physical process of energy storage. Ideally, the compression process should be smooth and without jamming. However, poor assembly, such as burrs on parts and insufficient lubrication, can lead to problems like jamming and impacts. These process defects are difficult to detect through final static inspection. Considering that a smooth energy storage process is mapped to the stability and continuity of pixel motion patterns in an image, the quality of the physical process can be indirectly evaluated by calculating the direction and distance of pixel motion. Therefore, this invention analyzes the pixel movement in the spring mask region of the target video to obtain the spring strain index of the target video.

[0050] Specifically, acquiring the spring optical flow field sequence of the target video includes:

[0051] The spring masks of each frame of the target video are arranged in chronological order to form a spring mask sequence. A dense optical flow estimation algorithm is then used to calculate the dense optical flow field between all consecutive frames of the spring mask sequence, forming a spring optical flow field sequence in chronological order. This dense optical flow field contains the optical flow vector of each pixel in each spring mask region. It should be noted that the optical flow vector reflects the direction and velocity of pixel movement. The dense optical flow estimation algorithm, such as the Farneback algorithm, is existing technology and will not be elaborated upon here.

[0052] It should be noted that the compression motion of a spring has a definite principal direction, and only the component of pixel motion along this principal compression direction contributes to the physical process of energy storage. Therefore, this invention projects the motion of all pixels onto the principal direction and aggregates them to characterize the instantaneous compression rate of the entire spring in terms of vision.

[0053] Preferably, the instantaneous strain of the spring optical flow field is calculated based on the degree of movement of each pixel relative to the main compression direction in the spring optical flow field sequence, including:

[0054] By manually selecting the center points at both ends of the spring, the principal direction of spring compression and the unit vector are calibrated; the length of the unit vector is 1.

[0055] The instantaneous strain of the spring optical flow field satisfies the following expression:

[0056] ;

[0057] In the formula, This represents the instantaneous strain degree of the t-th spring optical flow field; Let represent the set of pixels in the t-th spring optical flow field; This represents the optical flow vector of the i-th pixel in the t-th spring optical flow field; The dot product symbol; Represents a unit vector; Represents the absolute value symbol.

[0058] In the formula, This represents the component of the optical flow vector of the i-th pixel in the t-th spring optical flow field projected onto the main compression direction. This means taking the maximum value of the component and 0 to avoid the influence of values ​​in the opposite direction on the accuracy of the instantaneous strain calculation; This represents the modulus of the component. The larger the value, the greater the instantaneous strain of the i-th pixel in the t-th spring optical flow field, and the greater its contribution to compression energy storage. This represents the sum of instantaneous strain levels of all pixels in the t-th spring optical flow field, reflecting the cumulative contribution of all pixels in the t-th spring optical flow field to the compression energy storage. When the overall compression speed of the spring is fast, The larger the value, the lower the overall speed when jamming occurs. The value decreases, therefore The changes over time reflect the dynamic smoothness of the compression process.

[0059] Thus, the instantaneous strain degree of each spring's optical flow field was obtained.

[0060] It should be noted that in order to condense the entire dynamic process into a single stable characteristic, it is necessary to sum the instantaneous strain at all time points during the entire circuit breaker assembly process to obtain the energy storage status of the spring during the entire circuit breaker assembly process. On the other hand, in order to provide a reference for the values, this invention also obtains the standard spring strain index based on the assembly status of qualified products.

[0061] Preferably, the spring strain index of the target video is obtained based on the difference between the instantaneous strain degree of the spring optical flow field sequence and the standard spring strain index, including:

[0062] Set the instantaneous strain threshold, The time when the strain first exceeds the instantaneous strain threshold is recorded as the spring's initiation time. The moment immediately following the last instance of strain exceeding the instantaneous strain threshold is recorded as the spring cutoff time. It should be noted that since the target video includes a preparatory state, using all instantaneous strain levels as usable data would affect the evaluation of the spring state. Therefore, this invention sets a threshold to extract the effective strain period. For example, the instantaneous strain levels of all qualified circuit breaker assembly videos during non-operating periods are statistically analyzed, and their average value is calculated. and standard deviation Set the instantaneous strain threshold to It should be noted that, as Figure 2 This diagram illustrates the instantaneous strain level of a defective part. It shows the changes in the instantaneous strain level reflected in the assembly video of the defective part, including the example instantaneous strain threshold of 670.11, and also indicates the effective energy storage period from the spring start time of 15 seconds to the spring stop time of 42 seconds.

[0063] Acquire assembly videos of several circuit breakers confirmed as qualified; acquire the spring start time and spring stop time of all qualified circuit breaker assembly videos; the sum of the instantaneous strain levels of all video frames between the spring start time and spring stop time of any qualified circuit breaker assembly video is recorded as the total instantaneous strain level of the qualified circuit breaker; the average of the total instantaneous strain levels of all qualified circuit breakers is recorded as the standard spring strain index. For example, there are 10 qualified circuit breakers.

[0064] The spring strain exponent of the target video satisfies the expression:

[0065] ;

[0066] ;

[0067] In the formula, The spring strain index represents the target video. Indicates the sensitivity coefficient; Indicates the total instantaneous strain of the target video; Indicates the standard spring strain index; Indicates the spring's starting time; Indicates the spring's cutoff time; This represents the instantaneous strain degree of the t-th spring optical flow field; Represents the absolute value function; This represents the natural exponential function.

[0068] It should be noted that, It is based on the total instantaneous strain of non-conforming and conforming products in historical data. The distribution difference is used to determine the final score of the two classes of samples, aiming to maximize the final score of the two classes of samples. The distinguishing factor, for example, is calculated by determining the difference in the deviation of the total instantaneous strain of qualified and unqualified circuit breakers from the standard value, through calibration. This causes the spring strain index of qualified products to be significantly higher than that of unqualified products, for example, qualified products. substandard products , .

[0069] In the formula, This represents the difference between the total instantaneous strain of the target video and the standard spring strain index. This means that the difference is normalized using the standard spring strain index, eliminating the influence of dimensions and providing a reference for the spring strain index of the target video; This indicates that the sensitivity adjustment is applied to the normalized differences. The larger the value, the greater the deviation from the standard. Through the exponential decay function, the spring strain exponent of the target video is made smaller.

[0070] At this point, the spring strain index of the target video has been obtained.

[0071] S3: Obtain the key points of the hook release in the target video; extract the geometric features of the triangle topology formed by the key points; obtain the topological difference degree of the hook release in the target video based on the difference between the geometric features and the standard geometric features; obtain the hook engagement confidence score of the hook release in the target video based on the difference between the topological difference degree of the hook release in the target video and the maximum topological difference degree.

[0072] It should be noted that after the spring's energy storage ends, the release hook engages with the relevant components, forming a stable static locking structure. The reliability of this locking structure does not depend on the overall outline of the release hook or its absolute position in an image, but rather on the precision of the geometric relationships between functional points. Considering key points such as the hook tip, slot, and pivot, the resulting geometric topology, including distances and angles, directly determines the standardization of its functional geometric construction. Therefore, this invention combines the relative spatial relationships of all key points to establish a topology for the engagement key points.

[0073] Specifically, the key points for obtaining the hook release mechanism in the target video include:

[0074] In the final frame of the target video, a pre-trained keypoint detection network is used to locate at least three keypoints in the hook mask area, such as the hook tip, the slot support point, and the pivot center. It should be noted that the keypoint detection network preferably uses HRNet (High-Resolution Network) due to its advantage in maintaining high-resolution feature maps, making it suitable for accurate localization of small targets. The training dataset contains multiple hook images with manually labeled coordinates of the hook tip, slot support point, and pivot center. For example, one thousand hook images are used to ensure training accuracy.

[0075] It should be noted that, in order to eliminate the effects of overall translation and rotation of components in the image caused by installation errors, it is necessary to extract intrinsic geometric features that are unrelated to these rigid body transformations. Considering that the side length and angle of the triangle formed by the three key points are ideal topological invariants, this invention uses the side length and angle of the triangle formed by the three key points as the key features for establishing the topology of the hooked key points.

[0076] Preferably, the geometric features of the triangular topology formed by the key points are extracted: the geometric features include the hooking distance and the hooking angle, wherein the hooking distance is the Euclidean distance between the hook tip and the card slot bearing point, and the hooking angle is the angle between the two line segments connecting the hook tip and the card slot bearing point with the center of the rotation axis as the vertex. It should be noted that these two features together define the geometric accuracy of the hooking relationship.

[0077] It should be noted that by comparing the geometric features of the release hook in the target video with the standard geometric features, the differences between the release hook in the target video and the standard part can be obtained. For example... Figure 3 This is a geometric feature comparison diagram of the trip hook, showing a comparison of the geometric features of the trip hook in the target video and the trip hook in the design drawing.

[0078] Preferably, the topological difference degree of the hook-off in the target video is obtained based on the difference between the geometric features and the standard geometric features, including:

[0079] The hook engagement distance and angle of the corresponding product's release hook are extracted from the CAD design drawings and recorded as the standard hook engagement distance and standard hook engagement angle. It should be noted that the design drawings represent the theoretically most standard product structural geometry.

[0080] The topological difference of the hook release in the target video satisfies the expression:

[0081] ;

[0082] In the formula, Indicates the topological difference of the hook in the target video; , This indicates the hooking distance and hooking angle of the release hook in the target video; , Indicates the standard hook-in distance and standard hook-in angle; , Weighting coefficients representing the hooking distance and hooking angle; This represents the absolute value function. It should be noted that... and The importance of the hook-up function is allocated by the design engineers based on two characteristics: distance and angle. Exemplary , .

[0083] In the formula, This indicates the difference between the hook engagement distance of the target video and the standard hook engagement distance. This indicates the difference between the hook engagement angle of the target video and the standard hook engagement angle; , This means that by using standard hooking distance and standard hooking angle, the differences in hooking distance, hooking angle and standard geometric features of the hook in the target video are standardized to remove the influence of dimensions, and to provide a reference for the values ​​of hooking distance and hooking angle of the hook in the target video. This indicates that the deviations in hooking distance and hooking angle are weighted and distributed so that the relative errors of length and angle are combined into a total difference index.

[0084] Preferably, the hook engagement confidence score of the hook in the target video is obtained based on the difference between the topological difference degree of the hook's release and the maximum topological difference degree, including:

[0085] The maximum topology difference is obtained based on the upper limit of the engineering tolerance set according to functional requirements and experimental data.

[0086] The hook engagement confidence score of the target video's hook release mechanism satisfies the expression:

[0087] ;

[0088] In the formula, The hook-on confidence score represents the hook engagement of the target video. Indicates the topological difference of the hook in the target video; Indicates the maximum topological dissimilarity; This represents the maximum value function.

[0089] In the formula, This indicates that the actual topological dissimilarity is normalized relative to the maximum allowed dissimilarity. This means converting the normalized variance into a confidence score; This ensures that when the topological dissimilarity exceeds the allowable upper limit, the confidence score is 0, avoiding negative confidence scores. The formula establishes a linear mapping relationship: when the topological dissimilarity is 0, the hook confidence score is 1. As the topological dissimilarity increases, the hook confidence score decreases linearly. When the topological dissimilarity exceeds the maximum topological dissimilarity, the hook confidence score is 0, indicating that it is completely unacceptable.

[0090] At this point, the hook engagement confidence score of the target video's hook has been obtained.

[0091] S4: Based on the spring strain index and hook confidence score of the target video, the joint quality score is obtained by product fusion and a final judgment is made.

[0092] It should be noted that high-quality assembly requires both a smooth process and accurate results. The spring strain index of the target video reflects the smoothness of the assembly process from a process perspective, while the hook engagement confidence score of the target video reflects the accuracy of the assembly from a result perspective. Therefore, this invention combines the spring strain index and the hook engagement confidence score of the target video to calculate the combined assembly quality of the target video.

[0093] Specifically, the spring strain index of the target video is multiplied by the hook engagement confidence score of the target video, and this multiplication is recorded as the joint assembly quality of the target video. If the joint assembly quality of the target video is greater than or equal to the qualified threshold, the circuit breaker corresponding to the target video is recorded as qualified; if the joint assembly quality of the target video is less than the qualified threshold, the circuit breaker corresponding to the target video is recorded as unqualified. It should be noted that the fusion using multiplication is functionally equivalent to an AND gate. Only when both the process spring strain index and the hook engagement confidence score are close to 1 will the final joint assembly quality approach 1. Low values ​​in any dimension will lead to a decrease in the joint assembly quality. For example, the qualified threshold is 0.9.

[0094] The assembly and testing of the circuit breaker have been completed.

[0095] This invention also discloses an image-guided circuit breaker assembly system, including a processor and a memory. The memory stores computer program instructions, which, when executed by the processor, implement an image-guided circuit breaker assembly method according to the present invention.

[0096] The system also includes other components well known to those skilled in the art, such as communication buses and communication interfaces, the settings and functions of which are known in the art and will not be described in detail here.

[0097] While this specification has shown and described numerous embodiments of the invention, it will be apparent to those skilled in the art that such embodiments are provided by way of example only. Many modifications, alterations, and alternatives will occur to those skilled in the art without departing from the spirit and essence of the invention. It should be understood that various alternatives to the embodiments of the invention described herein may be employed in the practice of this invention.

Claims

1. A circuit breaker assembly method based on image guidance, characterized in that, include: The circuit breaker assembly video is acquired, and the instance segmentation neural network is used to process the circuit breaker assembly video frame by frame to obtain the target video containing the spring mask and the trip hook mask. Obtain the spring optical flow field sequence of the target video; calculate the instantaneous strain of the spring optical flow field based on the degree of movement of each pixel relative to the main compression direction in the spring optical flow field sequence. , ; Let be the set of pixels in the t-th spring optical flow field; Let be the optical flow vector of the i-th pixel in the t-th spring optical flow field; The dot product symbol; It is a unit vector; The absolute value symbol is used; the spring strain index of the target video is obtained based on the difference between the instantaneous strain degree of the spring optical flow field sequence and the standard spring strain index. , ; ； This is the sensitivity coefficient; The total instantaneous strain of the target video; The standard spring strain index; This is the spring's initial time; This refers to the spring's cutoff time. It is a natural exponential function; Obtain the key points of the hook release in the target video; extract the geometric features of the triangle topology formed by the key points; and obtain the topological difference degree of the hook release in the target video based on the difference between the geometric features and the standard geometric features. , ; , The hooking distance and hooking angle of the release hook in the target video; , Standard hooking distance and standard hooking angle; , These are the weighting coefficients for the hooking distance and hooking angle; Based on the difference between the topological difference degree and the maximum topological difference degree of the trip hook in the target video, the hook engagement confidence score of the trip hook in the target video is obtained; the spring strain exponent is multiplied by the hook engagement confidence score to obtain the joint assembly quality of the target video, and the circuit breaker assembly is judged as qualified based on the joint assembly quality.

2. The image-guided circuit breaker assembly method according to claim 1, characterized in that, The key points include: the hook tip, the bearing point of the slot, and the center of the rotating shaft.

3. The image-guided circuit breaker assembly method according to claim 1, characterized in that, The geometric features include: hooking distance and hooking angle, wherein the hooking distance is the Euclidean distance between the hook tip and the card slot bearing point, and the hooking angle is the included angle between the two line segments connecting the hook tip and the card slot bearing point with the center of the rotating shaft as the vertex.

4. The image-guided circuit breaker assembly method according to claim 1, characterized in that, The hook engagement confidence score of the target video's hook release satisfies the expression: ； In the formula, The hook-on confidence score represents the hook engagement of the target video. Indicates the topological difference of the hook in the target video; Indicates the maximum topological dissimilarity; This represents the maximum value function.

5. The image-guided circuit breaker assembly method according to claim 1, characterized in that, The acquisition of the standard spring strain index includes: Acquire assembly videos of several circuit breakers that have been confirmed as qualified; acquire the spring start time and spring stop time of the assembly videos of all qualified circuit breakers; the sum of the instantaneous strain levels of all video frames between the spring start time and spring stop time of the assembly video of any qualified circuit breaker is recorded as the total instantaneous strain level of the qualified circuit breaker; the average of the total instantaneous strain levels of all qualified circuit breakers is recorded as the standard spring strain index.

6. The image-guided circuit breaker assembly method according to claim 3, characterized in that, The acquisition of the standard hooking distance and standard hooking angle includes: Extract the hook engagement distance and angle of the release hook of the product corresponding to the target video from the CAD design drawings, and record them as the standard hook engagement distance and standard hook engagement angle.

7. A circuit breaker assembly system based on image guidance, characterized in that, include: A processor and a memory, the memory storing computer program instructions that, when executed by the processor, implement an image-guided circuit breaker assembly method according to any one of claims 1-6.

Images