Bolt cluster quality control method, equipment and medium based on machine vision

Abstract

The present application discloses a bolt cluster quality control method, equipment and medium based on machine vision, which relates to the field of production control. The method includes: generating a corresponding assembly model for a bolt cluster according to its corresponding assembly attributes; obtaining an assembly video based on the production process; performing machine vision analysis on the assembly video to determine its corresponding assembly attributes, and selecting a matching designated model from the generated assembly model; performing machine vision analysis on the assembly video to determine its corresponding bolt cluster assembly process, and controlling the bolt cluster assembly process through the designated model. The assembly process is digitally supervised, and full-process automated monitoring is achieved through machine vision, replacing traditional manual visual inspections, solving the problem of uncontrollable execution of work instructions, improving the accuracy of compliance inspections, and being able to capture illegal operations in real time, shortening alarm response time.

Description

Technical Field

[0001] The present application relates to the field of production control, and specifically to a bolt cluster quality control method, equipment and medium based on machine vision. Background Art

[0002] At present, the tightening quality of bolt cluster assembly, such as torque / tightening force and stress concentration, is usually guaranteed by the method of "operation instructions + torque wrench inspection". Among them, the torque wrench inspection is done by the operator himself, or the inspector can also use it when conducting relevant spot checks. The operation instructions are only used as a reference document for operators during the production process. Whether the operators comply with and strictly implement them depends on the operators' work attitude, professional ability, etc., and there is a lack of effective feedback and supervision.

[0003] Based on this, the guidance effect of the work instructions lacks supervision and guarantee, which easily leads to production problems such as irregular tightening sequence, lack of pre-tightening, uneven tightening force, etc. Moreover, after the assembly is completed, it is difficult for inspectors to trace and inspect the assembly process, making it difficult to supervise the tightening process. Summary of the invention

[0004] In order to solve the above problems, this application proposes a bolt cluster quality control method based on machine vision, including: Based on the production process, determine the bolt clusters included, and generate corresponding assembly models for the bolt clusters according to their corresponding assembly attributes; Obtaining an assembly video based on the production process through a machine vision device; Performing machine vision analysis on the assembly video to determine the corresponding assembly attributes, and selecting a matching designated model from the generated assembly model; Machine vision analysis is performed based on the assembly video to determine the corresponding bolt cluster assembly process, and the bolt cluster assembly process is controlled through the specified model.

[0005] In one example, for the bolt cluster, generating a corresponding assembly model according to its corresponding assembly attributes specifically includes: For the bolt cluster, determine the type of wrench to be used according to the production process; Determining, according to the wrench type, a tightening step for each bolt in the bolt cluster; For the bolt cluster, determining the corresponding bolt arrangement and the stiffness of each bolt therein; Determining a tightening sequence of the bolt cluster according to the bolt arrangement and the stiffness of each bolt; According to the tightening steps and the tightening sequence, a corresponding assembly model is generated.

[0006] In one example, determining the tightening sequence of the bolt cluster according to the bolt sorting method and the stiffness of each bolt includes: Determine a corresponding default tightening sequence according to the bolt sorting method, and determine the tightening priority between the bolts according to the default tightening sequence; Arrange the bolts in descending order according to their stiffness to obtain a tightening sequence of the bolt cluster; For bolts with the same stiffness, the sub-tightening sequence of some bolts with the same stiffness is adjusted according to their corresponding tightening priorities.

[0007] In one example, after determining the tightening sequence of the bolt cluster according to the bolt sorting method and the stiffness of each bolt, the method further includes: Determine a corresponding prohibited tightening sequence according to the bolt sorting mode, and determine whether the current tightening sequence of the bolt cluster hits the prohibited tightening sequence; If it is hit, then traverse to obtain all tightening sequences corresponding to the bolt cluster, and among all tightening sequences, determine the specified tightening sequence that does not hit the prohibited tightening sequence, and determine the adjustment score between each specified tightening sequence and the current tightening sequence; The specified tightening sequence with the lowest adjustment score is selected to replace the current tightening sequence for the bolt cluster.

[0008] In one example, the adjustment score is obtained by weighted summing of sub-adjustment scores corresponding to each bolt position in the bolt arrangement, the weight of each bolt position is determined according to its corresponding stiffness difference, and the sub-adjustment score is obtained according to the sequence difference of the bolt positions; Among them, the higher the stiffness difference, the higher the weight; the higher the rank difference, the higher the sub-adjustment score.

[0009] In one example, performing machine vision analysis on the assembly video to determine the corresponding assembly attributes includes: performing machine vision analysis based on the assembly video using a pre-trained neural network model; Determine the corresponding wrench type and bolt arrangement according to the first analysis result corresponding to the machine vision analysis; Machine vision analysis is performed based on the assembly video to determine the corresponding bolt cluster assembly process, specifically including: performing machine vision analysis based on the assembly video using a pre-trained neural network model; According to the second analysis result corresponding to the machine vision analysis, the corresponding tightening sequence and the tightening steps of each bolt are determined.

[0010] In one example, controlling the bolt cluster assembly process by using the specified model specifically includes: Determining that during the bolt cluster assembly process, there are abnormal behaviors that do not conform to the specified model; According to the abnormal behavior, determining the alarm category to which it belongs, and generating alarm information according to the alarm category and the abnormal behavior; According to the production process, the corresponding alarm method and alarm personnel are selected, and the alarm is issued through the alarm information.

[0011] In one example, the method further includes: For the production process, according to the preset control cycle, the alarm information generated within the control cycle is counted and visualized; According to the statistical results, the feature dimensions in which the probability of abnormal information occurring is higher than the preset probability are obtained, so as to facilitate optimization according to the feature dimensions.

[0012] On the other hand, the present application also proposes a bolt cluster quality control device based on machine vision, comprising: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the machine vision-based bolt cluster quality control method as described in any of the above examples.

[0013] On the other hand, the present application also proposes a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to be: a bolt cluster quality control method based on machine vision as described in any of the above examples.

[0014] The bolt cluster quality control method based on machine vision proposed in this application can bring the following beneficial effects: 1. The assembly process is digitally supervised, and full-process automated monitoring is achieved through machine vision, replacing traditional manual visual inspections, solving the problem of uncontrollable execution of work instructions, improving the accuracy of compliance inspections, and being able to capture illegal operations in real time, shortening alarm response time.

[0015] 2. Realize the traceability of the production process, associate and store video data with operation logs, support reverse tracing by bolt number, and locate abnormal events to specific workstations / operators / time nodes through management and control. It can also analyze historical data and identify potential process defects. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings: Figure 1 This is a flow chart of a bolt cluster quality control method based on machine vision in an embodiment of the present application; Figure 2 This is a schematic diagram of a bolt cluster quality control method based on machine vision in one scenario in an embodiment of the present application; Figure 3 This is a schematic diagram of the default tightening sequence in the first situation in the embodiment of the present application; Figure 4 This is a schematic diagram of the default tightening sequence in the second situation in the embodiment of the present application; Figure 5 This is a schematic diagram of the default tightening sequence in the third situation in the embodiment of the present application; Figure 6 This is a schematic diagram of a default tightening sequence in the fourth situation in the embodiment of the present application; Figure 7 This is a schematic diagram of the default tightening sequence in the fifth situation in the embodiment of the present application; Figure 8 This is a schematic diagram of a default tightening sequence in the sixth situation in the embodiment of the present application; Fig. 9 This is a schematic diagram of the default tightening sequence in the seventh situation in the embodiment of the present application; Fig.10 This is a schematic diagram of a bolt cluster quality control device based on machine vision in an embodiment of the present application. DETAILED DESCRIPTION

[0017] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0018] The technical solutions provided by various embodiments of the present application are described in detail below in conjunction with the accompanying drawings.

[0019] like Figure 1 and Figure 2 As shown, the embodiment of the present application provides a bolt cluster quality control method based on machine vision, comprising: S101: Based on the production process, a bolt cluster is determined, and for the bolt cluster, a corresponding assembly model is generated according to its corresponding assembly attributes.

[0020] Different equipment and parts have different production processes. In actual production and processing, the CNC files corresponding to the production process need to be loaded into the CNC production equipment in advance, or the production process is performed manually according to the production process to complete the production of the equipment.

[0021] In the embodiments of the present application, the main focus is on the production process corresponding to the bolt cluster, that is, the process of tightening the bolts. The bolt cluster refers to a plurality of bolts in a preset range area or a corresponding area on a single device, through which a single device or component is fixed.

[0022] For different production processes, the bolt clusters are also different. For example, the number, position, stiffness, etc. of the bolts may be different. Therefore, different assembly properties need to be set to make the bolt tightening process more stable.

[0023] Specifically, for bolt clusters, the type of wrench to be used is determined based on the production process. Generally speaking, wrenches can be divided into two types, ordinary wrenches and torque wrenches. Ordinary wrenches have a relatively simple structure and are usually made of structural steel made of carbon or alloy materials. Torque wrenches add torque measurement and control mechanisms to ordinary wrenches. For example, pointer-type torque wrenches indicate torque values ​​through mechanical pointers, while digital torque wrenches use electronic sensors and display screens to accurately display torque values.

[0024] Of course, the wrench here can be used manually, or used in an automatic generation device.

[0025] Determine the tightening steps for each bolt in the bolt cluster based on the wrench type. Generally speaking, when there are multiple bolts in a bolt cluster, in order to ensure uniform force and prevent deformation, it is usually not necessary to fully tighten a single bolt before tightening the next bolt. Instead, it is divided into multiple rounds. In each round, tighten all bolts to a certain degree before entering the next round, and tighten all the bolts further until they are all tightened.

[0026] Different bolt types work differently, so different tightening steps are set to adapt. For example, when using an ordinary wrench, tightening generally includes three steps. The first step is to use an ordinary wrench to pre-tighten and tighten to about 50% of the torque / tightening force. The second step is to use an ordinary wrench to tighten to near the threshold of the torque / tightening force. The third step is to use a torque wrench to tighten and check to make the torque / tightening force reach 100%. In other words, when using an ordinary wrench, first tighten all the bolts to about 50%, then tighten all the bolts to near the threshold, and then tighten and check all the bolts to 100%.

[0027] When using a torque wrench, tightening generally includes two steps. The first step is to use the torque wrench to pre-tighten and tighten to about 50% of the torque / tightening force. The second step is to use the torque wrench to tighten to 100%. For a torque wrench, since it has an additional torque measurement and control mechanism, the second step in an ordinary wrench can be omitted.

[0028] In addition, for the bolt cluster, determine its corresponding bolt arrangement and the stiffness of each bolt in it.

[0029] The bolt arrangement refers to the shape and quantity of the bolts. Figure 3~Figure 9 As shown, several common bolt arrangements are shown. In different production processes, the number and shape of bolts required are different, so the bolt arrangements are also different.

[0030] The stiffness of a bolt refers to its ability to resist deformation when subjected to force. It is determined based on the material, size, thread parameters, etc. of the bolt and can be determined by the factory parameters of the bolt.

[0031] In order to ensure uniform force and prevent deformation during bolt tightening, the tightening sequence of the bolt cluster can be set according to the bolt sorting method and the stiffness of each bolt. The tightening steps mentioned above refer to the degree to which a single bolt needs to be tightened in each round, and the tightening sequence refers to the order in which multiple bolts are tightened in each round.

[0032] At this time, the corresponding assembly model is generated according to the tightening steps and tightening sequence, and is added to the model library. Whenever there is a new assembly model, it can be added to the model library for storage.

[0033] Furthermore, when determining the tightening sequence, first determine the corresponding default tightening sequence according to the bolt sorting method, and determine the tightening priority between the bolts according to the default tightening sequence. The higher the priority in the default tightening sequence, the higher the tightening priority.

[0034] like Figure 3~Figure 9As shown, the default tightening sequence for several common situations is illustrated, which mainly follows the principle of "first the middle, then the two sides, diagonally, and clockwise", and tightens evenly in sequence.

[0035] Of course, the figure only shows some common examples. In actual production and processing, there may be other bolt sorting methods, and the default tightening sequence can be set according to expert experience. Alternatively, the default tightening sequence of the bolt sorting method shown in the figure can be adjusted based on different actual needs.

[0036] According to the stiffness of each bolt, they are sorted from high to low to get the tightening order of the bolt cluster. In the process of bolt tightening, the earlier the bolt is tightened, the greater the pressure it will bear. Based on this, when the stiffness is consistent, the default tightening order can be directly used as the tightening order of the bolt cluster. If there is an inconsistency in stiffness, the bolts with greater stiffness are tightened first.

[0037] At this time, there may be bolts with the same stiffness. For the bolts with the same stiffness, the sub-tightening sequence of some bolts with the same stiffness is adjusted according to their corresponding tightening priorities.

[0038] by Figure 4 For example, there are four bolts. Here, the bolt in the upper left corner is called bolt A, the bolt in the upper right corner is called bolt B, the bolt in the lower left corner is called bolt C, and the bolt in the lower right corner is called bolt D. Assuming that the stiffness is the same, the default tightening order is used as the final tightening order. Assuming that the stiffness of bolt B is the highest, and the stiffness of other bolts is lower than that of bolt B and is the same, the tightening order of bolt B is changed to the first place. Among the other bolts, they are sorted according to the tightening priority in the default tightening order (for example, set to bolt A, bolt D, bolt B, bolt C). Since bolt B has been determined, the final tightening order is: bolt B, bolt A, bolt D, bolt C. Compared with the default tightening order of bolt A, bolt D, bolt B, bolt C, bolt B is tightened at the beginning to reduce the probability of damage to the bolt.

[0039] Furthermore, after determining the tightening sequence of the bolt cluster, it may happen that the tightening sequence adjusted according to the stiffness does not comply with the requirements of the actual operation. Therefore, according to the bolt sorting method, the corresponding prohibited tightening sequence is determined, and it is determined whether the current tightening sequence of the bolt cluster hits the prohibited tightening sequence.

[0040] Among them, the prohibited tightening sequence is also pre-set, and it can only set some bolts, for example, only set the bolts in certain positions, and cannot be in the first sequence or the last sequence, etc.

[0041] Determine a corresponding prohibited tightening sequence according to the bolt sorting method, and determine whether the current tightening sequence of the bolt cluster hits the prohibited tightening sequence; If the current tightening sequence hits the forbidden tightening sequence, all tightening sequences corresponding to the bolt cluster are traversed. All tightening sequences here can be traversed by permutations and combinations, only considering the order of the bolts, and not considering other factors such as stiffness.

[0042] Among all the tightening sequences, a first round of screening is performed to determine the designated tightening sequences that do not hit the prohibited tightening sequences. Then a second round of screening is performed to determine the adjustment score between each designated tightening sequence and the current tightening sequence.

[0043] Among them, the sub-adjustment score corresponding to each bolt position in the bolt arrangement is obtained by weighted summation. The weight of each bolt position is determined according to its corresponding stiffness difference, and the sub-adjustment score is obtained according to the sequence difference of the bolt position. The bolt position refers to the position of the bolt hole, and does not refer to a specific bolt.

[0044] Generally speaking, the higher the stiffness difference, the greater the change in stiffness before and after the bolt position is adjusted, which means that the bolt position is relatively adjusted, so the higher the weight is set. When the stiffness difference is 0, the weight is the default value (for example, the current and subsequent default values ​​can be set to 1). Similarly, the higher the sequence difference, the greater the change in the tightening sequence before and after the bolt position is adjusted, so the higher the sub-adjustment score. When the sequence difference is 0, the sub-adjustment score is set to the default value. In this way, the weight is multiplied by the sub-adjustment score to obtain the score of a single bolt position, and the scores of all bolt positions are summed to obtain the total adjustment score.

[0045] The specified tightening sequence with the lowest adjustment score is selected to replace the current tightening sequence of the bolt cluster as the tightening sequence of the bolt cluster. In this way, when the current tightening sequence is not compliant, the specified tightening sequence with the smallest adjustment range is selected as a replacement. If there are multiple specified tightening sequences with the same lowest score, one of them can be randomly selected.

[0046] S102: Obtaining an assembly video based on the production process through a machine vision device.

[0047] The machine vision device can be an industrial camera, which can collect assembly videos of bolt clusters during the production process. The assembly videos are transmitted to the edge server through the workshop IoT industrial gateway and network for video analysis.

[0048] S103: Performing machine vision analysis on the assembly video to determine the corresponding assembly attributes, and selecting a matching designated model from the generated assembly model.

[0049] Machine vision analysis can use computer technology and image processing algorithms to simulate the human visual system to extract, analyze and understand information in images or videos. Image processing algorithms can include traditional image processing algorithms and processing through deep learning algorithms. For example, in traditional image processing algorithms, it can be implemented through sliding windows + feature extraction and classification, based on manual features and machine learning classifiers.

[0050] Taking the deep learning algorithm as an example, a pre-trained neural network model can be used to perform machine vision analysis based on the assembly video, so as to obtain the corresponding first analysis result based on the machine vision analysis. The first analysis result includes the wrench type and bolt arrangement as assembly attributes.

[0051] Among them, the neural network model can select a 3D convolutional neural network as the network architecture, which includes an input layer, a 3D convolutional layer, a pooling layer, a fully connected layer, and an output layer.

[0052] The input layer takes a video sequence as input, and samples the video into a series of continuous image frames at a fixed frame rate. Assuming the size of the video frame is H*W*C, where H is the height, W is the width, and C is the number of channels, and the number of sampled frames is T, the shape of the input tensor is (T, H, W, C).

[0053] The 3D convolution layer can extract features in both time and space dimensions. It uses multiple 3D convolution kernels of different sizes to perform convolution operations on the input video sequence. For example, the first 3D convolution layer uses The size is The convolution kernel is is the convolution kernel size in the time dimension, and is the convolution kernel size of the spatial dimension, and the step size of the first 3D convolution layer is , filled with After this convolution layer, the shape of the output feature map will change according to the parameters of the convolution operation. You can continue to stack multiple 3D convolution layers to extract more advanced features. Batch Normalization and ReLU activation functions can be used after each layer to speed up training and increase the nonlinearity of the model.

[0054] The pooling layer uses a 3D maximum pooling layer to reduce the size of the feature map, reduce the amount of calculation, and enhance the robustness of the features.

[0055] After passing through multiple 3D convolutional layers and pooling layers, the feature map of the last layer is flattened into a one-dimensional vector and then connected to multiple fully connected layers. The fully connected layer is used to map the extracted features to the category space of assembly attributes. For example, there are N categories of wrench types and bolt arrangements, so the output dimension of the last fully connected layer is N. Of course, when the model corresponds to the analysis of the bolt cluster assembly process, it can be mapped to the category space of assembly actions. The operator's assembly actions have N categories, so the output dimension of the last fully connected layer is N.

[0056] The output layer uses the Softmax activation function to convert the output of the fully connected layer into the probability distribution of the assembly action category of each assembly attribute, and outputs the predicted probability of each action category, thereby analyzing the bolt cluster assembly process.

[0057] During the training process, the assembled video data is first collected and annotated, and a training sample set is generated after preprocessing. For example, image sampling, normalization, data enhancement, etc. Initialize the weight parameters of the 3D convolutional neural network, for example, you can use pre-trained model weights or random initialization. Use the cross entropy loss function and select the stochastic gradient descent method as the optimizer to train the training samples in batches until the model converges.

[0058] Multiple assembly models have been generated in the model library in advance. When each assembly model is stored in the model library, a corresponding label is added to describe its corresponding assembly attributes, such as wrench type and bolt arrangement. At this time, the corresponding assembly model is selected as the designated model by label matching, which can also be called the current corresponding standard model.

[0059] S104: Perform machine vision analysis based on the assembly video to determine the corresponding bolt cluster assembly process, and control the bolt cluster assembly process through the specified model.

[0060] Similar to determining assembly properties, a pre-trained neural network model can be used to perform machine vision analysis based on the assembly video, and then a corresponding second analysis result can be obtained based on the machine vision analysis. The second analysis result includes the tightening sequence and the tightening steps of each bolt as the bolt cluster assembly process. Of course, for the convenience of comparison, the bolt cluster assembly process can also be standardized to generate an actual assembly model for comparison with the specified model.

[0061] If the comparison results are consistent, it is considered that the current production process has no abnormalities and meets the corresponding production requirements. If it is determined that there are abnormal behaviors that do not comply with the specified model during the bolt cluster assembly process, for example, there is an error in the tightening sequence of the bolts or the tightening step of a certain bolt, which does not comply with the regulations in the specified model. At this time, the alarm category to which it belongs is determined based on the abnormal behavior, for example, whether it is a tightening step violation or a tightening sequence violation. At the same time, an alarm message is generated based on the alarm classification and abnormal behavior. The alarm information includes information about the violating personnel, information about the violating workstation / production line / workshop / factory, and time of violation. Of course, the alarm information can also be accompanied by corresponding alarm content, which can be presented in the form of violation images, videos, animated images, etc.

[0062] According to the production process, select the corresponding alarm method and alarm personnel, and alarm through alarm information. Among them, the alarm method can be local alarm through sound and light alarm and cloud remote alarm through phone or text message. The alarm personnel can be the operator himself, team leader, section chief, factory manager, etc.

[0063] Through monitoring and alarm, operators can be urged to strictly abide by the requirements of the operating instructions to perform assembly operations, improve the quality and efficiency of operators' operations, comply with safety production regulations, reduce the accident rate, and provide decision-making basis for rewards and punishments for decision-makers.

[0064] Furthermore, for the production process, according to the preset control cycle, the alarm information generated within the control cycle is counted and visualized.

[0065] Among them, the content of the visual display may include the alarm content in, for example, the alarm content includes the video, image, animated picture, etc. of each compliant / illegal assembly operation. It is also possible to analyze the statistical results to obtain the characteristic dimensions where the probability of abnormal information occurrence is higher than the preset probability. For example, the characteristic dimensions include time period, alarm classification, alarm location, personnel, etc., among which the time period may include statistical periods such as day / week / month / quarter / year, and the alarm classification may include step violation, sequence violation, etc. The visualization of characteristic dimensions can be in the form of column charts, line charts, pie charts, bar charts, area charts, scatter charts, stock price charts, radar charts, combination charts, dynamic charts, etc., and the visualization carriers may include handheld terminals, workshop large screens, MES system cabs, etc.

[0066] Of course, it is also possible to optimize based on statistical results and feature dimensions, such as optimizing the production process or training operators.

[0067] 1. The assembly process is digitally supervised, and full-process automated monitoring is achieved through machine vision, replacing traditional manual visual inspections, solving the problem of uncontrollable execution of work instructions, improving the accuracy of compliance inspections, and being able to capture illegal operations in real time, shortening alarm response time.

[0068] 2. Realize the traceability of the production process, associate and store video data with operation logs, support reverse tracing by bolt number, and locate abnormal events to specific workstations / operators / time nodes through management and control. It can also analyze historical data and identify potential process defects.

[0069] like Fig.10 As shown, the embodiment of the present application also provides a bolt cluster quality control device based on machine vision, including: at least one processor; and, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the bolt cluster quality control method based on machine vision as described in any of the above embodiments.

[0070] An embodiment of the present application further provides a non-volatile computer storage medium storing computer executable instructions, wherein the computer executable instructions are configured to be: a bolt cluster quality control method based on machine vision as described in any of the above embodiments.

[0071] Each embodiment in this application is described in a progressive manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the device and medium embodiments, since they are basically similar to the method embodiments, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiments.

[0072] The devices and media provided in the embodiments of the present application correspond one-to-one to the methods. Therefore, the devices and media also have similar beneficial technical effects as the corresponding methods. Since the beneficial technical effects of the methods have been described in detail above, the beneficial technical effects of the devices and media will not be repeated here.

[0073] The above is only an embodiment of the present application and is not intended to limit the present application. For those skilled in the art, the present application may have various changes and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present application should be included in the scope of the claims of the present application.

Claims

1. A bolt cluster quality control method based on machine vision, characterized in that: include: Based on the production process, determine the bolt clusters included, and generate corresponding assembly models for the bolt clusters according to their corresponding assembly attributes; Obtaining an assembly video based on the production process through a machine vision device; Performing machine vision analysis on the assembly video to determine the corresponding assembly attributes, and selecting a matching designated model from the generated assembly model; Machine vision analysis is performed based on the assembly video to determine the corresponding bolt cluster assembly process, and the bolt cluster assembly process is controlled through the specified model.

2. The bolt cluster quality control method based on machine vision according to claim 1 is characterized in that: For the bolt cluster, a corresponding assembly model is generated according to its corresponding assembly attributes, specifically including: For the bolt cluster, determine the type of wrench to be used according to the production process; Determining, according to the wrench type, a tightening step for each bolt in the bolt cluster; For the bolt cluster, determining the corresponding bolt arrangement and the stiffness of each bolt therein; Determining a tightening sequence of the bolt cluster according to the bolt arrangement and the stiffness of each bolt; According to the tightening steps and the tightening sequence, a corresponding assembly model is generated.

3. The bolt cluster quality control method based on machine vision according to claim 2 is characterized in that: Determining the tightening sequence of the bolt cluster according to the bolt arrangement and the stiffness of each bolt includes: Determine a corresponding default tightening sequence according to the bolt sorting method, and determine the tightening priority between the bolts according to the default tightening sequence; Arrange the bolts in descending order according to their stiffness to obtain a tightening sequence of the bolt cluster; For bolts with the same stiffness, the sub-tightening sequence of some bolts with the same stiffness is adjusted according to their corresponding tightening priorities.

4. The bolt cluster quality control method based on machine vision according to claim 2 is characterized in that: After determining the tightening sequence of the bolt cluster according to the bolt arrangement and the stiffness of each bolt, the method further includes: Determine a corresponding prohibited tightening sequence according to the bolt sorting mode, and determine whether the current tightening sequence of the bolt cluster hits the prohibited tightening sequence; If it is hit, then traverse to obtain all tightening sequences corresponding to the bolt cluster, and among all tightening sequences, determine the specified tightening sequence that does not hit the prohibited tightening sequence, and determine the adjustment score between each specified tightening sequence and the current tightening sequence; The specified tightening sequence with the lowest adjustment score is selected to replace the current tightening sequence for the bolt cluster.

5. The bolt cluster quality control method based on machine vision according to claim 4 is characterized in that: The adjustment score is obtained by weighted summing of sub-adjustment scores corresponding to each bolt position in the bolt arrangement, the weight of each bolt position is determined according to its corresponding stiffness difference, and the sub-adjustment score is obtained according to the sequence difference of the bolt positions; Among them, the higher the stiffness difference, the higher the weight; the higher the rank difference, the higher the sub-adjustment score.

6. The bolt cluster quality control method based on machine vision according to claim 1 is characterized in that: Performing machine vision analysis on the assembly video to determine the corresponding assembly attributes, specifically including: performing machine vision analysis based on the assembly video using a pre-trained neural network model; Determine the corresponding wrench type and bolt arrangement according to the first analysis result corresponding to the machine vision analysis; Machine vision analysis is performed based on the assembly video to determine the corresponding bolt cluster assembly process, specifically including: performing machine vision analysis based on the assembly video using a pre-trained neural network model; According to the second analysis result corresponding to the machine vision analysis, the corresponding tightening sequence and the tightening steps of each bolt are determined.

7. The bolt cluster quality control method based on machine vision according to claim 1 is characterized in that: The bolt cluster assembly process is controlled by the specified model, specifically including: Determining that during the bolt cluster assembly process, there are abnormal behaviors that do not conform to the specified model; According to the abnormal behavior, determining the alarm category to which it belongs, and generating alarm information according to the alarm category and the abnormal behavior; According to the production process, the corresponding alarm method and alarm personnel are selected, and the alarm is issued through the alarm information.

8. The bolt cluster quality control method based on machine vision according to claim 7 is characterized in that: The method further comprises: For the production process, according to the preset control cycle, the alarm information generated within the control cycle is counted and visualized; According to the statistical results, the feature dimensions in which the probability of abnormal information occurring is higher than the preset probability are obtained, so as to facilitate optimization according to the feature dimensions.

9. A bolt cluster quality control device based on machine vision, characterized in that: include: at least one processor; as well as, a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the machine vision-based bolt cluster quality control method as described in any one of claims 1 to 8.

10. A non-volatile computer storage medium storing computer executable instructions, characterized in that: The computer executable instructions are set to: the bolt cluster quality control method based on machine vision as described in any one of claims 1 to 8.

Images