A method for analyzing abnormal categories of a tightening curve based on a multi-modal large model

Abstract

The application provides a method for analyzing abnormal categories of tightening curves based on a multimodal large model, which comprises: collecting tightening data of each station on a production line, and storing torque sequences and angle sequences in a tightening process into an object storage and generating a task queue by an edge processor; performing image standardization preprocessing on the torque sequences and the angle sequences; inputting a standardization-processed curve image to be detected, a normal curve sample image, corresponding normal curve geometric feature descriptions and abnormal judgment rules into a qwen2.5-vl multimodal large model; determining whether the curve to be detected is abnormal by using the qwen2.5-vl multimodal large model, and outputting a judgment result as normal or abnormal; if the qwen2.5-vl multimodal large model determines that the result is abnormal, analyzing the standardization-processed curve image to be detected, sample images of different abnormal categories and geometric feature descriptions of different abnormal categories, and outputting abnormal categories and analysis results.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing and industrial quality detection, in particular to a method for analyzing abnormal categories of tightening curves based on a multi-modal large model. BACKGROUND

[0002] The existing tightening process basically judges whether it is qualified according to the tightening result, in order to ensure the robustness of the tightening process, some parameters can be added as monitoring indicators: such as using torque as the tightening result, the tightening angle can be used as the monitoring parameter; such as using the tightening angle as the tightening result, the tightening torque can be used as the monitoring parameter. Since the tightening process involves many influencing factors, single parameter monitoring cannot effectively reflect the variation of the tightening process. The tightening curve is a comprehensive influence of multiple parameters, and through the analysis of the tightening curve, the abnormal category of the tightening process can be effectively determined. In the production process, several hundred tightening data will be generated every day for each tightening process. Due to the large amount of data, manual data processing is required, which needs to be compared and analyzed one by one, and the efficiency is very low, and it is easy to miss and misjudge.

[0003] In the prior art, specific calculation methods are used, such as torque increment corresponding to angle increment, angle increment corresponding to torque increment, peak value, slope corresponding to torque-angle curve, etc. Based on mathematical calculation, the processing speed of data is slow. Even if the abnormal detection scheme based on machine learning is used, due to the need for a large amount of positive and negative sample data for training, it not only consumes time, but also is extremely difficult to collect abnormal samples, resulting in that the machine learning method is difficult to land. SUMMARY

[0004] In view of the defects in the prior art, the purpose of the present application is to provide a method for analyzing abnormal categories of tightening curves based on a multi-modal large model.

[0005] According to the method for analyzing abnormal categories of tightening curves based on a multi-modal large model provided by the present application, the method comprises the following steps:

[0006] S1: Collecting tightening data of each station on the production line, and storing torque sequence TorqueTrace T(t) and angle sequence AngleTrace A(t) in the tightening process into object storage and generating a task queue by an edge processor;

[0007] S2: Image standardization preprocessing is performed on the torque sequence TorqueTrace T(t) and angle sequence AngleTrace A(t) data;

[0008] S3: Inputting the standardization-processed curve image to be detected, a plurality of standardization-processed normal curve sample images of the same process, corresponding normal curve geometric feature descriptions and abnormal judgment rules into a qwen2.5-vl multi-modal large model.

[0009] S4: determining whether the to-be-inspected curve is abnormal by using the qwen2.5-vl multi-modal large model, and outputting a determination result as normal or abnormal;

[0010] S5: if the determination result of the qwen2.5-vl multi-modal large model is abnormal, analyzing the standardized to-be-inspected curve image and the sample images of different abnormal categories and the geometric feature descriptions of different abnormal categories, and outputting an abnormal category;

[0011] S6: collecting normal quantity, abnormal quantity and proportion of each abnormal category from the dimensions of station, program and time, and generating a trend chart of each abnormal category;

[0012] The image standardization preprocessing step in the step S2 comprises:

[0013] S21: positioning an angle starting point at which the torque first reaches a preset torque value;

[0014] S22: cropping a curve segment in a preset angle interval after the angle starting point and enlarging the display;

[0015] S23: aligning the curve coordinate system to eliminate process differences.

[0016] Preferably, in the image standardization preprocessing step, the preset torque value of the angle starting point is 27.5 N•m, and the preset angle is 120°.

[0017] Preferably, the qwen2.5-vl multi-modal large model is a graph-text fusion model with Few-shot (few-shot) learning ability.

[0018] Preferably, the task queue contains station number, program name, program number, VIN, TorqueTrace, AngleTrace and timestamp information of the station.

[0019] Preferably, the task queue adopts a first-in first-out (FIFO) strategy, and the tasks entered first are processed first, and the maximum number of concurrent tasks is set based on the hardware configuration of the server and the graphics card to process the tasks in parallel.

[0020] Preferably, the abnormal categories include high friction, sleeve rupture or wear, connection adhesion, bolt yield deformation and part surface oil stain.

[0021] Compared with the prior art, the present application has the following beneficial effects:

[0022] 1. The method for analyzing abnormal categories of tightening curves based on a multi-modal large model provided by the application can effectively solve the full-scene analysis of tightening curves, and can solve the problems of missing, complex processing process and data redundancy caused by single-task analysis for panoramic analysis of curves.

[0023] 2. The method for analyzing abnormal categories of tightening curves based on a multi-modal large model provided by the application, the qwen2.5-vl multi-modal large model has Few-shot learning ability, and only a small amount of normal sample pictures are needed to accurately determine whether it is abnormal, without the need to collect a large amount of abnormal data.

[0024] 3. The method for analyzing abnormal categories of tightening curves based on a multi-modal large model provided by the application has high flexibility, and various types of normal curve business knowledge of natural language can be configured, without the need for a large amount of data to train the model, so that the algorithm can be migrated to multiple curve abnormality judgments. BRIEF DESCRIPTION OF DRAWINGS

[0025] Other features, objects and advantages of the application will become more apparent from the following detailed description of non-limiting embodiments, made with reference to the accompanying drawings:

[0026] Figure 1 The flowchart of the method for analyzing abnormal categories of tightening curves based on a multi-modal large model in the embodiment;

[0027] Figure 2 The first normal curve sample graph of the same process in the embodiment;

[0028] Figure 3 The second normal curve sample graph of the same process in the embodiment;

[0029] Figure 4 The third normal curve sample graph of the same process in the embodiment;

[0030] Figure 5 The first abnormal category curve sample graph in the embodiment;

[0031] Figure 6 The second abnormal category curve sample graph in the embodiment;

[0032] Figure 7 The third abnormal category curve sample graph in the embodiment;

[0033] Figure 8 The fourth abnormal category curve sample graph in the embodiment. DETAILED DESCRIPTION

[0034] In order to make the purposes, technical solutions and advantages of the embodiments of the present application clearer, the technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only some of the embodiments of the present application, rather than all the embodiments. The components of the embodiments of the present application described and shown in the drawings can be arranged and designed in various different configurations.

[0035] Therefore, the detailed description of the embodiments of the present application provided below in the drawings is not intended to limit the scope of the claimed present application, but only represents selected embodiments of the present application. All other embodiments obtained by those of ordinary skill in the art based on the embodiments in the present application without creative labor are within the scope of protection of the present application.

[0036] It should be noted that: similar reference numbers and letters represent similar items in the following drawings, therefore, once an item is defined in one drawing, it does not need to be further defined and explained in subsequent drawings. In addition, all directional indications (such as up, down, left, right, front, back, bottom, etc.) in the present application are only used to explain the relative positional relationship, movement condition, etc. between components in a certain posture (as shown in the drawings), if the certain posture changes, the directional indications also change accordingly. Further, the description involving "first", "second", etc. in the application is only for the purpose of description, and cannot be understood as indicating or implying the relative importance of the indicated technical features or implicitly indicating the number of the indicated technical features.

[0037] As shown in Figure 1 , it is a method flowchart for analyzing the abnormal category of the tightening curve based on a multi-modal large model in the present embodiment, the method includes the following steps:

[0038] S1: Collect the tightening data of each station on the production line, the edge processor stores the torque sequence TorqueTrace T(t) and the angle sequence AngleTrace A(t) in the tightening process into the object storage and generates a task queue, the task queue contains the station number, program name, program number, VIN, TorqueTrace, AngleTrace, timestamp information of the station; further, the task queue adopts the first-in first-out FIFO strategy, the tasks entering the queue first are processed first, and the maximum number of concurrent tasks is set based on the hardware configuration of the server and the graphics card to process the tasks in parallel, so as to maximize concurrent processing and avoid congestion and repeated processing.

[0039] S2: Perform image standardization preprocessing on the torque sequence TorqueTrace T(t) and the angle sequence AngleTrace A(t) data;

[0040] In the present embodiment, the steps of image standardization preprocessing include:

[0041] S21: Locate the angle start point where the torque first reaches the preset torque value. In this embodiment, the torque preset value of the angle start point is 27.5 N•m.

[0042] S22: Crop and enlarge the curve segment in the preset angle interval after the angle start point. In this embodiment, the preset angle is 120°.

[0043] S23: Align the curve coordinate system to eliminate process differences. The qwen2.5-vl multimodal large model can more easily find abnormalities.

[0044] S3: Input the standardized processed test curve image, standardized processed normal curve sample images of the same process, corresponding normal curve geometric features and quantitative descriptions, such as curve continuously rising within 120°, peak value in the specified range, and abnormality determination rules, such as key inflection point torque threshold greater than 150 N•m, peak range, geometric features, and other business rules, into the qwen2.5-vl multimodal large model. In this embodiment, 3 standardized processed normal curve sample images of the same process are input, as shown in Figures 2-4 .

[0045] S4: Determine whether the test curve is abnormal using the qwen2.5-vl multimodal large model, and output the determination result as normal or abnormal. The qwen2.5-vl multimodal large model is a graph-text fusion model with Few-shot (few-shot) learning capability, so that only a small number of example pictures and short, quantifiable text descriptions are input into the qwen2.5-vl multimodal large model each time of analysis, allowing the model to have a clear reference for determining “normal / abnormal” and “abnormal category”.

[0046] S5: If the qwen2.5-vl multimodal large model determines that the result is abnormal, analyze the standardized processed test curve image and at least one sample image of each different abnormal category, as shown in Figures 5-7 , and the geometric feature descriptions of different abnormal categories, and output the abnormal category. The abnormal category includes high friction, sleeve rupture or wear, connecting piece adhesion, bolt yield deformation, and part surface oil stain.

[0047] The text descriptions and sample images used in steps S4 and S5 can be configured specifically for curve types. When a new abnormal type appears, only sample images and brief descriptions need to be supplemented for expansion, without the need to train the model.

[0048] S6: From the workstation, program, and time dimensions, summarize the normal quantity, abnormal quantity, and proportion of each abnormal category, and generate a trend chart for each abnormal category.

[0049] The method for analyzing abnormal classes of a tightening curve based on a multimodal large model provided by the application can effectively solve the full-scene analysis of the tightening curve, and can solve the problems of missing, complex processing and data redundancy caused by single-task analysis for panoramic analysis of the curve; the qwen2.5-vl multimodal large model has the few-shot learning ability, and can accurately determine whether it is abnormal by using only a small amount of normal sample pictures, without collecting a large amount of abnormal data; the flexibility is high, and various types of business knowledge of normal curves can be configured through natural language, so that the algorithm can be migrated to multiple curve abnormality judgments without training the model with a large amount of data.

[0050] The specific embodiments of the application are described above, and through the above description, relevant personnel can make various changes and modifications without deviating from the technical idea of the application.

Claims

1. A method for analyzing a tightening curve abnormality category based on a multi-modal large model, characterized by, The method comprises the following steps: S1: Collecting tightening data of each station on the production line, and storing torque sequence TorqueTrace T(t) and angle sequence AngleTrace A(t) in the tightening process into object storage and generating a task queue by an edge processor; S2: Image standardization preprocessing is performed on the torque sequence TorqueTrace T(t) and angle sequence AngleTrace A(t) data; S3: Inputting the normalized processed inspection curve image, a plurality of normal curve sample images of the same process after standardization processing, corresponding normal curve geometric feature descriptions and abnormality judgment rules into a qwen2.5-vl multi-modal large model; S4: Using the qwen2.5-vl multi-modal large model to determine whether the inspection curve is abnormal, and outputting a determination result as normal or abnormal; S5: If the qwen2.5-vl multi-modal large model determination result is abnormal, then analyzing the normalized processed inspection curve image and sample images of different abnormal categories and geometric feature descriptions of different abnormal categories, and outputting an abnormal category; S6: From the station, program and time dimensions, the analysis result is summarized as normal quantity, abnormal quantity and proportion of each abnormal category, and a trend chart of each abnormal category is generated; The image standardization preprocessing step in step S2 comprises: S21: Positioning the angle starting point at which the torque first reaches a preset torque value; S22: Cropping and enlarging the curve segment in the preset angle interval after the angle starting point; S23: Aligning the curve coordinate system to eliminate process differences. 2.The method of claim 1, wherein, In the image standardization preprocessing step, the torque preset value of the angle starting point is 27.5 N•m, and the preset angle is 120°. 3.The method of claim 1, wherein, The qwen2.5-vl multi-modal large model is a graph-text fusion model with few-shot learning ability. 4.The method of claim 1, wherein, The task queue contains station number, program name, program number, VIN, TorqueTrace, AngleTrace, timestamp information of the station. 5.The method for analyzing a tightening curve abnormality category based on a multi-modal large model according to claim 1, wherein, The task queue adopts a first-in first-out FIFO strategy, and the tasks entered first are processed first, and the maximum number of concurrent tasks is set based on the hardware configuration of the server and the graphics card to process the tasks in parallel. 6.The method for analyzing a tightening curve abnormality category based on a multi-modal large model according to claim 1, wherein, The abnormal categories include high friction, sleeve rupture or wear, connector adhesion, bolt yield deformation and part surface oil stain.

Images