Production line operation AI detection system based on deep learning

Abstract

The invention discloses a production line operation AI detection system based on deep learning, and relates to the technical field of intelligent manufacturing. Comprising a multi-modal sensing module which monitors a product through an industrial camera and a laser sensor to obtain multi-modal data; the decision processing module is used for acquiring a comprehensive risk index according to the multi-modal data, setting a hierarchical control strategy according to the comprehensive risk index and determining a compensation instruction; and the multi-target processing module is used for determining a quality-efficiency tradeoff coefficient through the multi-mode sensing module and the decision processing module, and adjusting the operation mode of the production system. The defect probability, vibration and temperature rise data are fused to obtain the comprehensive risk index, and the production line state is divided into three levels of different responses through the comprehensive risk index, so that production continuity and quality control can be balanced, and unnecessary shutdown loss is reduced.

Description

Technical Field

[0001] The present invention relates to the field of intelligent manufacturing technology, and specifically to an AI detection system for production line operation based on deep learning. Background Art

[0002] In modern manufacturing, stable, efficient, and safe production line operation is a key element in ensuring product quality, improving production efficiency, and reducing operating costs. With the continuous advancement of industrial automation and intelligence, the need for real-time, accurate monitoring of production line operation status and anomaly detection is becoming increasingly urgent. Traditional production line monitoring relies primarily on the following methods:

[0003] 1. Rule-based threshold alarm systems: These systems pre-set thresholds for physical parameters of sensors (such as temperature, pressure, vibration, and current) and trigger alarms when the monitored values ​​exceed the set range. Their advantages are simplicity and low cost. However, they also have significant limitations, such as rigidity, poor adaptability, high rates of false alarms and missed alarms, and reliance on expert experience.

[0004] 2. Methods based on traditional signal processing and statistical analysis: such as Fourier transform, wavelet analysis, principal component analysis (PCA), and support vector machines (SVM). These methods can process more complex signal features, extract time-frequency domain information, or perform dimensionality reduction analysis. However, they not only require complex feature engineering and have limited generalization capabilities, but also struggle to process high-dimensional, unstructured data.

[0005] 3. Expert system or knowledge graph-based methods: These utilize domain knowledge bases for reasoning and judgment. However, not only is the knowledge acquisition process cumbersome, difficult, and costly, but the reasoning capabilities are also limited when faced with data noise, incomplete information, or ambiguity.

[0006] The Chinese invention patent with publication number CN110806743A discloses an artificial intelligence-based equipment fault detection and early warning system, including an acquisition module, a processing module and an early warning module. The processing module includes a data training unit, a prediction unit and a judgment unit: the acquisition module includes several sensors, which are installed on different components of the industrial control equipment and are used to collect real-time status data of the industrial control equipment; the data training unit is used to train the collected equipment status historical data and establish an equipment abnormality prediction model; the prediction unit is used to predict the equipment status after the collected new equipment status data is input into the equipment abnormality prediction model; the judgment unit is used to notify the early warning module to issue an early warning when it predicts that the equipment status is abnormal.

[0007] While the aforementioned technical solution can provide early warning of equipment failures based on the equipment's operating status, preventing sudden failures and minimizing the economic losses caused by downtime for repairs, during intelligent automotive welding inspections, when the AI ​​quality inspection system, based on solder joint image feature analysis, gives a 92% confidence level of compliance, while manual inspectors determine the presence of a false solder joint based on visual inspection or process standards, the production line must be interrupted to await manual review, reducing production efficiency and impacting the JIT production cycle. Summary of the Invention

[0008] The purpose of the present invention is to provide a production line operation AI detection system based on deep learning to solve the problems raised in the above background technology.

[0009] To achieve the above objectives, the present invention provides the following technical solutions: a production line operation AI detection system based on deep learning, comprising:

[0010] Multimodal perception module: monitors products through industrial cameras and laser sensors to obtain multimodal data of products;

[0011] Decision-making processing module: obtains a comprehensive risk index based on the multimodal data, sets a hierarchical control strategy based on the comprehensive risk index, and determines compensation instructions, including:

[0012] SB1: Dynamic decision-making: Defect probability, assembly clearance to be measured, and vibration amplitude are integrated to determine a comprehensive risk index, and a hierarchical control strategy is set based on the comprehensive risk index;

[0013] SB2: Self-learning processing: Determines compensation instructions for re-inspected products based on critical assembly tolerances for different product models;

[0014] Multi-objective processing module: determines the quality-efficiency trade-off coefficient through the multi-modal perception module and the decision processing module, and adjusts the operation mode of the production system according to the quality-efficiency trade-off coefficient.

[0015] Furthermore, multimodal data of the product is obtained, including:

[0016] SA1: Visual Inspection: The preprocessed product image is used as the input of the improved ResNet-50 model. The output is the flattened weighted feature map. Based on the flattened weighted feature map, the linearly transformed raw score is determined. The defect probability is obtained from the linearly transformed raw score.

[0017] SA2: Laser ranging: The assembly gap is measured by a laser sensor to obtain the assembly gap to be measured;

[0018] SA3: Vibration detection: The Lucas-Kanade optical flow algorithm and the feature point image are used to determine the displacement of the feature points, and the vibration amplitude is determined based on the displacement of the feature points.

[0019] Furthermore, a hierarchical control strategy is set up, including:

[0020] SB1.1: Build a risk quantification model: Based on the defect probability, the assembly clearance to be tested, and the vibration amplitude, determine the comprehensive risk index, specifically:

[0021]

[0022] in: is the comprehensive risk index, is the base of natural logarithm, is the defect probability after Sigmoid function conversion, is the assembly clearance to be measured, is the vibration amplitude, 、 、 is the weight coefficient;

[0023] SB1.2: Hierarchical Control: Compare the comprehensive risk index with the preset risk threshold range and determine the control strategy based on the comparison results, specifically:

[0024] When the comprehensive risk index is less than the lower limit threshold of the preset risk threshold range, the product will continue to be inspected; when the comprehensive risk index is within the preset risk threshold range, the transmission rate of the conveyor belt will be reduced and the product will be re-inspected; when the comprehensive risk index is greater than the upper limit threshold of the preset risk threshold range, an alarm signal will be triggered, and the machine will be shut down. At the same time, the defective part will be scanned by 3D line laser to obtain the three-dimensional point cloud data of the defect.

[0025] Furthermore, based on the defect probability, assembly gap and vibration amplitude of multiple groups of samples, the normalized defect probability, normalized assembly gap and normalized vibration amplitude are obtained. At the same time, based on the set initial weight coefficient, normalized defect probability, normalized assembly gap and normalized vibration amplitude, the initial comprehensive risk index of multiple samples is determined, and the initial comprehensive risk index and the detection results are used as inputs of the maximum likelihood estimation model, and the updated weight coefficient is obtained as output.

[0026] Furthermore, the comprehensive risk index is modified according to the temperature rise rate and the vibration amplitude, including:

[0027] SB1.1.1: Temperature rise monitoring: Scan product components with a thermal imager to obtain infrared images of the product components, and determine the temperature rise rate based on the infrared images;

[0028] SB1.1.2: Determine Correction Factors: Determine a primary correction factor based on the temperature rise rate and the preset temperature rise threshold range. Determine a secondary correction factor based on the vibration amplitude and the preset vibration threshold range. Combine the primary and secondary correction factors to construct a correction factor table.

[0029] SB1.1.3: Determine the correction index: Based on the vibration amplitude and temperature rise rate, determine the correction coefficient through the correction coefficient table, and correct the comprehensive risk index based on the correction coefficient to obtain the corrected comprehensive risk index.

[0030] Furthermore, the temperature rise rate is compared with a preset temperature rise threshold range, and a first-level correction coefficient is determined based on the comparison result, specifically:

[0031] When the temperature rise rate is less than the lower limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 1; when the temperature rise rate is within the preset temperature rise threshold range, the first-level correction coefficient is set to 1.5; when the temperature rise rate is greater than the upper limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 2;

[0032] The vibration amplitude is compared with a preset vibration threshold range, and a secondary correction coefficient is determined based on the comparison result, specifically:

[0033] When the vibration amplitude is less than the lower limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 1; when the vibration amplitude is within the preset vibration threshold range, the secondary correction coefficient is set to 1.5; when the vibration amplitude is greater than the upper limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 2.

[0034] Furthermore, compensation instructions for re-inspected products are determined, including:

[0035] SB2.1: Tolerance Library Management: Build a product tolerance library based on the critical tolerance and allowable compensation times for each product model. Determine the compensation status and deviation ratio of the current product based on the product tolerance library.

[0036] SB2.2: Compensation Instruction Generation: Compare the deviation ratio with a preset deviation threshold range and determine a compensation action based on the comparison result, specifically:

[0037] When the deviation ratio is less than the lower limit threshold of the preset deviation threshold range, no compensation is made for the re-inspection behavior of the current product; when the deviation ratio is within the preset deviation threshold range, force compensation is made for the re-inspection behavior of the current product; when the deviation ratio is greater than the upper limit threshold of the preset deviation threshold range, an alarm signal is triggered, and the machine is shut down. At the same time, the process of the current product is reviewed and the production process of the current product is re-determined.

[0038] Furthermore, when force compensation is performed on the re-inspection behavior of the current product, the compensation force increment is determined based on the assembly clearance to be tested and the critical tolerance of the current product, specifically:

[0039]

[0040] in: To compensate for the force increment, is the proportionality coefficient, is the integration coefficient, is the assembly clearance to be measured, is the critical tolerance.

[0041] Furthermore, the operating mode of the production system is adjusted, including:

[0042] SC1: Determine the trade-off coefficient: Determine the product's quality loss and detection delay time through the multimodal perception module and the decision processing module, determine the product's standard cycle time through the MES system, and determine the quality-efficiency trade-off coefficient based on the product's quality loss, detection delay time, and standard cycle time;

[0043] SC2: Determine the working mode: Compare the quality-efficiency trade-off coefficient with a preset trade-off threshold, and determine the operating mode of the production system based on the comparison result, specifically:

[0044] When the quality-efficiency trade-off coefficient is greater than a preset trade-off threshold, the quality priority mode is activated; otherwise, the efficiency priority mode is activated.

[0045] Furthermore, based on the quality-efficiency trade-off coefficients of the plurality of sample data, a trade-off coefficient range of the plurality of sample data is determined, and the trade-off coefficient range is divided into a plurality of intervals. At the same time, the plurality of sample data are divided according to the intervals. Based on the average total cost of the plurality of sample data in each interval and the trade-off coefficient interval, a trade-off coefficient-cost fitting formula is constructed. The preset trade-off threshold is determined by the quadratic term coefficient and the linear term coefficient in the trade-off coefficient-cost fitting formula, specifically:

[0046]

[0047] in: To preset the trade-off threshold, is the coefficient of the quadratic term, is the coefficient of the first-order term.

[0048] Compared with the prior art, the present invention has the following beneficial effects:

[0049] First, the present invention integrates defect probability, vibration, and temperature rise data to obtain a comprehensive risk index. This comprehensive risk index then divides the production line status into three different response levels, thereby balancing production continuity and quality control and reducing unnecessary downtime losses.

[0050] Second, the present invention dynamically matches the tolerance library with the product model, triggers PID compensation based on the deviation ratio, and determines the corresponding compensation force increment, thereby correcting the assembly deviation and improving production accuracy.

[0051] Thirdly, the present invention updates the weight coefficients in the comprehensive risk index through maximum likelihood estimation, so that the model can continuously adapt to changes in the production line, thereby improving the model's adaptability;

[0052] Fourthly, the present invention sets preset trade-off thresholds through the extreme points of the cost model and intelligently switches between quality / efficiency priority modes, thereby maximizing production efficiency while ensuring key quality indicators. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] Figure 1 This is a system block diagram of the AI ​​detection system for the production line of the present invention;

[0054] Figure 2 It is a fitting diagram between the trade-off coefficient and cost in the present invention;

[0055] Figure 3 This is a comparison chart of the effects of the AI ​​detection system in the present invention;

[0056] Figure 4 This is a thermodynamic diagram of the temperature rise-vibration combined correction coefficient in the present invention. DETAILED DESCRIPTION

[0057] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0058] In the process of intelligent inspection of automobile welding, when the AI ​​quality inspection system gives a 92% confidence level based on the analysis of the weld image features, and the manual quality inspector determines the existence of a hidden danger of cold welding based on visual inspection or process standards, the production line must be interrupted to wait for manual review, which will reduce production efficiency and affect the JIT production rhythm. The technical solution of this application uses a multimodal perception module to collect product defect probability, assembly gap and vibration amplitude data in real time, and at the same time fuses the collected multi-source information to generate a comprehensive risk index and implement corresponding control strategies. At the same time, the production mode is dynamically adjusted through the quality-efficiency trade-off coefficient, thereby realizing the coordinated optimization of quality and efficiency in intelligent manufacturing.

[0059] Example 1

[0060] refer to Figure 1-Figure 3 This embodiment provides a deep learning-based AI detection system for production line operation. The system includes a multimodal perception module, a decision-making processing module, and a multi-target processing module. In this embodiment, the multimodal perception module is used to identify defects such as scratches and missing materials on the product surface, obtain the corresponding normalized defect probability, and monitor the assembly gap in real time using a laser displacement sensor and obtain the vibration amplitude of moving parts using a high-speed camera. The details are as follows:

[0061] Step SA1: Visual Inspection. This involves capturing images of products on a conveyor belt using an industrial camera (such as the Basler ace acA2000-50gm). At least three images of each product are taken from different angles, covering the front, center, and back, ultimately covering the entire surface. Furthermore, the pixel values ​​in each acquired image are normalized to map them to the range [0, 1]. Histogram equalization is then performed on the normalized pixel values ​​to produce a pre-processed product image.

[0062] In this example, a channel attention module is inserted after the third residual block of the ResNet-50 model to construct an improved ResNet-50 model, which enhances sensitivity to small defects (such as 0.1mm wide scratches). Specifically, the preprocessed product image is used as input to the improved ResNet-50 model, and the output is the corresponding flattened weighted feature map data.

[0063] Furthermore, based on the flattened data of the weighted feature map obtained from the output, the corresponding original score after linear transformation is determined, specifically:

[0064]

[0065] in: is the original score after linear transformation, is the weight parameter of the fully connected layer, is the data after flattening the weighted feature map, is the bias term of the fully connected layer, is the inversion of the weight parameter.

[0066] Furthermore, the corresponding defect probability is determined based on the original score obtained after linear transformation, specifically:

[0067]

[0068] in: is the defect probability after Sigmoid function conversion, is the base of natural logarithm, is the original score after linear transformation.

[0069] That is to say, when the data is flattened by the weighted feature map and the original score after linear transformation is obtained as 2.1, the corresponding defect probability is 0.89.

[0070] Step SA2: Laser ranging. A Keyence LJ-V7080 laser sensor is mounted vertically above the surface being measured, 50 mm from the reference surface, with the angle between the incident laser and the reflected light path set at 30°. Specifically, the laser sensor in this embodiment emits a 650 nm red laser with a spot diameter of 0.8 mm.

[0071] In this embodiment, a laser sensor emits a 650nm laser beam to measure the assembly gap, and the corresponding assembly gap is obtained, specifically:

[0072]

[0073] in: is the assembly clearance to be measured, is the fixed distance from the sensor to the reference surface, is the angle between the reflected light and the incident light.

[0074] Specifically, when the reflection angle measurement value is 32° and the fixed distance from the sensor to the reference surface in this embodiment is 50 mm, the corresponding assembly gap is 14.34 mm.

[0075] Step SA3: Vibration Detection. A Photron FASTCAM Mini AX200 camera is mounted on a three-axis adjustable gimbal, with the lens axis perpendicular to the motor housing surface. The camera then captures the detection feature points in pre-marked areas of the motor housing (such as bolt heads and cooling fin edges) to obtain corresponding feature point images.

[0076] In this embodiment, the Lucas-Kanade optical flow algorithm is used to determine the corresponding feature point displacement based on the acquired feature point image, specifically:

[0077]

[0078] in: is the instantaneous displacement of the i-th feature point, is the horizontal gradient of the image, is the vertical gradient of the image, is the grayscale change between adjacent frames, is the optical flow velocity in the x direction, is the optical flow velocity in the y direction.

[0079] Furthermore, the corresponding instantaneous vibration amplitude is determined based on the displacement of multiple characteristic points obtained, specifically:

[0080]

[0081] in: is the vibration amplitude, is the number of effective feature points, is the index of the valid feature point, is the instantaneous displacement of the i-th feature point.

[0082] In this embodiment, the decision processing module is used to integrate the defect probability, the assembly gap to be measured, and the vibration amplitude obtained by the multimodal perception module to determine the corresponding comprehensive risk index. Based on the determined comprehensive risk index, a hierarchical control strategy is set. Based on the hierarchical control strategy, corresponding compensation instructions are determined. The details are as follows:

[0083] Step SB1: Dynamic decision-making. This involves determining the corresponding comprehensive risk index based on the defect probability, the assembly gap to be measured, and the vibration amplitude. A hierarchical control strategy is then set based on the determined comprehensive risk index. The details are as follows:

[0084] Step SB1.1: Construct a risk quantification model. This is to determine the corresponding comprehensive risk index based on the defect probability, assembly gap to be measured, and vibration amplitude obtained from the multimodal perception module. Specifically, it is:

[0085]

[0086] in: is the comprehensive risk index, is the base of natural logarithm, is the defect probability after Sigmoid function conversion, is the assembly clearance to be measured, is the vibration amplitude, 、 、 is the weight coefficient.

[0087] In this example, a three-month historical database of the production line is obtained through SQL or a time-series database. This database then contains 5,000 sets of samples, each of which includes defect probability, assembly clearance, vibration amplitude, and test results. Furthermore, based on the test results, the samples are divided into normal and faulty samples, and the ratio between normal and faulty samples is adjusted to 3:1 to avoid bias in determining the weight coefficients.

[0088] Furthermore, all defect probabilities, assembly gaps, and vibration amplitudes in the 5,000 groups of samples are normalized to obtain normalized defect probabilities, normalized assembly gaps, and normalized vibration amplitudes. Specifically, in this embodiment, the three weight coefficients in the initially constructed risk quantification model are set to 1, 1, and 0, respectively, and the initial comprehensive risk index corresponding to the 5,000 groups of samples is determined based on the normalized defect probability, normalized assembly gap, normalized vibration amplitude, and initial weight coefficients. At the same time, the initial comprehensive risk index and test results corresponding to each sample are used as inputs to the maximum likelihood estimation model, and through corresponding optimization iterations, the corresponding updated weight coefficients are output.

[0089] Specifically, the formula for obtaining the maximum likelihood estimation model in this embodiment is:

[0090]

[0091] in: is the log-likelihood function, is the total number of training samples, is the index of the training sample, is the true label of the j-th sample, is the comprehensive risk index of the jth sample.

[0092] In the specific implementation process, the updated weight coefficients are obtained through the output of the maximum likelihood estimation model: 1.8, 2 and -0.5. At the same time, according to the above-obtained defect probability of 0.89, the assembly gap is 14.34mm, and the vibration amplitude is 35mm / s.2 , the corresponding comprehensive risk index is 1.

[0093] Step SB1.2: Hierarchical control. This involves comparing the comprehensive risk index obtained in step SB1 with a preset risk threshold range (which can be set based on actual operating data and is not specifically explained in this embodiment. This embodiment uses 0.3-0.75 as an example for illustration). Based on the comparison results, a corresponding control strategy is determined, specifically:

[0094] If the obtained comprehensive risk index is less than the lower limit of the preset risk threshold range (i.e., 0.3), product inspection continues. If the obtained comprehensive risk index is within the preset risk threshold range (i.e., 0.3-0.75), the conveyor belt speed is reduced and the product is re-inspected. If the obtained comprehensive risk index is greater than the upper limit of the preset risk threshold range (i.e., 0.75), an alarm signal is immediately triggered, the machine is shut down, and the defect area is scanned using a 3D line laser, and the 3D point cloud data of the defect is uploaded.

[0095] Step SB2: Self-learning process. That is, according to the critical assembly tolerances of different product models, during the re-inspection process of the re-inspected products determined in step 1.2, corresponding compensation instructions are determined. The details are as follows:

[0096] Step SB2.1: Tolerance Library Management. This involves building a product tolerance library based on the critical tolerances and allowable compensation times for each product model. In this embodiment, the critical tolerances for each product model are revised quarterly based on new production data to ensure the accuracy and real-time nature of the data in the product tolerance library.

[0097] In this embodiment, two product models are provided, and their corresponding product tolerance libraries are shown in Table 1 below:

[0098] Table 1: Product tolerance library

[0099] Product Model Critical tolerance / mm Allowable compensation times A 0.2 3 B 0.15 5

[0100] During the specific implementation process, based on the model of the current re-inspected product and the corresponding assembly clearance, the critical tolerance and allowable number of compensations corresponding to the product model are obtained from the constructed product tolerance library. When the allowable number of compensations is still remaining, that is, it is not 0, the current product can be compensated. Otherwise, the process of the product model is reviewed and the production process of the product model is re-determined.

[0101] Furthermore, in the process of compensating the current product, the corresponding deviation ratio is determined based on the assembly clearance and critical tolerance corresponding to the current product, specifically:

[0102]

[0103] in: is the deviation ratio, is the assembly clearance to be measured, is the critical tolerance.

[0104] Step SB2.2: Compensation instruction generation. This involves comparing the deviation ratio determined in step SB2.1 with a preset deviation threshold range (which can be set based on actual operating data and is not specifically explained in this embodiment, but is illustrated using 0.9-1.1 as an example). Based on the comparison result, a corresponding compensation action is determined, specifically:

[0105] When the obtained deviation ratio is less than the lower limit of the preset deviation threshold range (i.e., 0.9), no compensation is required for the re-inspection of the current product. When the obtained deviation ratio is within the preset deviation threshold range (i.e., 0.9-1.1), compensation is performed for the re-inspection of the current product. When the obtained deviation ratio is greater than the upper limit of the preset deviation threshold range (i.e., 1.1), an alarm signal is immediately triggered, the production is shut down, and the process for the product model is reviewed and the production process for the product model is re-determined.

[0106] In this embodiment, during the force compensation process for the re-inspection behavior of the current product, the corresponding compensation force increment is determined according to the assembly clearance to be tested and the critical tolerance corresponding to the current product, specifically:

[0107]

[0108] in: To compensate for the force increment, is the proportionality coefficient, is the integration coefficient, is the assembly clearance to be measured, is the critical tolerance.

[0109] Furthermore, the proportionality coefficient in this embodiment can be set based on the material's elastic modulus. For example, the proportionality coefficient for the steel contact surface is 50 N / mm, and the integral coefficient in this embodiment is set to 10 N / (mm·s). In practice, the current product's corresponding assembly clearance and critical tolerance are 0.22 mm and 0.2 mm, respectively, resulting in a corresponding compensation force increment of 1 N.

[0110] In this embodiment, the multi-objective processing module is used to adjust the operation mode of the production system in real time based on the obtained quality-efficiency trade-off coefficient. The details are as follows:

[0111] Step SC1: Determine the trade-off coefficient. Specifically, based on the number of defective products determined by the multimodal perception module and the decision-making processing module and the current total number of units produced, the corresponding product defect rate is obtained. Furthermore, based on the determined product defect rate and the quality cost per unit, the corresponding quality loss is determined. Simultaneously, the average production time of a consecutive number of workpieces (10 in this embodiment) is obtained through the MES system and used as the standard takt time data. Furthermore, the multimodal perception module and decision-making processing module obtain the product's inspection time, which is then combined with the data transmission and analysis time to determine the corresponding inspection delay.

[0112] In this embodiment, the corresponding quality-efficiency trade-off coefficient is determined based on the determined standard cycle time, detection delay time, and quality loss, specifically:

[0113]

[0114] in: is the quality-efficiency trade-off coefficient, is the standard takt time, To detect the delay time, For quality loss, is the proportional weight.

[0115] In this embodiment, the proportional weight is set to 21.33, the quality loss is 0.015, the standard cycle time is 30, and the detection delay time is 8, so the corresponding quality-efficiency trade-off coefficient is 1.2.

[0116] Step SC2: Determine the working mode. That is, compare the quality-efficiency trade-off coefficient obtained in step SC1 with the preset trade-off threshold, and determine the corresponding operating mode based on the comparison result, specifically:

[0117] When the obtained quality-efficiency trade-off coefficient is greater than the preset trade-off threshold, the quality priority mode is activated. Conversely, when the obtained quality-efficiency trade-off coefficient is not greater than the preset trade-off threshold, the efficiency priority mode is activated.

[0118] That is, when the cost of quality loss is higher than the time efficiency loss, product quality is prioritized, such as in the case of motor end cap processing with a unit cost of 500 yuan. When the time efficiency loss is more critical, efficiency is prioritized, such as in the case of plastic housing injection molding with a unit cost of 20 yuan.

[0119] In this embodiment, a three-month historical database of the production line is obtained through SQL or a time series database. 120 samples are obtained from the historical database, and the corresponding actual quality-efficiency trade-off coefficients are determined based on the quality loss and efficiency loss corresponding to the 120 sample data. The 120 samples are then divided into three intervals based on the actual quality-efficiency trade-off coefficient corresponding to each sample. In other words, the actual quality-efficiency trade-off coefficients are divided into three trade-off coefficient intervals based on the range of the actual quality-efficiency trade-off coefficients, and the 120 samples are grouped based on the trade-off coefficient interval sizes.

[0120] Furthermore, based on the actual total cost corresponding to each sample, the average total cost corresponding to each trade-off coefficient interval is obtained, and based on the average total cost and the trade-off coefficient interval, a fitting formula between the trade-off coefficient and the cost is constructed, specifically:

[0121]

[0122] in: is the total production cost, is the quality-efficiency trade-off coefficient, is the coefficient of the quadratic term, is the coefficient of the first-order term, is a constant term.

[0123] That is, according to the obtained quadratic term coefficient and linear term coefficient, the corresponding preset trade-off threshold can be determined, specifically:

[0124]

[0125] in: To preset the trade-off threshold, is the coefficient of the quadratic term, is the coefficient of the first-order term.

[0126] refer to Figure 2 In the figure, the coefficient of the quadratic term is 116.2, the coefficient of the linear term is -325.1, and the constant term is 604.5, so the corresponding preset trade-off threshold is 1.4.

[0127] refer to Figure 3, the traditional mode has a unit output of 110 pieces / hour and a defect rate of 9.6%. The unit output of the AI ​​efficiency-first mode reaches 129 pieces / hour, which is about 17% higher than the traditional mode. The output of the AI ​​quality-first mode is 102 pieces / hour, which is about 7% lower than the traditional mode, but its defect rate is only 2.1%, which is 78% lower than the traditional mode. At the same time, the defect rate of the AI ​​efficiency-first mode is 8.3%, which is higher than the AI ​​quality-first mode, but 13% lower than the traditional mode. In other words, by choosing the AI ​​efficiency-first mode, you can pursue maximum output, while the AI ​​quality-first mode is suitable for quality-sensitive scenarios.

[0128] Example 2

[0129] This embodiment provides a deep learning-based AI-powered production line operation detection system. Its specific implementation method is the same as that of Example 1, differing in that an infrared thermal imager is used to obtain the corresponding temperature rise rate, and the comprehensive risk index is corrected based on the obtained temperature rise rate and the vibration amplitude obtained by the multimodal perception module. The following describes the invention with reference to the specific implementation methods of this embodiment.

[0130] In this embodiment, the magnitude of the comprehensive risk index is corrected according to the obtained temperature rise rate and vibration amplitude, as follows:

[0131] Step SB1.1.1: Temperature Rise Monitoring. Aim the FLIR A615 thermal imager at a 45° angle toward the motor winding end to scan the motor winding end and obtain an infrared image of the motor winding end. Based on the infrared image of the motor winding end, determine the corresponding temperature rise rate, specifically:

[0132]

[0133] in: is the temperature rise rate, is the temperature at time t+△t, is the temperature at time t, is the time interval.

[0134] During the specific implementation process, the temperature of the heat sink of the motor was 47.8°C 10 seconds after monitoring, and at the same time, the temperature of the heat sink of the motor reached 51.3°C 20 seconds after monitoring, so the corresponding temperature rise rate was 21°C / min.

[0135] Step SB1.1.2: Determine the correction coefficient. This involves comparing the temperature rise rate obtained in step SB1.1.1 with a preset temperature rise threshold range (which can be set based on actual operating data and is not specifically explained in this embodiment; this embodiment uses 10°C / min-20°C / min as an example). Based on the comparison result, determine the corresponding first-level correction coefficient, specifically:

[0136] When the obtained temperature rise rate is less than the lower limit of the preset temperature rise threshold range (i.e., 10°C / min), the first-level correction coefficient is set to 1. When the obtained temperature rise rate is within the preset temperature rise threshold range (i.e., 10°C / min-20°C / min), the first-level correction coefficient is set to 1.5. When the obtained temperature rise rate is greater than the upper limit of the preset temperature rise threshold range (i.e., 20°C / min), the first-level correction coefficient is set to 2.

[0137] Furthermore, according to the vibration amplitude obtained in the multimodal sensing module and the preset vibration threshold range (which can be specifically set according to the actual operation data, and is not specifically explained in this embodiment, and is only 15mm / s in this embodiment), 2 -30mm / s 2 As an example, the corresponding secondary correction coefficient is determined based on the comparison result, which is as follows:

[0138] When the vibration amplitude is less than the lower limit of the preset vibration threshold range (i.e. 15 mm / s 2 ), the secondary correction coefficient is set to 1. When the vibration amplitude obtained is within the preset vibration threshold range (i.e. 15mm / s 2 -30mm / s 2 ), the secondary correction coefficient is set to 1.5. When the vibration amplitude obtained is greater than the upper limit threshold of the preset vibration threshold range (i.e. 30mm / s 2 ), the secondary correction factor is set to 2.

[0139] In this embodiment, the determined primary correction coefficient and secondary correction coefficient are combined to construct a correction coefficient table, as shown in Table 2 below:

[0140] Table 2: Correction coefficient table

[0141]

[0142] Step SB1.1.3: Determine the correction index. That is, based on the real-time acquired vibration amplitude and temperature rise rate, determine the corresponding correction coefficient from the correction coefficient table constructed in step SB1.1.2, and correct the obtained comprehensive risk index based on the determined correction coefficient to obtain the corrected comprehensive risk index, which is specifically:

[0143]

[0144] in: is the revised comprehensive risk index, is the comprehensive risk index, is the correction factor.

[0145] refer to Figure 4 The green area in the figure corresponds to a correction factor of 1. At this time, when the temperature rise rate is low or the temperature rise is moderate but the vibration is slight, no additional correction to the comprehensive risk index is required, and the equipment is in a safe operating condition. The yellow area in the figure corresponds to a correction factor of 1.5. At this time, when the temperature rise rate is moderate and the vibration is moderate, a correction factor of 1.5 is required, indicating that the equipment may face a mild risk and requires increased monitoring. The red area in the figure corresponds to a correction factor of 2. In other words, when the temperature rise rate is high or the temperature rise is moderate and the vibration is severe, the correction factor increases to 2 times, which is an extreme operating condition, significantly reducing equipment reliability and requiring immediate intervention.

[0146] Although embodiments of the present invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is limited by the accompanying embodiments and their equivalents.

Claims

1. A production line operation AI detection system based on deep learning, characterized by: Includes: Multimodal perception module: monitors products through industrial cameras and laser sensors to obtain multimodal data of products; Decision-making processing module: obtains a comprehensive risk index based on the multimodal data, sets a hierarchical control strategy based on the comprehensive risk index, and determines compensation instructions, including: SB1: Dynamic decision-making: Defect probability, assembly clearance to be measured, and vibration amplitude are integrated to determine a comprehensive risk index, and a hierarchical control strategy is set based on the comprehensive risk index; SB2: Self-learning processing: Determines compensation instructions for re-inspected products based on critical assembly tolerances for different product models; Multi-objective processing module: determines the quality-efficiency trade-off coefficient through the multi-modal perception module and the decision processing module, and adjusts the operation mode of the production system according to the quality-efficiency trade-off coefficient.

2. The deep learning-based production line operation AI detection system according to claim 1, characterized in that: Obtain multimodal data of the product, including: SA1: Visual Inspection: The preprocessed product image is used as the input of the improved ResNet-50 model. The output is the flattened weighted feature map. Based on the flattened weighted feature map, the linearly transformed raw score is determined. The defect probability is obtained from the linearly transformed raw score. SA2: Laser ranging: The assembly gap is measured by a laser sensor to obtain the assembly gap to be measured; SA3: Vibration detection: The Lucas-Kanade optical flow algorithm and the feature point image are used to determine the displacement of the feature points, and the vibration amplitude is determined based on the displacement of the feature points.

3. The production line operation AI detection system based on deep learning according to claim 1, characterized in that: Set up hierarchical control strategies, including: SB1.1: Build a risk quantification model: Based on the defect probability, the assembly clearance to be tested, and the vibration amplitude, determine the comprehensive risk index, specifically: in: is the comprehensive risk index, is the base of natural logarithm, is the defect probability after Sigmoid function conversion, is the assembly clearance to be measured, is the vibration amplitude, 、 、 is the weight coefficient; SB1.2: Hierarchical Control: Compare the comprehensive risk index with the preset risk threshold range and determine the control strategy based on the comparison results, specifically: When the comprehensive risk index is less than the lower limit threshold of the preset risk threshold range, the product will continue to be inspected; when the comprehensive risk index is within the preset risk threshold range, the transmission rate of the conveyor belt will be reduced and the product will be re-inspected; when the comprehensive risk index is greater than the upper limit threshold of the preset risk threshold range, an alarm signal will be triggered, and the machine will be shut down. At the same time, the defective part will be scanned by 3D line laser to obtain the three-dimensional point cloud data of the defect.

4. The deep learning-based production line operation AI detection system according to claim 3, characterized in that: According to the defect probability, assembly gap and vibration amplitude of multiple groups of samples, the normalized defect probability, normalized assembly gap and normalized vibration amplitude are obtained. At the same time, according to the set initial weight coefficient, normalized defect probability, normalized assembly gap and normalized vibration amplitude, the initial comprehensive risk index of multiple samples is determined, and the initial comprehensive risk index and the detection results are used as the input of the maximum likelihood estimation model, and the updated weight coefficient is obtained as the output.

5. The deep learning-based production line operation AI detection system according to claim 3, characterized in that: The comprehensive risk index is modified according to the temperature rise rate and vibration amplitude, including: SB1.1.1: Temperature rise monitoring: Scan product components with a thermal imager to obtain infrared images of the product components, and determine the temperature rise rate based on the infrared images; SB1.1.2: Determine Correction Factors: Determine a primary correction factor based on the temperature rise rate and the preset temperature rise threshold range. Determine a secondary correction factor based on the vibration amplitude and the preset vibration threshold range. Combine the primary and secondary correction factors to construct a correction factor table. SB1.1.3: Determine the correction index: Based on the vibration amplitude and temperature rise rate, determine the correction coefficient through the correction coefficient table, and correct the comprehensive risk index based on the correction coefficient to obtain the corrected comprehensive risk index.

6. The production line operation AI detection system based on deep learning according to claim 5, characterized in that: The temperature rise rate is compared with the preset temperature rise threshold range, and a first-level correction coefficient is determined based on the comparison result, specifically: When the temperature rise rate is less than the lower limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 1; when the temperature rise rate is within the preset temperature rise threshold range, the first-level correction coefficient is set to 1.5; when the temperature rise rate is greater than the upper limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 2; The vibration amplitude is compared with a preset vibration threshold range, and a secondary correction coefficient is determined based on the comparison result, specifically: When the vibration amplitude is less than the lower limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 1; when the vibration amplitude is within the preset vibration threshold range, the secondary correction coefficient is set to 1.5; when the vibration amplitude is greater than the upper limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 2.

7. The production line operation AI detection system based on deep learning according to claim 1, characterized in that: Determine compensation instructions for re-inspected products, including: SB2.1: Tolerance Library Management: Build a product tolerance library based on the critical tolerance and allowable compensation times for each product model. Determine the compensation status and deviation ratio of the current product based on the product tolerance library. SB2.2: Compensation Instruction Generation: Compare the deviation ratio with a preset deviation threshold range and determine a compensation action based on the comparison result, specifically: When the deviation ratio is less than the lower limit threshold of the preset deviation threshold range, no compensation is made for the re-inspection behavior of the current product; when the deviation ratio is within the preset deviation threshold range, force compensation is made for the re-inspection behavior of the current product; when the deviation ratio is greater than the upper limit threshold of the preset deviation threshold range, an alarm signal is triggered, and the machine is shut down. At the same time, the process of the current product is reviewed and the production process of the current product is re-determined.

8. The deep learning-based production line operation AI detection system according to claim 7, characterized in that: When performing force compensation on the re-inspection behavior of the current product, the compensation force increment is determined based on the assembly clearance to be tested and the critical tolerance of the current product. Specifically, in: To compensate for the force increment, is the proportionality coefficient, is the integration coefficient, is the assembly clearance to be measured, is the critical tolerance.

9. The deep learning-based production line operation AI detection system according to claim 1, characterized in that: Adjust the operating mode of the production system, including: SC1: Determine the trade-off coefficient: Determine the product's quality loss and detection delay time through the multimodal perception module and the decision processing module, determine the product's standard cycle time through the MES system, and determine the quality-efficiency trade-off coefficient based on the product's quality loss, detection delay time, and standard cycle time; SC2: Determine the working mode: Compare the quality-efficiency trade-off coefficient with a preset trade-off threshold, and determine the operating mode of the production system based on the comparison result, specifically: When the quality-efficiency trade-off coefficient is greater than a preset trade-off threshold, the quality priority mode is activated; otherwise, the efficiency priority mode is activated.

10. The deep learning-based production line operation AI detection system according to claim 9, characterized in that: Based on the quality-efficiency trade-off coefficients of the plurality of sample data, a trade-off coefficient range of the plurality of sample data is determined, and the trade-off coefficient range is divided into a plurality of intervals. At the same time, the plurality of sample data is divided according to the intervals. Based on the average total cost of the plurality of sample data in each interval and the trade-off coefficient interval, a trade-off coefficient-cost fitting formula is constructed. The preset trade-off threshold is determined by the quadratic term coefficient and the linear term coefficient in the trade-off coefficient-cost fitting formula, specifically: in: To preset the trade-off threshold, is the coefficient of the quadratic term, is the coefficient of the first-order term.

Images