A deep learning-based production line operation AI detection system

Abstract

The application discloses a kind of production line operation AI detection systems based on deep learning, it is related to intelligent manufacturing technical field.The application includes:multimodal perception module: through industrial camera and laser sensor, product is monitored, and multimodal data is acquired;Decision processing module: according to the multimodal data, comprehensive risk index is acquired, while according to the comprehensive risk index, hierarchical control strategy is set, and compensation instruction is determined;Multi-object processing module: through the multimodal perception module and decision processing module, quality-efficiency trade-off coefficient is determined, and the operation mode of production system is adjusted.The application obtains comprehensive risk index by fusing defect probability, vibration and temperature rise data, and divides the state of production line into three different responses through comprehensive risk index, so that the production continuity and quality control can be balanced, and unnecessary downtime loss is reduced.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of intelligent manufacturing, in particular to a production line operation AI detection system based on deep learning. BACKGROUND

[0002] In modern manufacturing, stable, efficient and safe operation of production lines is the core element to ensure product quality, improve production efficiency and reduce operating costs. With the continuous improvement of industrial automation and intelligence, the demand for real-time and accurate monitoring and abnormal detection of production line operation status is increasingly urgent. Traditional production line monitoring mainly relies on the following methods:

[0003] 1. Threshold alarm system based on rules: This type of system sets threshold values for physical parameters of sensors (such as temperature, pressure, vibration, current, etc.) in advance, and triggers an alarm when the monitored value exceeds the set range. Its advantages are simple implementation and low cost. However, it has obvious limitations such as rigidity, poor adaptability, high false alarm and missed alarm rates, and dependence on expert experience.

[0004] 2. Methods based on traditional signal processing and statistical analysis: such as Fourier transform, wavelet analysis, principal component analysis (PCA), support vector machine (SVM), etc. These methods can handle more complex signal features, extract time-frequency domain information or perform dimensionality reduction analysis. However, they have limited generalization ability and are difficult to handle high-dimensional and unstructured data.

[0005] 3. Methods based on expert systems or knowledge graphs: use domain knowledge base for reasoning and judgment. However, the knowledge acquisition process is tedious and difficult, and the cost is high. At the same time, when faced with data noise, incomplete or ambiguous information, the reasoning ability is limited.

[0006] Chinese invention patent with publication number CN110806743A discloses a device fault detection and early warning system based on artificial intelligence, which includes a collection module, a processing module and an early warning module. The processing module includes a data training unit, a prediction unit and a judgment unit: the collection module includes a plurality of sensors installed on different parts of the industrial control device for collecting real-time state data of the industrial control device; the data training unit is used to train the collected device state historical data to establish a device anomaly prediction model; the prediction unit is used to predict the device state after inputting the collected new device state data into the device anomaly prediction model; the judgment unit is used to notify the early warning module to issue a warning when the predicted device state is abnormal.

[0007] The above technical solution can give an early warning of equipment failure according to the working state of the equipment, avoid the occurrence of sudden failure, and reduce the economic loss caused by shutdown for maintenance. However, in the process of intelligent detection of automobile welding, when the AI quality detection system gives a 92% qualified confidence based on the analysis of the image features of the welding points, and the artificial quality inspector determines that there is a virtual welding risk according to visual inspection or process standards, the production line must be interrupted for production to wait for manual review, thereby reducing the production efficiency and affecting the JIT production rhythm. SUMMARY

[0008] The purpose of the present application is to provide a production line operation AI detection system based on deep learning to solve the problems raised in the background art.

[0009] To achieve the above purpose, the present application provides the following technical solution: a production line operation AI detection system based on deep learning, comprising:

[0010] A multi-modal perception module: monitoring products through an industrial camera and a laser sensor to obtain multi-modal data of the products;

[0011] A decision processing module: obtaining a comprehensive risk index based on the multi-modal data, setting a hierarchical control strategy based on the comprehensive risk index, and determining a compensation instruction, comprising:

[0012] SB1: dynamic decision making: fusing defect probability, measured assembly gap, and vibration amplitude to determine a comprehensive risk index, and setting a hierarchical control strategy based on the comprehensive risk index;

[0013] SB2: self-learning processing: determining a compensation instruction for rechecking products based on critical assembly tolerances of different product models;

[0014] A multi-objective processing module: determining a quality-efficiency trade-off coefficient through the multi-modal perception module and the decision processing module, and adjusting the operation mode of the production system based on the quality-efficiency trade-off coefficient.

[0015] Further, the multi-modal data of the products comprises:

[0016] SA1: visual inspection: inputting the preprocessed product image into an improved ResNet-50 model as input, outputting weighted feature map flattened data, and determining linearly transformed original scores based on the weighted feature map flattened data, and obtaining defect probability through the linearly transformed original scores;

[0017] SA2: laser ranging: measuring the assembly gap through a laser sensor to obtain the measured assembly gap;

[0018] SA3: Vibration detection: determining the displacement of the feature points by Lucas-Kanade optical flow algorithm and feature point images, and determining the vibration amplitude according to the displacement of the feature points.

[0019] Further, a hierarchical control strategy is set, including:

[0020] SB1.1: Constructing a risk quantification model: determining a comprehensive risk index according to the defect probability, the assembly gap to be measured, and the vibration amplitude, specifically:

[0021]

[0022] wherein: is the comprehensive risk index, is the natural logarithm base, is the defect probability converted by the Sigmoid function, is the assembly gap to be measured, is the vibration amplitude, , , is the weight coefficient;

[0023] SB1.2: Hierarchical control: comparing the comprehensive risk index with a preset risk threshold range, and determining a control strategy according to the comparison result, specifically:

[0024] When the comprehensive risk index is less than the lower threshold of the preset risk threshold range, the product continues to be detected; when the comprehensive risk index is within the preset risk threshold range, the transmission rate of the conveyor belt is reduced, and the product is re-inspected; when the comprehensive risk index is greater than the upper threshold of the preset risk threshold range, an alarm signal is triggered, and a shutdown process is performed, and the defect site is scanned by a 3D line laser to obtain three-dimensional point cloud data of the defect.

[0025] Further, according to the defect probability, the assembly gap, and the vibration amplitude of multiple groups of samples, normalized defect probability, normalized assembly gap, and normalized vibration amplitude are obtained, and according to the initial weight coefficient, the normalized defect probability, the normalized assembly gap, and the normalized vibration amplitude, the initial comprehensive risk index of the multiple samples is determined, and the initial comprehensive risk index and the detection result are taken as the input of a maximum likelihood estimation model to output an updated weight coefficient.

[0026] Further, according to the temperature rise rate and the vibration amplitude, the comprehensive risk index is corrected, including:

[0027] SB1.1.1: Temperature rise monitoring: Scan product components with a thermal imager to obtain infrared images of the product components, and determine the temperature rise rate based on the infrared images;

[0028] SB1.1.2: Determine Correction Factors: Determine a primary correction factor based on the temperature rise rate and the preset temperature rise threshold range. Determine a secondary correction factor based on the vibration amplitude and the preset vibration threshold range. Combine the primary and secondary correction factors to construct a correction factor table.

[0029] SB1.1.3: Determine the correction index: Based on the vibration amplitude and temperature rise rate, determine the correction coefficient through the correction coefficient table, and correct the comprehensive risk index based on the correction coefficient to obtain the corrected comprehensive risk index.

[0030] Furthermore, the temperature rise rate is compared with a preset temperature rise threshold range, and a first-level correction coefficient is determined based on the comparison result, specifically:

[0031] When the temperature rise rate is less than the lower limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 1; when the temperature rise rate is within the preset temperature rise threshold range, the first-level correction coefficient is set to 1.5; when the temperature rise rate is greater than the upper limit threshold of the preset temperature rise threshold range, the first-level correction coefficient is set to 2;

[0032] The vibration amplitude is compared with a preset vibration threshold range, and a secondary correction coefficient is determined based on the comparison result, specifically:

[0033] When the vibration amplitude is less than the lower limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 1; when the vibration amplitude is within the preset vibration threshold range, the secondary correction coefficient is set to 1.5; when the vibration amplitude is greater than the upper limit threshold of the preset vibration threshold range, the secondary correction coefficient is set to 2.

[0034] Furthermore, compensation instructions for re-inspected products are determined, including:

[0035] SB2.1: Tolerance Library Management: Build a product tolerance library based on the critical tolerance and allowable compensation times for each product model. Determine the compensation status and deviation ratio of the current product based on the product tolerance library.

[0036] SB2.2: Compensation Instruction Generation: Compare the deviation ratio with a preset deviation threshold range and determine a compensation action based on the comparison result, specifically:

[0037] When the deviation ratio is less than a lower threshold of a preset deviation threshold range, no compensation is performed on the re-inspection behavior of the current product; when the deviation ratio is within the preset deviation threshold range, compensation is performed on the re-inspection behavior of the current product; and when the deviation ratio is greater than an upper threshold of the preset deviation threshold range, an alarm signal is triggered, and a shutdown process is performed, and the process of the current product is reviewed to re-determine the production process of the current product.

[0038] Further, when the re-inspection behavior of the current product is compensated, the compensation force increment size is determined according to the to-be-measured assembly gap and the critical tolerance of the current product, specifically:

[0039]

[0040] Wherein: is the compensation force increment, is a proportional coefficient, is an integral coefficient, is the to-be-measured assembly gap, is the critical tolerance.

[0041] Further, the running mode of the production system is adjusted, including:

[0042] SC1: determining a trade-off coefficient: determining the quality loss and the detection delay time of the product through the multi-modal perception module and the decision processing module, determining the standard tact time of the product through the MES system, and determining the quality-efficiency trade-off coefficient according to the quality loss, the detection delay time and the standard tact time of the product;

[0043] SC2: determining a working mode: comparing the quality-efficiency trade-off coefficient with a preset trade-off threshold, and determining the running mode of the production system according to the comparison result, specifically:

[0044] When the quality-efficiency trade-off coefficient is greater than the preset trade-off threshold, the quality priority mode is started; otherwise, the efficiency priority mode is started.

[0045] Further, according to the quality-efficiency trade-off coefficients of a plurality of sample data, a trade-off coefficient range of the plurality of sample data is determined, and the trade-off coefficient range is divided into a plurality of intervals, and the plurality of sample data is divided according to the intervals, and a trade-off coefficient-cost fitting formula is constructed according to the average total cost and the trade-off coefficient interval of the plurality of sample data in each interval, and the preset trade-off threshold is determined through the quadratic term coefficient and the linear term coefficient in the trade-off coefficient-cost fitting formula, specifically:

[0046]

[0047] Wherein: is a preset trade-off threshold, is a quadratic term coefficient, is a linear term coefficient.

[0048] Compared with the prior art, the beneficial effects of the present application are:

[0049] Firstly, the present application obtains a comprehensive risk index by fusing defect probability, vibration and temperature rise data, and divides the production line state into three different responses through the comprehensive risk index, so as to balance production continuity and quality control, and reduce unnecessary downtime loss.

[0050] Secondly, the present application dynamically matches the tolerance library with the product model, and triggers PID compensation according to the deviation ratio, to determine the corresponding compensation force increment, so as to correct the assembly deviation and improve production precision.

[0051] Thirdly, the present application updates the weight coefficient in the comprehensive risk index through maximum likelihood estimation, so that the model can continuously adapt to the changes of the production line, and improve the adaptability of the model.

[0052] Fourthly, the present application sets a preset trade-off threshold through the extreme point of the cost model, and intelligently switches the quality / efficiency priority mode, so as to maximize production efficiency while ensuring key quality indicators. BRIEF DESCRIPTION OF DRAWINGS

[0053] Figure 1 is a system block diagram of the production line operation AI detection system in the present application;

[0054] Figure 2 is a fitting graph between the trade-off coefficient and the cost in the present application;

[0055] Figure 3 is an effect comparison graph of the AI detection system in the present application;

[0056] Figure 4 is a thermal map of the temperature rise-vibration combined correction coefficient in the present application. DETAILED DESCRIPTION

[0057] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0058] In the process of intelligent detection of automobile welding, when the AI quality detection system gives a 92% qualified confidence based on the analysis of the image features of the welding points, but the artificial quality inspector determines that there are virtual welding risks according to visual inspection or process standards, the production line must be interrupted for production to wait for manual review, thereby reducing production efficiency and affecting JIT production rhythm. The technical scheme of the present application collects product defect probability, assembly gap and vibration amplitude data in real time through a multi-modal perception module, simultaneously fuses the collected multi-source information, generates a comprehensive risk index and implements the corresponding control strategy. At the same time, the production mode is dynamically adjusted through the quality-efficiency trade-off coefficient, thereby realizing the collaborative optimization of quality and efficiency in intelligent manufacturing.

[0059] Embodiment 1

[0060] Reference Figures 1-3 The embodiment provides an AI detection system for production line operation based on deep learning, which comprises a multi-modal perception module, a decision processing module and a multi-target processing module. In the embodiment, the multi-modal perception module is used to identify defects such as scratches and material shortages on the surface of the product, obtain the corresponding normalized defect probability, and monitor the assembly gap in real time through a laser displacement sensor, and obtain the vibration amplitude of the moving parts through a high-speed camera. Specifically as follows:

[0061] Step SA1: visual detection. That is, an industrial camera (such as Basler ace acA2000-50gm) is used to take pictures of the products transported on the conveyor belt, and at least 3 images of different angles are taken for each product to cover different perspectives of the front, middle and back of the product, thereby covering the entire surface of the product. Further, the pixel values in each obtained image are normalized to map the pixel values of the image to the range [0, 1]. Then, according to the normalized pixel values, histogram equalization is performed to obtain the preprocessed product image.

[0062] In the embodiment, a channel attention module is inserted after the third residual block of the ResNet-50 model to construct an improved ResNet-50 model, thereby enhancing the sensitivity to small defects (such as 0.1mm wide scratches). Specifically, the preprocessed product image is taken as the input of the improved ResNet-50 model, and the weighted feature map flattened data is obtained as the output.

[0063] Further, according to the weighted feature map flattened data obtained as the output, the linearly transformed original score is determined, specifically as follows:

[0064]

[0065] Wherein: the original score after linear transformation, the weight parameter of the full connection layer, the weighted feature map flattened data, the bias term of the full connection layer, the inverse of the weight parameter.

[0066] Further, according to the obtained original score after linear transformation, the corresponding defect probability is determined, specifically:

[0067]

[0068] wherein: the defect probability after Sigmoid function conversion, the natural logarithm base, the original score after linear transformation.

[0069] That is, when the original score after linear transformation is 2.1 through the weighted feature map flattened data, the corresponding defect probability is 0.89.

[0070] Step SA2: laser ranging. That is, the laser sensor of Keyence LJ-V7080 is vertically installed on the upper end of the measured assembly surface, and the distance from the reference surface is 50 mm, and the included angle between the incident laser and the reflected light path is set to 30°. Specifically, the laser sensor in the embodiment can emit 650 nm red laser, and the spot diameter is 0.8 mm.

[0071] In the embodiment, the laser sensor emits a 650 nm laser beam to measure the assembly gap and obtain the corresponding assembly gap, specifically:

[0072]

[0073] wherein: the measured assembly gap, the fixed distance from the sensor to the reference surface, the included angle between the reflected light and the incident light.

[0074] Specifically, when the reflection angle measurement value is 32°, and the fixed distance from the sensor to the reference surface in the embodiment is 50 mm, the corresponding assembly gap is 14.34 mm.

[0075] Step SA3: vibration detection. That is, a Photron FASTCAM Mini AX200 camera is installed on the three-axis adjustable gimbal, and the lens axis is arranged perpendicular to the surface of the motor housing. At the same time, according to the pre-marked area of the motor housing (such as the bolt head, the edge of the heat dissipation fin), the characteristic points of the pre-marked area are photographed by the camera to obtain the corresponding characteristic point image.

[0076] In this embodiment, according to the obtained characteristic point image, the corresponding characteristic point displacement is determined by the Lucas-Kanade optical flow algorithm, specifically:

[0077]

[0078] Wherein: is the instantaneous displacement of the i-th characteristic point, is the horizontal gradient of the image, is the vertical gradient of the image, is the gray scale change between adjacent frames, is the x-direction optical flow velocity, is the y-direction optical flow velocity.

[0079] Further, according to the size of the obtained plurality of characteristic point displacements, the corresponding instantaneous vibration amplitude is determined, specifically:

[0080]

[0081] Wherein: is the vibration amplitude, is the number of effective characteristic points, is the index of the effective characteristic point, is the instantaneous displacement of the i-th characteristic point.

[0082] In this embodiment, the decision processing module is used to fuse the defect probability, the to-be-measured assembly gap and the vibration amplitude obtained in the multi-modal perception module to determine the corresponding comprehensive risk index, and set a hierarchical control strategy according to the determined comprehensive risk index. And according to the hierarchical control strategy, the corresponding compensation instruction is determined. Specifically as follows:

[0083] Step SB1: dynamic decision. That is, according to the defect probability, the to-be-measured assembly gap and the vibration amplitude, the corresponding comprehensive risk index is determined, and the hierarchical control strategy is set according to the determined comprehensive risk index. Specifically as follows:

[0084] Step SB1.1: Construct risk quantification model. That is, according to the defect probability, the to-be-measured assembly gap and the vibration amplitude obtained in the multi-modal perception module, the corresponding comprehensive risk index is determined, specifically:

[0085]

[0086] wherein: is a comprehensive risk index, is a natural logarithm base, is a defect probability converted by a Sigmoid function, is an assembly gap to be tested, is a vibration amplitude, , , is a weight coefficient.

[0087] In the present embodiment, a 3-month historical database on the production line is obtained through SQL or a time series database, 5000 groups of samples are obtained through the historical database, each group of samples including a defect probability, an assembly gap, a vibration amplitude and a detection result. Further, according to the detection result, the samples are divided into normal samples and fault samples, and the sample ratio between the normal samples and the fault samples is adjusted to 3:1 to avoid bias in the process of determining the weight coefficient.

[0088] Further, all the defect probabilities, assembly gaps and vibration amplitudes in the 5000 groups of samples are normalized to obtain normalized defect probabilities, normalized assembly gaps and normalized vibration amplitudes. Specifically, in the present embodiment, the three weight coefficients in the initially constructed risk quantification model are set to 1, 1 and 0 respectively, and the initial comprehensive risk index corresponding to the 5000 groups of samples is determined according to the normalized defect probabilities, normalized assembly gaps, normalized vibration amplitudes and initial weight coefficients. At the same time, the initial comprehensive risk index corresponding to each sample and the detection result are taken as the input of the maximum likelihood estimation model, and the corresponding updated weight coefficient is output through corresponding optimization iteration.

[0089] Specifically, the acquisition formula of the maximum likelihood estimation model in the present embodiment is specifically:

[0090]

[0091] wherein: is a log-likelihood function, is the total number of training samples, is the index of the training sample, is the true label of the jth sample, is the comprehensive risk index of the jth sample.

[0092] In the process of specific implementation, the updated weight coefficient obtained through the output of the maximum likelihood estimation model is: 1.8, 2 and -0.5, and the defect probability 0.89, the assembly gap 14.34 mm and the vibration amplitude 35 mm / s obtained above2 If the corresponding comprehensive risk index is 1, then the corresponding comprehensive risk index is 1.

[0093] Step SB1.2: hierarchical control. That is, the comprehensive risk index obtained in step SB1 is compared with the preset risk threshold range (which can be set according to actual operation data, which will not be specifically described in this embodiment, and only 0.3-0.75 is taken as an example in this embodiment), and according to the comparison result, the corresponding control strategy is determined, which is:

[0094] When the obtained comprehensive risk index is less than the lower threshold of the preset risk threshold range (i.e. 0.3), the product is continuously detected. When the obtained comprehensive risk index is within the preset risk threshold range (i.e. 0.3-0.75), the transmission rate of the conveyor belt is reduced, and the product is re-inspected. When the obtained comprehensive risk index is greater than the upper threshold of the preset risk threshold range (i.e. 0.75), an alarm signal is triggered immediately, and a shutdown process is performed, and the defect site is scanned by a 3D line laser, and the defect three-dimensional point cloud data is uploaded.

[0095] Step SB2: self-learning processing. That is, according to the critical assembly tolerance of different product models, the corresponding compensation instructions are determined in the process of re-inspecting the products determined in step 1.2. The specific process is as follows:

[0096] Step SB2.1: tolerance library management. That is, according to the critical tolerance and the allowed compensation times of each product model, a product tolerance library is constructed. At the same time, in this embodiment, the critical tolerance of each product model is corrected every quarter according to the new production data of the product, so as to ensure the accuracy and real-time of the data in the product tolerance library.

[0097] In this embodiment, there are two product models, and the corresponding product tolerance library is shown in Table 1:

[0098] Table 1: Product tolerance library

[0099] Product model Critical tolerance / mm Allowed number of compensations A 0.2 3 B 0.15 5

[0100] In the process of specific implementation, according to the model and the corresponding assembly gap of the current re-inspected product, the critical tolerance and the allowed compensation times of the product model are obtained from the constructed product tolerance library. When the allowed compensation times are remaining, that is, not 0, the current product can be compensated, otherwise the process of the product model is reviewed, and the production process of the product model is re-determined.

[0101] Further, in the process of compensating the current product, according to the corresponding assembly gap and critical tolerance of the current product, the corresponding deviation ratio is determined, which is:

[0102]

[0103] wherein: is the deviation ratio, is the measured assembly gap, is the critical tolerance.

[0104] Step SB2.2: compensation instruction generation. That is, the size of the deviation ratio determined in step SB2.1 is compared with the preset deviation threshold range (which can be set according to actual operation data, and will not be specifically described in the present embodiment, and only 0.9-1.1 is taken as an example for illustration in the present embodiment). According to the comparison result, the corresponding compensation behavior is determined, which is specifically:

[0105] When the obtained deviation ratio is less than the lower threshold (i.e. 0.9) of the preset deviation threshold range, no compensation is needed for the re-inspection behavior of the current product. When the obtained deviation ratio is within the preset deviation threshold range (i.e. 0.9-1.1), the re-inspection behavior of the current product is compensated. When the obtained deviation ratio is greater than the upper threshold (i.e. 1.1) of the preset deviation threshold range, an alarm signal is triggered immediately, and the machine is stopped, and the process of the product is reviewed, and the production process of the product is re-determined.

[0106] In the present embodiment, in the process of compensating the re-inspection behavior of the current product, the corresponding compensation force increment is determined according to the measured assembly gap and the critical tolerance corresponding to the current product, which is specifically:

[0107]

[0108] wherein: is the compensation force increment, is the proportional coefficient, is the integral coefficient, is the measured assembly gap, is the critical tolerance.

[0109] Further, the proportional coefficient in the present embodiment can be set according to the elastic modulus of the material, for example, the proportional coefficient of the steel contact surface is 50N / mm, and the integral coefficient in the present embodiment is set to 10N / (mm·s). In the process of specific implementation, the assembly gap and the critical tolerance corresponding to the current product are 0.22mm and 0.2mm respectively, and the corresponding compensation force increment is 1N.

[0110] In the present embodiment, the multi-target processing module is used to adjust the running mode of the production system in real time according to the obtained quality-efficiency trade-off coefficient. Specifically as follows:

[0111] Step SC1: determining the trade-off coefficient. That is, according to the number of defective products determined in the multi-modal perception module and the decision processing module and the current total production quantity, the corresponding product defect rate is obtained, and according to the determined product defect rate and the quality cost of a single product, the corresponding quality loss size is determined. At the same time, through the MES system, the average production time of a plurality of (10 in this embodiment) product workpieces is obtained, and the average production time is taken as the standard tact time data. Further, through the multi-modal perception module and the decision processing module, the detection time consumption time of the product is obtained, and the detection time consumption time of the product is combined with the data transmission time and the analysis time consumption time to determine the corresponding detection delay time.

[0112] In this embodiment, according to the determined standard tact time, detection delay time and quality loss size, the corresponding quality-efficiency trade-off coefficient is determined, specifically:

[0113]

[0114] Among them: is the quality-efficiency trade-off coefficient, is the standard tact time, is the detection delay time, is the quality loss, is the proportional weight.

[0115] In this embodiment, the size of the proportional weight is set to 21.33, and the quality loss is 0.015, the standard tact time is 30, and the detection delay time is 8. Then the corresponding quality-efficiency trade-off coefficient is 1.2.

[0116] Step SC2: determining the working mode. That is, the quality-efficiency trade-off coefficient obtained in step SC1 is compared with the preset trade-off threshold, and according to the comparison result, the corresponding running mode is determined, specifically:

[0117] When the obtained quality-efficiency trade-off coefficient is greater than the preset trade-off threshold, the quality priority mode is started. On the contrary, when the obtained quality-efficiency trade-off coefficient is not greater than the preset trade-off threshold, the efficiency priority mode is started.

[0118] That is, when the quality loss cost is higher than the time efficiency loss, the product quality is ensured first, for example, the motor end cover processing with a single piece cost of 500 yuan. When the time efficiency loss is more critical, the efficiency is first, for example, the plastic shell injection with a single piece cost of 20 yuan.

[0119] In the embodiment, the 3-month historical database on the production line is obtained through SQL or time series database, 120 samples are obtained through the historical database, and the actual quality-efficiency trade-off coefficient corresponding to the 120 sample data is determined according to the quality loss and efficiency loss corresponding to the 120 sample data. Meanwhile, the 120 samples are divided into 3 intervals according to the actual quality-efficiency trade-off coefficient corresponding to each sample. That is, the actual quality-efficiency trade-off coefficient is divided into 3 trade-off coefficient intervals according to the range of the actual quality-efficiency trade-off coefficient, and the 120 samples are grouped according to the trade-off coefficient interval size.

[0120] Further, the average total cost size corresponding to each trade-off coefficient interval is obtained according to the actual total cost corresponding to each sample, and a fitting formula between the trade-off coefficient and the cost is constructed according to the average total cost size and the trade-off coefficient interval, specifically:

[0121]

[0122] Wherein: is the total production cost, is the quality-efficiency trade-off coefficient, is the quadratic coefficient, is the linear coefficient, is the constant term.

[0123] That is, the corresponding preset trade-off threshold can be determined according to the obtained quadratic coefficient and linear coefficient, specifically:

[0124]

[0125] Wherein: is the preset trade-off threshold, is the quadratic coefficient, is the linear coefficient.

[0126] Referring to Figure 2 , the quadratic coefficient in the figure is 116.2, the linear coefficient is -325.1, and the constant term is 604.5, so the corresponding preset trade-off threshold is 1.4.

[0127] Referring to Figure 3, the traditional mode unit output is 110 pieces / hour, and the defect rate is 9.6%. The AI efficiency priority mode unit output reaches 129 pieces / hour, which is about 17% higher than the traditional mode. The AI quality priority mode output is 102 pieces / hour, which is about 7% lower than the traditional mode, but its defect rate is only 2.1%, which is 78% lower than the traditional mode. At the same time, the defect rate of the AI efficiency priority mode is 8.3%, which is higher than the AI quality priority mode, but it is 13% lower than the traditional mode. That is, by selecting the AI efficiency priority mode, the maximum output can be pursued, and the AI quality priority mode is suitable for quality sensitive scenarios.

[0128] Embodiment 2

[0129] The embodiment provides a production line operation AI detection system based on deep learning, and the specific implementation method is the same as that of embodiment 1, and the difference is that the corresponding temperature rise rate is obtained through the infrared thermal imager, and the size of the comprehensive risk index is corrected according to the obtained temperature rise rate and the vibration amplitude obtained in the multi-modal perception module. The present application is illustrated by combining the specific implementation of the embodiment.

[0130] In this embodiment, the size of the comprehensive risk index is corrected according to the obtained temperature rise rate and vibration amplitude, as follows:

[0131] Step SB1.1.1: temperature rise monitoring. That is, the thermal imager of FLIR A615 is inclined at 45° and aimed at the motor winding end, so as to scan the motor winding end through the thermal imager and obtain the infrared image of the motor winding end. And according to the infrared image of the motor winding end, the corresponding temperature rise rate is determined, which is:

[0132]

[0133] Wherein: is the temperature rise rate, is the temperature at t+△t, is the temperature at t, is the time interval.

[0134] In the process of specific implementation, the temperature of the heat dissipation fin of the motor is 47.8℃ after 10s of monitoring, and the temperature of the heat dissipation fin of the motor reaches 51.3℃ after 20s of monitoring, so the corresponding temperature rise rate is 21℃ / min.

[0135] Step SB1.1.2: Determine the correction coefficient. This involves comparing the temperature rise rate obtained in step SB1.1.1 with a preset temperature rise threshold range (which can be set based on actual operating data and is not specifically explained in this embodiment; this embodiment uses 10°C / min-20°C / min as an example). Based on the comparison result, determine the corresponding first-level correction coefficient, specifically:

[0136] When the obtained temperature rise rate is less than the lower limit of the preset temperature rise threshold range (i.e., 10°C / min), the first-level correction coefficient is set to 1. When the obtained temperature rise rate is within the preset temperature rise threshold range (i.e., 10°C / min-20°C / min), the first-level correction coefficient is set to 1.5. When the obtained temperature rise rate is greater than the upper limit of the preset temperature rise threshold range (i.e., 20°C / min), the first-level correction coefficient is set to 2.

[0137] Furthermore, according to the vibration amplitude obtained in the multimodal sensing module and the preset vibration threshold range (which can be specifically set according to the actual operation data, and is not specifically explained in this embodiment, and is only 15mm / s in this embodiment), 2 -30mm / s 2 As an example, the corresponding secondary correction coefficient is determined based on the comparison result, which is as follows:

[0138] When the vibration amplitude is less than the lower limit of the preset vibration threshold range (i.e. 15 mm / s 2 ), the secondary correction coefficient is set to 1. When the vibration amplitude obtained is within the preset vibration threshold range (i.e. 15mm / s 2 -30mm / s 2 ), the secondary correction coefficient is set to 1.5. When the vibration amplitude obtained is greater than the upper limit threshold of the preset vibration threshold range (i.e. 30mm / s 2 ), the secondary correction factor is set to 2.

[0139] In this embodiment, the determined primary correction coefficient and secondary correction coefficient are combined to construct a correction coefficient table, as shown in Table 2 below:

[0140] Table 2: Correction coefficient table

[0141]

[0142] Step SB1.1.3: determining the correction index. That is, according to the vibration amplitude and temperature rise rate obtained in real time, the corresponding correction coefficient is determined from the correction coefficient table constructed in step SB1.1.2, and the obtained comprehensive risk index is corrected according to the determined correction coefficient to obtain the corrected comprehensive risk index, which is specifically:

[0143]

[0144] Wherein: is the corrected comprehensive risk index, is the comprehensive risk index, is the correction coefficient.

[0145] Referring to Figure 4 , the green area in the figure corresponds to a correction coefficient of 1, at this time the temperature rise rate is low or the temperature rise is moderate but the vibration is slight, so there is no need to make additional correction to the comprehensive risk index, and the equipment is in a safe working condition. The yellow area in the figure corresponds to a correction coefficient of 1.5, at this time the moderate temperature rise rate is superimposed with moderate vibration, and a correction coefficient of 1.5 times is required, that is, the equipment may face a mild risk and needs to be monitored. The red area in the figure corresponds to a correction coefficient of 2, that is, when the high temperature rise rate or the moderate temperature rise is superimposed with severe vibration, the correction coefficient rises to 2 times, which belongs to an extreme working condition, and the equipment reliability decreases significantly, so immediate intervention is required.

[0146] Although embodiments of the present application have been shown and described, it will be understood by those of ordinary skill in the art that various changes, modifications, substitutions and alterations can be made thereto without departing from the principles and spirit of the present application, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A deep learning-based production line operation AI detection system, characterized by, The application relates to a production system risk control method and device, and belongs to the technical field of production system risk control. The application comprises the following steps: A multi-modal sensing module: through an industrial camera and a laser sensor, a product is monitored to obtain multi-modal data of the product; A decision processing module: according to the multi-modal data, a comprehensive risk index is obtained, and a hierarchical control strategy is set according to the comprehensive risk index, and a compensation instruction is determined, comprising: SB1: dynamic decision: the defect probability, the to-be-tested assembly gap and the vibration amplitude are fused to determine the comprehensive risk index, the comprehensive risk index is compared with a preset risk threshold range, and the control strategy is determined according to the comparison result, specifically: When the comprehensive risk index is less than the lower limit threshold of the preset risk threshold range, the product continues to be detected; when the comprehensive risk index is within the preset risk threshold range, the transmission rate of the conveying belt is reduced, and the product is rechecked; when the comprehensive risk index is greater than the upper limit threshold of the preset risk threshold range, an alarm signal is triggered, and a shutdown process is performed, and a 3D line laser is used to scan a defect position to obtain defect three-dimensional point cloud data; SB2: self-learning processing: according to the critical assembly tolerance of different product models, a compensation instruction for rechecking the product is determined, comprising: SB2.1: tolerance library management: according to the critical tolerance and the allowable compensation times of each product model, a product tolerance library is constructed, and the compensation state and the deviation ratio of the current product are determined according to the product tolerance library; SB2.2: compensation instruction generation: the deviation ratio is compared with a preset deviation threshold range, and the compensation behavior is determined according to the comparison result, specifically: When the deviation ratio is less than the lower limit threshold of the preset deviation threshold range, the rechecking behavior of the current product is not compensated; when the deviation ratio is within the preset deviation threshold range, the rechecking behavior of the current product is compensated; when the deviation ratio is greater than the upper limit threshold of the preset deviation threshold range, an alarm signal is triggered, and a shutdown process is performed, and the process of the current product is reviewed, and the production process of the current product is re-determined; ； wherein: is a force increment, is a proportional coefficient, is an integral coefficient, is an assembly gap to be measured, is a critical tolerance; When the rechecking behavior of the current product is compensated, the compensation force increment size is determined according to the to-be-tested assembly gap and the critical tolerance of the current product, specifically: A multi-objective processing module: through the multi-modal sensing module and the decision processing module, a quality-efficiency trade-off coefficient is determined, and the running mode of the production system is adjusted according to the quality-efficiency trade-off coefficient, comprising: ； wherein: is a quality-efficiency trade-off coefficient, is a standard beat time, is a detection delay time, is a quality loss, is a proportional weight; SC1: determining the trade-off coefficient: the quality loss and the detection delay time of the product are determined through the multi-modal sensing module and the decision processing module, the standard tact time of the product is determined through an MES system, and the quality-efficiency trade-off coefficient is determined according to the quality loss, the detection delay time and the standard tact time of the product, specifically: SC2: determining the working mode: the quality-efficiency trade-off coefficient is compared with a preset trade-off threshold, and the running mode of the production system is determined according to the comparison result, specifically: When the quality-efficiency trade-off coefficient is greater than the preset trade-off threshold, a quality priority mode is started; otherwise, an efficiency priority mode is started. 2.The deep learning-based production line operation AI detection system of claim 1, wherein The multi-modal data of the product is obtained, including: SA1: visual inspection: the preprocessed product image is input into the improved ResNet-50 model, and the weighted feature map flattened data is output, and the linearly transformed original score is determined according to the weighted feature map flattened data, and the defect probability is obtained through the linearly transformed original score; SA2: laser ranging: the assembly gap is measured by a laser sensor to obtain the measured assembly gap; SA3: vibration detection: the Lucas-Kanade optical flow algorithm and the feature point image are used to determine the feature point displacement, and the vibration amplitude is determined according to the feature point displacement. 3.The deep learning-based production line operation AI detection system of claim 1, wherein According to the defect probability, the measured assembly gap and the vibration amplitude, the comprehensive risk index is determined, specifically: ； wherein: is the comprehensive risk index, is the natural logarithm base, is the defect probability converted by the Sigmoid function, is the assembly gap to be tested, is the vibration amplitude, , , is the weight coefficient. 4.The deep learning-based production line operation AI detection system of claim 3, wherein According to the defect probability, the assembly gap and the vibration amplitude of a plurality of samples, the normalized defect probability, the normalized assembly gap and the normalized vibration amplitude are obtained, and according to the initial weight coefficient, the normalized defect probability, the normalized assembly gap and the normalized vibration amplitude, the initial comprehensive risk index of the plurality of samples is determined, and the initial comprehensive risk index and the detection result are input into the maximum likelihood estimation model to output the updated weight coefficient. 5.The deep learning-based production line operation AI detection system of claim 3, wherein According to the temperature rise rate and the vibration amplitude, the comprehensive risk index is corrected, including: SB1.1.1: temperature rise monitoring: the product component is scanned by a thermal imager to obtain an infrared image of the product component, and the temperature rise rate is determined according to the infrared image; SB1.1.2: determining the correction coefficient: according to the temperature rise rate and the preset temperature rise threshold range, the first correction coefficient is determined, according to the vibration amplitude and the preset vibration threshold range, the second correction coefficient is determined, and the first correction coefficient and the second correction coefficient are combined to construct a correction coefficient table; SB1.1.3: determining the correction index: according to the vibration amplitude and the temperature rise rate, the correction coefficient is determined through the correction coefficient table, and the comprehensive risk index is corrected according to the correction coefficient to obtain the corrected comprehensive risk index. 6.The deep learning-based production line operation AI detection system of claim 5, wherein, The temperature rise rate and the preset temperature rise threshold range are compared, and according to the comparison result, the first correction coefficient is determined, specifically: When the temperature rise rate is less than the lower limit threshold of the preset temperature rise threshold range, the first correction coefficient is set to 1; when the temperature rise rate is within the preset temperature rise threshold range, the first correction coefficient is set to 1.5; when the temperature rise rate is greater than the upper limit threshold of the preset temperature rise threshold range, the first correction coefficient is set to 2; The vibration amplitude and the preset vibration threshold range are compared, and according to the comparison result, the second correction coefficient is determined, specifically: When the vibration amplitude is less than the lower limit threshold of the preset vibration threshold range, the second correction coefficient is set to 1; when the vibration amplitude is within the preset vibration threshold range, the second correction coefficient is set to 1.5; when the vibration amplitude is greater than the upper limit threshold of the preset vibration threshold range, the second correction coefficient is set to 2. 7.The deep learning-based production line operation AI detection system of claim 1, wherein According to the quality-efficiency trade-off coefficients of the plurality of sample data, a trade-off coefficient range of the plurality of sample data is determined, and the trade-off coefficient range is divided into a plurality of intervals, and meanwhile, the plurality of sample data is divided according to the intervals, a trade-off coefficient-cost fitting formula is constructed according to the average total cost and the trade-off coefficient interval of the plurality of sample data in each interval, and a preset trade-off threshold is determined through the quadratic term coefficient and the linear term coefficient in the trade-off coefficient-cost fitting formula, specifically: ； wherein: is a preset trade-off threshold, is a quadratic term coefficient, is a linear term coefficient.

Images