A plastic material taking system with visual identification

Abstract

The application relates to the technical field of industrial automation, and discloses a plastic taking system with visual recognition, which comprises a motor one, the output end of the motor one is fixedly connected with a conveying belt, the outer side of the conveying belt is sequentially provided with a centering assembly, a turnover assembly and a taking manipulator along the conveying direction of a to-be-detected object, the top of the turnover assembly is provided with a three-dimensional reconstruction camera array, the inside of the taking system is provided with a control system, the control system receives and processes sensing data output by the centering assembly, the turnover assembly and the three-dimensional reconstruction camera array, generates action parameters for controlling the taking manipulator based on the sensing data, and the taking manipulator performs a grabbing action according to the action parameters. Through the centering assembly, self-adaptive centering of to-be-detected objects with different sizes is realized, through the turnover assembly, stable turnover of the to-be-detected objects is realized, and through the control system, multi-modal data are fused, and closed-loop optimization of an injection molding manufacturing process is realized.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of industrial automation, in particular to a plastic taking system with visual recognition. BACKGROUND

[0002] In the injection molding industry, after the production of plastic parts is completed, an automatic system is usually required to complete subsequent processes such as taking from the mold or the conveyor belt, quality detection, and classification and stacking. In order to ensure product quality, the quality detection link integrated in the automatic system is crucial.

[0003] When detecting the quality of the plastic parts after injection molding, in order to obtain the complete three-dimensional appearance, internal structure and other multi-dimensional physical properties of the object to be detected, it is usually necessary to perform a flipping operation on the object to be detected to achieve comprehensive detection of all surfaces. However, the flipping mechanism in the prior art is usually simple in structure, for example, using a pneumatic lever or a fixed stopper, and the flipping force applied is difficult to control. When facing objects to be detected with different weights, shapes or surface friction coefficients, insufficient or excessive flipping will inevitably occur. This unstable flipping process not only interrupts the detection process, but also causes the subsequent sensor to be unable to collect accurate data at the predetermined position, seriously affecting the reliability of the entire detection system.

[0004] In addition, the traditional taking and detection system usually only undertakes the function of sorting, i.e. removing unqualified products detected. The detection system and the upstream injection molding production link are independent of each other. When the detection system finds that the product has quality deviations (such as warping or center of gravity deviation), these physical property data are not effectively utilized to feedback to the upstream production equipment to adjust the process parameters, resulting in the production process being unable to form a closed loop, and the production equipment continuously producing products with the same deviations. The system can only passively screen after the fact, making it difficult to ensure the consistency of product quality from the source, resulting in waste of raw materials and production capacity. SUMMARY

[0005] In view of the shortcomings of the prior art, the present application provides a plastic taking system with visual recognition, which solves the problems of difficulty in controlling the flipping of the object to be detected, inability to adapt to different sizes of the object to be detected when centering, and inability to integrate multi-dimensional perception data to realize closed-loop automatic calibration and control of the upstream production process.

[0006] To achieve the above purpose, the present application realizes the following technical solutions:

[0007] A plastic material handling system with visual recognition includes a motor, the output end of which is fixedly connected to a conveyor belt. A centering component, a flipping component, and a material handling robot are sequentially arranged along the conveying direction of the object to be measured on the outer side of the conveyor belt. A three-dimensional reconstruction camera array is arranged on the top of the flipping component. A control system is installed inside the material handling system. The control system receives and processes the perception data output by the centering component, the flipping component, and the three-dimensional reconstruction camera array, and generates motion parameters for controlling the material handling robot based on the perception data. The material handling robot performs a grasping action according to the motion parameters.

[0008] The flipping assembly includes two support frames. A second motor is fixedly connected to the outer side of one of the support frames. A rotating rod is fixedly connected to the output end of the second motor. Two connecting seats are fixedly connected to the outer side of the rotating rod. Multiple flipping rods are fixedly connected to the outer side of the connecting seats. A sensing pad is provided on the outer side of the flipping rod. An impact force sensor and a structural sound sensor are provided on the outer side of the flipping rod. A connecting plate is fixedly connected at the top between the two support frames.

[0009] Preferably, the centering assembly includes two bases, a fixing plate is fixedly connected to the top of the bases, a cylinder is fixedly connected to the outside of the fixing plate, a baffle is fixedly connected to the output end of the cylinder, multiple guide rods are fixedly connected to the bottom of the baffle, and multiple acoustic emission sensors and multiple temperature sensors are provided on the outside of the baffle.

[0010] Preferably, the sensing data output by the centering component includes frictional vibration signals and temperature rise data; the acoustic emission sensor is used to collect the frictional vibration signals, and the temperature sensor is used to measure the temperature rise; the control system is configured to analyze the frictional vibration signals to determine the surface acoustic texture of the object under test. The control system analyzes the frictional vibration signals... Perform time-frequency analysis, such as short-time Fourier transform (STFT), to obtain the spectral characteristics of the friction vibration signal. :

[0011] ;

[0012] in, The friction vibration signal is located at the center of the time window. Nearby, angular frequency The complex amplitude at a given point represents the intensity and phase information of that frequency component, respectively. For integration operations, representing time... Integrating over the entire domain, the overall contribution of the frictional vibration signal to a certain frequency component is calculated; The original time-domain frictional vibration signal acquired by the acoustic emission sensor is a signal that varies with time. A changing amplitude function; The window function is located at the center of the time window. A function that takes a non-zero value in the vicinity of time and a zero value or close to zero at other time points is used to obtain... A local spectral analysis is performed by extracting a time period from the data. The frictional vibration signal after windowing represents the position at the center of the time window. The nearby intercepted signal fragments; is the base of the natural logarithm; The imaginary unit; It is a component of the formula, representing an angular frequency of... The complex exponential basis function is used to perform correlation calculation with the friction vibration signal and extract the frequency component.

[0013] The control system analyzes the The square of the modulus, that is The spectrum (i.e., the surface acoustic texture) is constructed and compared with a preset defect feature library to identify whether there are burrs, scratches or flow marks on the surface of the object to be tested.

[0014] The control system is also configured to estimate the dynamic friction coefficient of the object under test based on the temperature rise. The control system estimates the dynamic friction coefficient using the following formula. :

[0015] ;

[0016] in, The dynamic friction coefficient is mentioned above; These are calibration coefficients, which are related to the specific heat capacity and thermal conductivity of the baffle material and the thermal response characteristics of the sensor, and are obtained through pre-calibration. The normal pressure applied to the object under test by the baffle is determined by the driving pressure of the cylinder; The speed of the conveyor belt is the relative sliding speed between the object to be measured and the surface of the baffle. This is the related quantity of frictional work per unit time; The temperature rise rate measured by the temperature sensor represents the temperature change caused by friction per unit time. The formula for calculating the temperature rise rate is:

[0017] ;

[0018] in, To be in the time interval The measured temperature change.

[0019] Preferably, the sensing data outputted by the flipping assembly includes landing pressure, impact force and internal structure acoustic signal; the sensing mat is used to measure the landing pressure, the impact force sensor is used to measure the impact force, and the structure acoustic sensor is used to collect the internal structure acoustic signal of the object to be measured; the control system is further configured to determine the mass and elastic modulus of the object to be measured according to the impact force. The control system calculates the mass of the object to be measured according to the impulse momentum theorem by time integration of the impact force .

[0020] ;

[0021] wherein, M is the mass of the object to be measured; is a definite integral operation, indicating integration of the function in the parentheses from the impact start time to the impact end time ; is the impact force measured by the impact force sensor, which changes over time ; represents the total impulse received by the object to be measured during the flipping process; is the instantaneous speed of the rotating rod before impact, which is determined by the encoder of the second motor; is the initial projection speed of the object to be measured after impact, which can be calculated through subsequent dynamic capture of the three-dimensional reconstruction camera array.

[0022] The control system is further configured to determine the internal structure defects of the object to be measured according to the internal structure acoustic signal. The control system performs spectral analysis on the internal structure acoustic signal, and judges whether the object to be measured has internal cracks or bubbles by identifying the shift of the resonance peak or the change of the modal damping ratio. The control system is further configured to determine the center of gravity position and the warping deformation state of the object to be measured according to the landing pressure. The sensing mat is an array sensor, which outputs a pressure distribution map. The control system determines the center of gravity projection coordinates of the object to be measured by calculating the centroid of the pressure distribution map .

[0023] ;

[0024] ;

[0025] wherein, and are the projection positions of the center of gravity of the object to be measured in the sensing mat coordinate system Coordinates and Coordinates; and are coordinate variables in the coordinate system of the sensing mat plane respectively; is the pressure value measured by the sensing mat at coordinate ; is the differential of coordinate; is the differential of coordinate; and are the total pressure moments on axis and axis respectively; is the total vertical force acting on the sensing mat, i.e. the weight of the object to be measured.

[0026] The control system determines whether there is a warping deformation by analyzing the uniformity of the pressure value distribution of .

[0027] Preferably, the perception data output by the three-dimensional reconstruction camera array is continuous image information, and the three-dimensional reconstruction camera array is used to collect continuous image information in three stages: before the object to be measured is turned over, image information for reconstructing an initial top surface three-dimensional model of the object to be measured is collected; during the turning over of the object to be measured, image information for reconstructing a dynamic three-dimensional model of the object to be measured is collected; after the turning over of the object to be measured is completed, image information for reconstructing a new top surface three-dimensional model of the object to be measured is collected; the control system is further configured to fuse the image information of the three stages to generate a complete three-dimensional model of the object to be measured. The control system uses a multi-view stereo vision algorithm to reconstruct a three-dimensional model in a static state, and uses a spatio-temporal multi-view reconstruction algorithm to reconstruct a three-dimensional model in a dynamic process, and finally fuses point cloud data of the three stages into the complete three-dimensional model through a point cloud registration algorithm.

[0028] Preferably, a support rod is arranged outside the conveying belt, two cameras are arranged outside the support rod, and the cameras are arranged opposite to the working range of the material taking manipulator; the control system is further configured to compensate and correct the end position of the material taking manipulator according to image information collected by the cameras before the material taking manipulator performs a grabbing action. The control system obtains the positions of the end effector of the material taking manipulator and a target grabbing point in an image through the cameras, calculates an image error vector , and calculates a required joint speed instruction vector through an image Jacobian matrix to drive the material taking manipulator to eliminate the image error:

[0029] ;

[0030] wherein, is a joint velocity command vector of the material handling robot; is a control gain coefficient; is a pseudo-inverse matrix of the image Jacobian matrix ; is an error vector in the image space, representing the deviation of the current image position of the end effector from the target image position.

[0031] Preferably, the baffle is rotationally connected to the top of the base, the guide rod is rotationally connected to the outside of the base, the baffle abuts the object to be measured, the base is fixedly connected to the outside of the conveying belt, the rotating rod is rotationally connected between the two support frames, and the three-dimensional reconstruction camera array is arranged outside the connecting plate.

[0032] Preferably, the control system interacts with the upstream injection molding process in data, and the control system comprises a cross-domain digital twin model for representing a mapping relationship between process parameters of the injection molding process and physical properties of the object to be measured. The cross-domain digital twin model is a mapping function, and the relationship is represented by the following formula:

[0033] ;

[0034] wherein, is a process parameter vector of the injection molding process; is a physical property vector of the object to be measured.

[0035] A process parameter generation module, based on the cross-domain digital twin model, takes a preset ideal physical property as a target, and reversely calculates optimal process parameters. The process parameter generation module calculates the optimal process parameter vector by solving the following optimization problem:

[0036] ;

[0037] wherein, is the optimal process parameter vector calculated; is an optimization operation, aiming to find a process parameter vector that minimizes the expression in the parentheses; is a process parameter vector of the injection molding process; is a mapping function represented by the cross-domain digital twin model; is a physical property vector predicted by the model according to the process parameter vector ; is a preset ideal physical property vector; is a loss function used to calculate the difference between the predicted attribute and the ideal attribute; is a first part of the composite objective function, representing the deviation between the attribute predicted by the model and the ideal attribute; is a regularization term used to adjust the process parameter vector applies a constraint to ensure that the process parameter vector is within a safe or reasonable range; is a regularization coefficient used to adjust the weight of the regularization term; is a second part of the composite objective function, i.e. the weighted regularization term.

[0038] A process control instruction module receives the optimal process parameters output by the process parameter generation module and generates executable control instructions for adjusting the injection molding process.

[0039] Preferably, the physical attributes include: impact force measured by the impact force sensor, structural acoustic signals collected by the structural acoustic sensor, friction vibration signals collected by the acoustic emission sensor, dynamic friction coefficient estimated from the measurement results of the temperature sensor, center of gravity position and warping deformation state characterized by the landing pressure measured by the sensor mat, and complete three-dimensional model generated by the three-dimensional reconstruction camera array.

[0040] Preferably, the process parameter generation module is configured to: when the control system determines that the physical attribute of the current object under test deviates from the ideal physical attribute, start reverse calculation to generate optimal process parameters that can make the physical attribute of the object under test produced subsequently return to the ideal physical attribute. The process control instruction module is configured to: before generating the executable control instruction, evaluate the prediction confidence of the cross-domain digital twin model, and when the prediction confidence is lower than a preset threshold, stop generating the executable control instruction and issue an alarm.

[0041] The present application provides a plastic taking system with visual recognition. It has the following advantages:

[0042] 1、The present application sets up a turnover assembly, which includes a rotating rod and a turnover rod provided with a sensor mat. Under the cooperative control of the control system, the rotating rod drives the turnover rod to rotate, thereby exerting a controllable turnover effect on the object under test, realizing stable and controllable turnover of the object under test, and effectively avoiding the problems of insufficient turnover or over-turning.

[0043] 2. The application sets the centering assembly, which includes a cylinder and a baffle. In the initial state, the baffle is in the open state, forming a dynamic centering channel, which facilitates the entry of the measured object. In the centering process, the cylinder drives the baffle to rotate and gradually clamps, realizing self-adaptive centering of different sizes of measured objects. The size of the centering channel can be dynamically changed according to the measured object, improving the application range of the centering process.

[0044] 3. The application sets the control system, which receives and processes the perception data output by the centering assembly, the overturning assembly and the three-dimensional reconstruction camera array, realizing multi-dimensional data fusion of the visual three-dimensional data, physical impact data and surface texture data of the measured object. The control system also generates a cross-domain digital twin model and a process parameter generation module, which reversely calculates and optimizes the process parameters of the upstream injection molding process according to the fused data, realizes closed-loop calibration and automatic regulation and control of the injection molding process, and ensures the consistency of product quality. BRIEF DESCRIPTION OF DRAWINGS

[0045] Figure 1 It is a perspective view of the application;

[0046] Figure 2 It is a structural schematic view of the strut of the application;

[0047] Figure 3 It is a structural schematic view of the baffle of the application;

[0048] Figure 4 It is a structural schematic view of the guide rod of the application;

[0049] Figure 5 It is a structural schematic view of the overturning assembly of the application;

[0050] Figure 6 It is a structural schematic view of the three-dimensional reconstruction camera array of the application;

[0051] Figure 7 It is a control system function module and work flow schematic view of the embodiment of the application.

[0052] 1, conveyor belt; 2, motor one; 3, material taking manipulator; 4, strut; 5, camera; 6, support frame; 7, motor two; 8, rotating rod; 9, connecting seat; 10, overturning rod; 11, sensing pad; 12, impact force sensor; 13, structural sound sensor; 14, connecting plate; 15, three-dimensional reconstruction camera array; 16, base; 17, fixed plate; 18, cylinder; 19, baffle; 20, guide rod; 21, acoustic emission sensor; 22, temperature sensor; 23, cross-domain digital twin model; 24, process parameter generation module; 25, process control instruction module. DETAILED DESCRIPTION

[0053] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the specification of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative work fall within the scope of protection of the present application.

[0054] Referring to the drawings Figure 1 - the drawings Figure 7 The present application provides a plastic taking system with visual recognition, comprising a motor 2, the output end of the motor 2 is fixedly connected with a conveying belt 1, the conveying belt 1 is provided with a centering assembly, a turnover assembly and a taking manipulator 3 in sequence along the conveying direction of the object to be measured on the outer side, the turnover assembly is provided with a three-dimensional reconstruction camera array 15 at the top, and the inside of the taking system is provided with a control system.

[0055] Specifically, the motor 2 provides power source for the conveying belt 1, can control the running speed of the conveying belt 1, ensures that the object to be measured is conveyed at a preset rhythm, and guarantees that the subsequent processes can match the material transmission speed; the conveying belt 1 serves as a conveying carrier for the object to be measured, can stably transmit the object to be measured along a fixed direction, and provides continuous and orderly material supply for the subsequent centering, turnover, taking and other processes; the centering assembly is used for self-adaptive position correction of the object to be measured, ensures that the object to be measured of different sizes can be at the center position of conveying; the turnover assembly is used for realizing stable and controllable turnover of the object to be measured, avoiding insufficient or excessive turnover; the taking manipulator 3 executes the grabbing action according to the action parameters generated by the control system, completes the grabbing and transfer of the object to be measured, replaces manual taking, and improves the taking efficiency and precision; the three-dimensional reconstruction camera array 15 collects continuous image information in three stages of before turnover, during turnover and after turnover of the object to be measured.

[0056] The control system receives and processes the perception data output by the centering assembly, the turnover assembly and the three-dimensional reconstruction camera array 15, and generates action parameters for controlling the taking manipulator 3 based on the perception data, and the taking manipulator 3 executes the grabbing action according to the action parameters.

[0057] The control system also interacts with the upstream injection molding process. In terms of function, the control system includes a cross-domain digital twin model 23, a process parameter generation module 24 and a process control instruction module 25. The cross-domain digital twin model 23 is used to represent the mapping relationship between the injection molding process parameters and the physical properties of the object to be measured; the process parameter generation module 24 is used to reversely calculate the optimal process parameters according to the deviation between the detected physical properties and the ideal properties; and the process control instruction module 25 is used to receive the optimal process parameters output by the process parameter generation module 24, and generate executable control instructions for adjusting the injection molding process.

[0058] Reference drawingsFigure 1 -attach Figure 4 The centering assembly includes two bases 16, the top of which is fixedly connected with a fixed plate 17, the outer side of which is fixedly connected with a pneumatic cylinder 18, the output end of which is fixedly connected with a baffle 19, the bottom of which is fixedly connected with a plurality of guide rods 20, and the outer side of which is provided with a plurality of acoustic emission sensors 21 and a plurality of temperature sensors 22. The baffle 19 is rotatably connected to the top of the base 16, and the guide rod 20 is rotatably connected to the outer side of the base 16. The baffle 19 abuts against the object to be measured, and the base 16 is fixedly connected to the outer side of the conveying belt 1.

[0059] Specifically, the base 16 is a basic support structure of the centering assembly, providing an installation carrier for other components in the assembly and ensuring the stability of each component of the centering assembly during operation; the fixed plate 17 is used to fixedly install the pneumatic cylinder 18, stably connecting the pneumatic cylinder 18 to the base 16 and ensuring that the pneumatic cylinder 18 does not shake when driving the baffle 19 to move; the pneumatic cylinder 18 provides driving force for the baffle 19, can control the movement speed of the baffle 19, and drives the baffle 19 to realize opening and centering actions, thereby adapting to the centering needs of objects to be measured of different sizes; the baffle 19 realizes centering operation by abutting against the object to be measured during centering, and performs position correction on the object to be measured by driving the baffle 19 to move to the center of the conveying belt 1; the guide rod 20 is used to ensure the smooth movement of the baffle 19 along the preset trajectory; the acoustic emission sensor 21 and the temperature sensor 22 collect sensing data to assist in judging the surface state of the object to be measured.

[0060] The baffle 19 can rotate around the top of the base 16, cooperate with the pneumatic cylinder 18 to drive the baffle 19 to realize opening and centering actions, and adapt to the centering needs of objects to be measured of different sizes; the guide rod 20 provides stable guidance when the baffle 19 moves, ensures that the baffle 19 moves smoothly along the preset trajectory to correct the position of the object to be measured, and avoids the decline of centering accuracy caused by the deviation of the baffle 19; the base 16 stably fixes the centering assembly to the outer side of the conveying belt 1, ensures that the centering assembly matches the conveying rhythm of the conveying belt 1, and avoids displacement of the assembly during centering.

[0061] The sensing data output by the centering assembly includes friction vibration signals and temperature rise data; the temperature sensor 22 and the acoustic emission sensor 21 are arranged adjacent to each other to ensure that the temperature change of the same friction area can be captured. The temperature sensor 22 is used to measure the local temperature rise data caused when the object to be measured rubs against the surface of the baffle 19. The acoustic emission sensor 21 and the temperature sensor 22 are both electrically connected to the data acquisition interface of the control system through a signal transmission line.

[0062] When the object to be measured is transported to the centering assembly, the centering assembly begins to work. In the initial state, the baffle 19 controlled by the air cylinder 18 is in the open state, forming a dynamic centering channel, facilitating the smooth entry of the object to be measured. Subsequently, the air cylinder 18 drives the baffle 19 to rotate and gradually center, and the inner surface of the baffle 19 contacts the object to be measured until the position of the object to be measured is corrected to the center line of the conveying belt 1. The guide rod 20 provides motion guidance during the rotation of the baffle 19, ensuring that the baffle 19 smoothly corrects the position of the object to be measured along the preset trajectory.

[0063] In the process of the object to be measured being centered and sliding between the inner surfaces of the baffle 19 along the conveying belt 1, the acoustic emission sensor 21 collects the original time-domain friction vibration signal generated when the surface of the object to be measured rubs against the surface of the baffle 19 . At the same time, the temperature sensor 22 measures the temperature rise data generated by the above-mentioned friction on the same area of the surface of the baffle 19. The centering assembly outputs the collected friction vibration signal and temperature rise data to the control system in real time.

[0064] The control system receives the original time-domain friction vibration signal , and performs time-frequency analysis on it to determine the surface acoustic texture of the object to be measured. In a specific implementation, the control system performs short-time Fourier transform STFT on the original time-domain friction vibration signal to obtain its time-frequency characteristics :

[0065] ;

[0066] wherein, is the complex amplitude of the friction vibration signal near the center position of the time window , the angular frequency , whose amplitude and phase represent the intensity and phase information of the frequency component, respectively; is the integral operation, indicating the integration of the friction vibration signal over time to calculate the overall contribution of the friction vibration signal at a certain frequency component; is the original time-domain friction vibration signal collected by the acoustic emission sensor 21, which is a function of the amplitude varying with time ; is the window function, which takes a non-zero value near the center position of the time window , and takes a zero value or a value close to zero at other time points, used to extract a time period from for local spectral analysis; is the windowed friction vibration signal, representing the signal segment extracted near the center position of the time window ; is the base of the natural logarithm; is the imaginary unit; is a component of the formula, representing a complex exponential base function with an angular frequency , which is used for correlation calculation with the friction vibration signal to extract the frequency component.

[0067] The control system analyzes the square of the modulus of , i.e. , to form a spectrum (i.e. surface acoustic texture), and compares it with a pre-set defect feature library to identify whether the surface of the object under test has burrs, scratches or flow marks.

[0068] The control system is also configured to estimate the dynamic friction coefficient of the object under test according to the temperature rise data measured by the temperature sensor 22. The control system estimates the dynamic friction coefficient by the following formula:

[0069] ;

[0070] wherein, is the dynamic friction coefficient; is a calibration coefficient, which is related to the material specific heat capacity, thermal conductivity of the baffle 19 and the thermal response characteristics of the sensor, and is obtained by pre-calibration; is the normal pressure applied by the baffle 19 to the object under test, which is determined by the driving pressure of the air cylinder 18; is the running speed of the conveyor belt 1, i.e. the relative sliding speed between the object under test and the surface of the baffle 19; is the correlation quantity of friction work per unit time; is the temperature rise rate measured by the temperature sensor 22, representing the temperature change per unit time due to friction, and the calculation formula of the temperature rise rate is:

[0071] ;

[0072] wherein, is the temperature change amount measured within the time interval .

[0073] Reference is made to the accompanying drawings, wherein: Figure 1 Figure 5 and Figure 6The overturning assembly comprises two support frames 6, one of which is fixedly connected with a motor 7 outside, the output end of the motor 7 is fixedly connected with a rotating rod 8, the rotating rod 8 is fixedly connected with two connecting seats 9 outside, the connecting seats 9 are fixedly connected with a plurality of overturning rods 10 outside, the overturning rods 10 are provided with sensing pads 11 outside, the overturning rods 10 are provided with impact force sensors 12 and structural sound sensors 13 outside, and the two support frames 6 are fixedly connected with a connecting plate 14 at the top.

[0074] Specifically, the support frame 6 is a basic support structure of the overturning assembly, which provides an installation carrier for other components in the assembly and ensures the structural stability of each component of the overturning assembly during work; the motor 7 provides driving force for the rotating rod 8 and can control the rotation speed and angle of the rotating rod 8, thereby providing accurate impact force and overturning progress for the object to be measured, avoiding insufficient or excessive overturning; the rotating rod 8 rotates under the drive of the motor 7 and can apply accurate overturning impact force to the object to be measured to push the object to complete the overturning action; the connecting seat 9 connects the rotating rod 8 and the overturning rod 10 and stably fixes the overturning rod 10 on the rotating rod 8; the overturning rod 10 supports the object to be measured during the overturning process and cooperates with the rotating rod 8 to realize stable overturning of the object to be measured; the sensing pads 11, the impact force sensors 12 and the structural sound sensors 13 collect sensing data during the overturning process of the object to be measured; the connecting plate 14 is used to enhance the stability of the overall structure of the overturning assembly and provides an installation position for the three-dimensional reconstruction camera array 15, ensuring that the camera array can be aligned with the object to be measured and ensuring the image acquisition effect. The support frame 6 is used to enable the rotating rod 8 to stably rotate around the support frame 6 under the drive of the motor 7.

[0075] The sensing data output by the overturning assembly includes landing pressure, impact force and internal structural sound signals, which are used to collect physical property data of the object to be measured during the overturning and landing processes;

[0076] In a preferred embodiment, the sensing pads 11 are arrayed pressure sensors, which can measure the pressure size and distribution when the object to be measured lands on the surface of the sensing pads 11 and output a pressure distribution map ; the impact force sensors 12 are used to measure the impact force function when the object to be measured lands on the overturning rod 10 ; and the structural sound sensors 13 are used to collect the structural sound signals radiated and propagated inside the object to be measured during the impact and landing processes.

[0077] The sensing mat 11, the impact force sensor 12 and the structural sound sensor 13 are all electrically connected with the data acquisition interface of the control system through signal transmission lines. The control system receives the landing pressure, impact force and internal structural sound signals output by these sensors for subsequent analysis of physical properties such as mass, center of gravity, internal defects, etc.

[0078] After the object to be measured is centered, it enters the working area of the turnover assembly with the conveying belt 1. The control system controls motor two 7 to rotate at a preset speed and angle, thereby driving the rotating rod 8 to rotate, and the turnover rod 10 and the sensing mat 11 form a receiving platform, cooperating with the operation of the conveying belt 1 to realize stable and controllable turnover of the object to be measured. In the turnover process, the impact force sensor 12 and the structural sound sensor 13 respectively collect the physical signals generated when the object to be measured lands.

[0079] After the control system receives the impact force output by the impact force sensor 12, the mass of the object to be measured is calculated by the impact force. In a specific implementation, the mass is calculated by the following impulse-momentum relationship:

[0080] ;

[0081] Wherein, is the mass of the object to be measured; is a definite integral operation, indicating the integration of the function in the parentheses from the impact start time to the impact end time ; is the impact force measured by the impact force sensor 12 that changes over time ; represents the total impulse received by the object to be measured during the turnover process; is the instantaneous speed of the rotating rod 8 before impact, determined by the encoder of motor two 7; is the initial projectile speed of the object to be measured after impact, which can be calculated through the subsequent dynamic capture of the three-dimensional reconstruction camera array 15.

[0082] At the same time, the control system analyzes the waveform characteristics of the impact force such as peak height, pulse width and rise time, and compares them with the preset material physical model to determine the equivalent elastic modulus of the object to be measured.

[0083] The control system also receives the structural acoustic signal collected by the structural acoustic sensor 13. The control system performs a spectral analysis (e.g. fast Fourier transform, FFT) on the structural acoustic signal to obtain the spectral feature of the structural acoustic signal. By comparing the spectral feature with a pre-stored standard spectrum of a good product without defects, the control system can identify the spectral abnormality (e.g. the appearance or shift of a resonance peak at a specific frequency) caused by defects such as internal cracks, cavities or delamination, and determine whether the object under test has internal structural defects.

[0084] After the object under test is stably stationary on the sensing mat 11, the control system receives the pressure distribution map output by the sensing mat 11 . The control system calculates the barycentric projection coordinates of the object under test in the stationary state by performing an integral operation on the pressure distribution map:

[0085] ;

[0086] ;

[0087] wherein and are the coordinate and coordinate of the projection position of the barycenter of the object under test in the coordinate system of the sensing mat 11, respectively; and are coordinate variables in the coordinate system of the sensing mat 11 plane; is the pressure value measured by the sensing mat 11 at the coordinate ; is the differential of the coordinate; is the differential of the coordinate; and are the total moments of the total pressure about the axis and the axis, respectively; is the total vertical force acting on the sensing mat 11, i.e. the weight of the object under test.

[0088] In addition, the control system also determines the degree of warping deformation of the bottom surface of the object under test by analyzing the shape of the pressure distribution map , such as the distribution of pressure concentrated areas and pressure free areas.

[0089] Reference is made to the accompanying drawings Figure 6, the field of view range of the three-dimensional reconstruction camera array 15 overlooks the entire working area of the flipping assembly. The three-dimensional reconstruction camera array 15 includes a plurality of high-frame-rate cameras. The plurality of cameras are arranged from different spatial angles to ensure that multi-view images of the object under test can be captured. The control system is configured to receive and fuse the image data captured by the three-dimensional reconstruction camera array 15 in the three stages to generate a complete three-dimensional model of the object under test.

[0090] The control system is configured to control the three-dimensional reconstruction camera array 15 to perform image capture in the three stages in synchronization with the different stages of flipping of the object under test:

[0091] First stage (before flipping): when the object under test is transported by the conveyor belt 1 to the working area of the flipping assembly and is in a stationary state before the flipping action is performed, the control system controls the three-dimensional reconstruction camera array 15 to capture multi-view still images of the initial top surface (i.e., the first surface) of the object under test.

[0092] Second stage (during flipping): when the control system drives the motor 7 and the rotating rod 8 to operate, the object under test starts to flip and is in a state of motion in the air, the control system controls the three-dimensional reconstruction camera array 15 to continuously capture a dynamic multi-view image sequence of the object under test during the flipping process at a high frame rate (e.g., more than 100 frames per second).

[0093] Third stage (after flipping): after the object under test completes the flipping and lands stably on the conveyor belt 1, the control system controls the three-dimensional reconstruction camera array 15 to capture multi-view still images of the new top surface (i.e., the original bottom surface, or the second surface) of the object under test.

[0094] The control system receives and fuses all the image data captured in the above three stages. The control system reconstructs high-precision three-dimensional point clouds of the first surface and the second surface of the object under test through multi-view stereo vision algorithms (e.g., binocular or multi-view stereo matching based on the still images in the first and third stages).

[0095] At the same time, the control system reconstructs three-dimensional point clouds of the side surfaces and connecting edges exposed during the flipping of the object under test through spatiotemporal reconstruction algorithms (e.g., voxel carving or silhouette fusion based on the dynamic image sequence in the second stage).

[0096] Finally, the control system automatically registers and fuses the three-dimensional point cloud models of the first surface, the second surface, and the side surfaces and connecting edges in a unified coordinate system to ultimately generate a complete three-dimensional model of the object under test without dead angles.

[0097] Reference is made to the accompanying drawings Figure 1 and the accompanying drawings Figure 2The outer side of the conveying belt 1 is provided with a support rod 4, and the outer side of the support rod 4 is provided with two cameras 5, which are arranged opposite the working range of the material taking manipulator 3; the control system is further configured to compensate and correct the end position of the material taking manipulator 3 according to the image information collected by the cameras 5 before the material taking manipulator 3 performs the grabbing action.

[0098] Specifically, the support rod 4 is used to provide a stable support structure for the cameras 5, so as to ensure that the cameras 5 can always be in the preset position opposite the working range of the material taking manipulator 3; the cameras 5 collect image information before the material taking manipulator 3 performs the grabbing action, and the control system can compensate and correct the end position of the manipulator according to the information, so as to avoid grabbing failure caused by mechanical error or position deviation and improve the grabbing precision. The material taking manipulator 3 is a multi-axis industrial robot, and the end thereof is provided with a suction cup.

[0099] The control system is used to receive the images collected by the two cameras 5 in real time before performing the grabbing action. The control system extracts the current image feature vector of the end effector of the material taking manipulator 3 based on the images collected in real time . The control system also projects the target grabbing pose of the object to be tested (which is determined based on the complete three-dimensional model and the center of gravity position ) to the image planes of the two cameras 5 to obtain the target image feature vector .

[0100] The control system calculates the joint speed instruction vector of driving the joints of the material taking manipulator 3 to move, so as to compensate and correct the position deviation of the end effector and ensure that the material taking manipulator 3 accurately aligns with the target grabbing pose. One specific calculation method of the joint speed instruction vector is as follows:

[0101] ;

[0102] Wherein, is the joint speed instruction vector of the material taking manipulator 3; is the control gain coefficient; is the pseudo-inverse matrix of the image Jacobian matrix ; and is the error vector in the image space, which represents the deviation between the current image position of the end effector and the target image position.

[0103] In a specific implementation, the error vector is calculated as follows:

[0104] ;

[0105] Wherein, a current image feature vector extracted from the current images of the two cameras 5 in real time by the control system; a target image feature vector, calculated by the control system according to the preset grasping pose and camera model.

[0106] Reference is made to the accompanying drawings Figure 7 The control system of the present application integrates the functions of upstream process optimization, midstream multi-modal perception, and downstream mechanical grasping.

[0107] In a preferred embodiment, the workflow first includes an offline or inter-period process optimization phase. The process parameter generation module 24 in the control system, based on the cross-domain digital twin model 23 (characterized as a mapping function ), reversely calculates the optimal process parameter vector with the preset ideal physical properties as the target.

[0108] Subsequently, the process control instruction module 25 converts the optimal process parameter vector into executable instructions and sends them to the upstream injection molding process to guide the production of the object under test.

[0109] After the production of the object under test is completed, it is placed on the conveyor belt 1 driven by motor one 2, and begins to enter the online detection and material taking process of the present system.

[0110] The object under test first arrives at the centering assembly. The control system drives the cylinder 18 to make the baffle 19 clamp and position-center the object under test. In this process, the acoustic emission sensor 21 and the temperature sensor 22 respectively collect the original time-domain friction vibration signal and temperature rise data.

[0111] The control system receives the above-mentioned sensing signals and performs real-time calculation and processing: the short-time Fourier transform is performed on to obtain time-frequency features to determine the surface acoustic texture (used to identify burrs, scratches, etc.); and the dynamic friction coefficient is calculated from the temperature rise data.

[0112] Next, the object under test is conveyed to the turnover assembly. The control system drives motor two 7 and the rotating rod 8 to make the object under test realize turnover under the action of the conveyor belt 1.

[0113] During the turnover and landing process, the three-dimensional reconstruction camera array 15 collects multi-view images at three stages: before turnover, during turnover, and after turnover. The control system fuses all the images to reconstruct the complete three-dimensional model of the object under test.

[0114] At the instant the object under test lands on the sensing pad 11, the impact force sensor 12 and the structure acoustic sensor 13 respectively collect the impact force. and structural acoustic signals. The control system controls... Perform integration to calculate the mass They also determined internal structural defects by analyzing the spectrum of the structural acoustic signal.

[0115] After the object under test lands and comes to a stop, the sensing pad 11 collects the pressure distribution map. The control system, through... To perform weighted integration, the projected coordinates of the center of gravity of the object under test are calculated. And the bottom surface warping and deformation.

[0116] At this point, the control system has collected all the physical properties of the object under test. (including surface texture, coefficient of friction) ,quality Internal defects, center of gravity (and warping deformation, etc.) as well as a complete three-dimensional model of the object under test.

[0117] The control system is based on a complete 3D model and center of gravity coordinates. The stable grasping pose for the material handling robot 3 is calculated and projected onto the image plane of the camera 5 to obtain the target image feature vector. .

[0118] The control system activates the vision servo program. Camera 5 observes the end effector of the robotic arm 3 and the object to be measured in real time, extracting the feature vector of the current image. And calculate the error vector. .

[0119] The control system calculates the joint speed command vector in real time based on the image Jacobian matrix servo control formula. It drives the end effector of the picking robot 3 to accurately and stably align with the target grasping posture and perform the grasping.

[0120] In the final stage of the workflow (closed-loop feedback), the control system will monitor all the actual physical properties acquired online. The process parameters used to produce the object under test As a new data pair, it is used to update the cross-domain digital twin model 23, thereby improving the accuracy of the cross-domain digital twin model 23 and enabling the process parameter generation module 24 to calculate more optimized parameters in the next production cycle. .

[0121] The control system is the core of the entire detection and optimization closed loop, and in terms of hardware, it is a combination of one or more industrial computers, embedded systems, or PLCs. The control system achieves electrical connection and data communication with conveyor belt 1, motor 1, material handling robot 3, camera 5, motor 2, sensing pad 11, impact force sensor 12, structural acoustic sensor 13, 3D reconstruction camera array 15, cylinder 18, acoustic emission sensor 21, and temperature sensor 22 through a high-speed data bus and I / O interface.

[0122] The control system integrates multiple functional modules, including a module for real-time data acquisition and signal processing, a module for performing time-frequency analysis and physical parameter estimation, a module for 3D model reconstruction, a module for performing visual servo control, and a core process optimization closed-loop module.

[0123] The process optimization closed-loop module includes a cross-domain digital twin model 23, a process parameter generation module 24, and a process control instruction module 25.

[0124] The cross-domain digital twin model 23 is a core mathematical model established within the control system, representing the upstream manufacturing process parameter vector. Multimodal physical property vectors of downstream products Nonlinear mapping relationship between .

[0125] In one specific implementation, the cross-domain digital twin model 23 uses a mapping function To represent the relationship:

[0126] ;

[0127] in, This is a vector of process parameters for the injection molding process; This is the physical property vector of the object to be tested.

[0128] The establishment and optimization of the cross-domain digital twin model 23 is an ongoing process. The control system will compare each set of actual data acquired in the online detection process with... (i.e., process parameters used in production) And the physical properties actually measured under this parameter ) as new training samples, used for mapping functions in the cross-domain digital twin model 23 Iterative updates and corrections can be made, for example, by adjusting the parameters of the cross-domain digital twin model23 through the backpropagation algorithm to continuously improve its prediction accuracy.

[0129] The process parameter generation module 24 is a core computing unit within the control system, connected to the cross-domain digital twin model 23 and the process control instruction module 25.

[0130] The function of the process parameter synthesis module 24 is to solve the optimal upstream process parameter vector according to the pre-set, desired ideal physical property vector and using the mapping function provided by the cross-domain digital twin model 23 .

[0131] In one specific implementation, the process parameter synthesis module 24 calculates the optimal process parameter vector by solving the following optimization problem

[0132] ;

[0133] wherein, is the calculated optimal process parameter vector; is the optimization operation, aiming to find the process parameter vector that minimizes the expression in the parentheses; is the process parameter vector of the injection molding process; is the mapping function represented by the cross-domain digital twin model 23; is the physical property vector predicted by the cross-domain digital twin model 23 according to the process parameter vector ; is the pre-set ideal physical property vector; is the loss function used to calculate the difference between the predicted property and the ideal property; is the first part of the composite objective function, representing the deviation between the property predicted by the cross-domain digital twin model 23 and the ideal property; is the regularization term, used to impose constraints on the process parameter vector to ensure that the process parameter vector is within a safe or reasonable range; is the regularization coefficient used to adjust the weight of the regularization term is the second part of the composite objective function, i.e., the weighted regularization term.

[0134] After completing the above optimization calculation, the process parameter synthesis module 24 outputs the obtained optimal process parameter vector to the process control instruction module 25 for subsequent processing.

[0135] The process control instruction module 25 is a functional module within the control system, which receives the output from the process parameter synthesis module 24 on the data flow.

[0136] In a preferred embodiment, the process control instruction module 25 is electrically connected with the process parameter synthesis module 24 for receiving the optimal process parameter vector calculated by the process parameter synthesis module 24. .

[0137] The process control instruction module 25 also implements a communication connection with the controller of the upstream manufacturing equipment (e.g. an injection molding machine) through an industrial communication bus (e.g. EtherNet / IP, Profinet or OPCUA).

[0138] The core function of the process control instruction module 25 is to perform a conversion operation from the optimal process parameter vector to a specific set of process control instructions . is a mathematical vector (e.g. containing temperature, pressure, time, etc. numerical values), while is a formatted data packet or signal (e.g. a write command to a specific PLC register) compliant with the specific equipment communication protocol, this conversion process can be characterized by a conversion function :

[0139] ;

[0140] where, is the specific set of process control instructions generated by the process control instruction module 25; is the conversion function or protocol mapping table implemented internally by the process control instruction module 25, defined according to the specific communication protocol and control logic of the upstream manufacturing equipment, ensuring that each component in is correctly mapped to the corresponding parameter address and data format of the equipment controller; is the calculated optimal process parameter vector.

[0141] After the conversion is completed, the process control instruction module 25 sends the specific set of process control instructions to the controller of the upstream manufacturing equipment through the communication interface.

[0142] The upstream manufacturing equipment receives and executes the specific set of process control instructions to produce the next batch of objects under test according to the conditions set by the optimal process parameter vector . This process completes the entire closed-loop feedback from online multi-modal perception to upstream process parameter optimization.

Claims

1. A plastic material handling system with visual recognition, characterized in that, include: Motor 1 (2), the output end of which is fixedly connected to a conveyor belt (1), and a centering component, a flipping component and a picking robot (3) are arranged sequentially on the outside of the conveyor belt (1) along the conveying direction of the object to be measured. A three-dimensional reconstruction camera array (15) is arranged on the top of the flipping component. A control system is arranged inside the picking system. The control system receives and processes the perception data output by the centering component, the flipping component and the three-dimensional reconstruction camera array (15), and generates motion parameters for controlling the picking robot (3) based on the perception data. The picking robot (3) performs a grasping action according to the motion parameters. The flipping assembly includes two support frames (6), one of which is fixedly connected to a motor (7) on its outer side. The output end of the motor (7) is fixedly connected to a rotating rod (8). Two connecting seats (9) are fixedly connected to the outer side of the rotating rod (8). Multiple flipping rods (10) are fixedly connected to the outer side of the connecting seats (9). A sensing pad (11) is provided on the outer side of the flipping rod (10). An impact force sensor (12) and a structural sound sensor (13) are provided on the outer side of the flipping rod (10). A connecting plate (14) is fixedly connected at the top between the two support frames (6).

2. The plastic material handling system with visual recognition according to claim 1, characterized in that, The centering assembly includes two bases (16), with a fixed plate (17) fixedly connected to the top of the bases (16), a cylinder (18) fixedly connected to the outside of the fixed plate (17), a baffle (19) fixedly connected to the output end of the cylinder (18), a plurality of guide rods (20) fixedly connected to the bottom of the baffle (19), and a plurality of acoustic emission sensors (21) and a plurality of temperature sensors (22) arranged on the outside of the baffle (19).

3. The plastic material handling system with visual recognition according to claim 2, characterized in that, The sensing data output by the centering component includes friction vibration signals and temperature rise data; The acoustic emission sensor (21) is used to collect the friction vibration signal, and the temperature sensor (22) is used to measure the temperature rise; The control system is configured to analyze the friction vibration signal, determine the surface acoustic texture of the object under test, and estimate the dynamic friction coefficient of the object under test based on the temperature rise.

4. The plastic material handling system with visual recognition according to claim 1, characterized in that, The sensing data output by the flipping component includes landing pressure, impact force, and internal structural acoustic signals. The sensing pad (11) is used to measure the landing pressure, the impact force sensor (12) is used to measure the impact force, and the structural acoustic sensor (13) is used to collect the internal structural acoustic signal of the object under test. The control system is further configured to determine the mass and elastic modulus of the object under test based on the impact force, determine the internal structural defects of the object under test based on the internal structural acoustic signal, and determine the center of gravity position and warping deformation state of the object under test based on the landing pressure.

5. The plastic material handling system with visual recognition according to claim 1, characterized in that, The sensory data output by the three-dimensional reconstruction camera array (15) is continuous image information, and the three-dimensional reconstruction camera array (15) is used to acquire continuous image information in three stages: Before the object under test is flipped, image information for reconstructing the initial top surface three-dimensional model of the object under test is acquired; During the flipping process of the object under test, image information is acquired for reconstructing the dynamic three-dimensional model of the object under test; After the object under test is flipped, image information for reconstructing the new top surface 3D model of the object under test is acquired; The control system is further configured to fuse the image information from the three stages to generate a complete three-dimensional model of the object under test.

6. The plastic material handling system with visual recognition according to claim 1, characterized in that, A support rod (4) is provided on the outside of the conveyor belt (1), and two cameras (5) are provided on the outside of the support rod (4). The cameras (5) are located opposite the working range of the material handling robot (3). The control system is further configured to compensate and correct the end position of the material handling robot (3) based on the image information collected by the camera (5) before the material handling robot (3) performs the grasping action.

7. A plastic material handling system with visual recognition according to claim 2, characterized in that, The baffle (19) is rotatably connected to the top of the base (16), the guide rod (20) is rotatably connected to the outside of the base (16), the baffle (19) and the object to be tested abut against each other, the base (16) is fixedly connected to the outside of the conveyor belt (1), the rotating rod (8) is rotatably connected between the two support frames (6), and the three-dimensional reconstruction camera array (15) is set on the outside of the connecting plate (14).

8. A plastic material handling system with visual recognition according to claim 2, characterized in that, The control system interacts with the upstream injection molding process, and the control system includes: A cross-domain digital twin model (23) is used to characterize the mapping relationship between the process parameters of the injection molding process and the physical properties of the object under test; The process parameter generation module (24), based on the cross-domain digital twin model (23), calculates the optimal process parameters in reverse by taking the preset ideal physical properties as the target. The process control instruction module (25) receives the optimal process parameters output by the process parameter generation module (24) and generates executable control instructions for adjusting the injection molding process.

9. A plastic material handling system with visual recognition according to claim 8, characterized in that, The physical properties include: The impact force measured by the impact sensor (12), the structural acoustic signal collected by the structural acoustic sensor (13), the friction vibration signal collected by the acoustic emission sensor (21), the dynamic friction coefficient estimated by the measurement results of the temperature sensor (22), the center of gravity position and warping deformation state characterized by the landing pressure measured by the sensing pad (11), and the complete three-dimensional model generated by the three-dimensional reconstruction camera array (15).

10. A plastic material handling system with visual recognition according to claim 8, characterized in that, The process parameter generation module (24) is configured to: when the control system determines that the physical properties of the current test object deviate from the ideal physical properties, start reverse calculation to generate the optimal process parameters that enable the physical properties of the test objects produced subsequently to return to the ideal physical properties; The process control instruction module (25) is configured to: evaluate the prediction confidence of the cross-domain digital twin model (23) before generating executable control instructions, and stop generating the executable control instructions and issue an alarm when the prediction confidence is lower than a preset threshold.

Images