MES cooperative scheduling method and system based on PLC and visual positioning

Abstract

The invention provides an MES collaborative scheduling method and system based on a PLC and visual localization, and the method comprises the steps: collecting a production material image through a dual-mode visual sensor, and generating the visual localization of a material; the PLC system receives the visual positioning, synchronously acquires the current joint state of the mechanical arm, dynamically plans an initial path by applying an optimization A * algorithm, and performs track correction through real-time PID control to obtain track data of the mechanical arm; the mechanical arm track data are input into an MES system, minimization of a busy window of a mechanical arm and high-temperature environment constraint are taken as optimization targets, a task scheduling instruction and path planning of the AGV are optimized, and AGV scheduling output is generated; aGV scheduling output is executed, meanwhile, execution process data are collected, performance deviation is analyzed, thermal risk parameters and control threshold values are periodically updated, and system self-adaptive optimization is achieved. The stability and efficiency of the whole production process are improved.

Description

Technical Field

[0001] The present invention belongs to the field of MES collaborative scheduling, and in particular relates to a MES collaborative scheduling method and system based on PLC and visual positioning. Background Art

[0002] In modern manufacturing, the prevalence of automated production lines has significantly improved production efficiency and precision. However, existing automation technologies still face many challenges in some high-precision production processes, such as material stacking, material handling, and assembly in kiln sintering lines. Traditional manual operations are often difficult to synchronize with the automated pace of modern production lines, resulting in low production efficiency and increased operational errors. For example, traditional manual placement operations are not only limited by the operator's physical strength and energy, but are also prone to problems such as material accumulation and breakage due to unstable manual operations, which directly affects the smoothness of the production line and the quality of the products. In addition, manual operations also pose a high level of safety risks, especially in environments with high temperature, high humidity, or toxic gases, where operators are easily injured or exposed to dangerous environments.

[0003] To improve production efficiency and reduce the negative impact of manual operations, many production lines are beginning to introduce robotic arms and automated systems to replace manual labor for material handling and stacking. However, existing robotic arm systems still face issues such as inaccurate positioning and inflexible path adjustment. While some systems attempt to address this issue through visual positioning technology, in practice, the accuracy and response speed of visual systems are often insufficient to cope with the rapid changes in the production process. This is especially true in high-temperature environments such as kiln sintering, where visual systems are easily affected by factors such as temperature and light fluctuations.

[0004] Furthermore, existing scheduling and material management systems often rely on manual or simple preset programs, lacking the ability to adjust production processes in real time. This results in low equipment utilization and inefficient task scheduling. While these existing technologies have improved automation to some extent, they still require significant improvement. This is particularly true in high-precision, high-security, and high-efficiency automated production environments, where existing technologies present significant limitations. Summary of the Invention

[0005] The purpose of this invention is to propose an MES collaborative scheduling method and system based on PLC and visual positioning. By introducing the collaborative work of PLC and visual positioning technology, combined with advanced sensing technology and automated scheduling system, more accurate material stacking and handling can be achieved, eliminating the unstable factors caused by manual operation.

[0006] In a first aspect, an embodiment of the present invention provides a MES collaborative scheduling method based on PLC and visual positioning, the method comprising:

[0007] S1. Capturing images of production materials through a dual-modal vision sensor to generate visual positioning of the materials; the visual positioning includes material position coordinates and posture information;

[0008] S2, the PLC system receives the visual positioning, synchronously collects the current joint state of the robot arm, applies the optimized A* algorithm to dynamically plan the initial path, and performs trajectory correction through real-time PID control to obtain robot arm trajectory data; wherein the robot arm trajectory data includes the robot arm motion trajectory and correction instructions;

[0009] S3. Input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output;

[0010] S4. Execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

[0011] Furthermore, the dual-modal vision sensor includes a high-resolution industrial RGB camera and an infrared thermal imaging camera;

[0012] Said S1 further includes:

[0013] S101: Perform image alignment and fusion preprocessing on the production material images, specifically:

[0014] Performing geometric calibration on the production material images captured by the high-resolution industrial RGB camera and the infrared thermal imaging camera, and calculating the camera extrinsic parameter matrix using a pre-calibrated checkerboard;

[0015] Apply pixel-level weighted fusion to superimpose the temperature matrix of thermal imaging onto the RGB pixels to form a fused image;

[0016] S102, performing edge detection and noise removal, applying an improved Canny edge detection algorithm to the fused image to generate a processed fused image;

[0017] S103 , calculating three-dimensional coordinates based on the processed fused image using an enhanced stereo vision method, and fitting edge point sets using a weighted least squares method to calculate posture information.

[0018] Furthermore, the S102 is specifically as follows:

[0019] First, a Gaussian filter is used to smooth the image to suppress dust noise. Second, the Sobel operator is used to calculate the gradient amplitude and direction. Third, non-maximum suppression is performed to retain the local maximum edge. Fourth, a dual threshold is used to connect the edges, and adaptive adjustment is introduced. When the local temperature gradient increases during thermal imaging, the high threshold is dynamically increased to 0.15 to retain the true edge and filter out thermal noise artifacts.

[0020] Furthermore, the S2 is specifically:

[0021] The PLC system receives the visual positioning and loads the material position coordinates and posture information into the control logic through a real-time communication protocol;

[0022] Apply the optimized A* algorithm to generate the initial path in the three-dimensional space grid;

[0023] monitoring the deviation between the visual feedback result and the initial path, performing dynamic path correction, and generating correction feedback;

[0024] According to the visual positioning, initial path and correction feedback, a smooth motion trajectory is calculated, parameterized using an improved Bezier curve, and the robot arm trajectory data is generated.

[0025] Furthermore, the synchronous acquisition of the current joint state of the robotic arm is obtained from feedback from a built-in encoder.

[0026] Furthermore, the S3 is specifically:

[0027] The MES system loads the robot arm motion trajectory and correction instructions through the database interface;

[0028] The MES system analyzes the production task queue, calculates the robot's busy window based on the robot's motion trajectory, and assigns the AGV's restocking task to a non-conflicting time period. The correction instructions are used to predict the robot's cycle time to prevent the risk of material supply interruption.

[0029] Monitor the robot arm's status and use a genetic algorithm to generate a set of candidate AGV paths in the factory map. Each path represents a grid sequence from the AGV's current position to the target task point.

[0030] Scoring the paths, calculating a comprehensive score for each path, and selecting the path with the smallest comprehensive score as the selected path;

[0031] The corresponding speed of the AGV is calculated based on the selected path to obtain a speed adjustment value and a corresponding scheduling window.

[0032] Furthermore, the paths are scored and a comprehensive score of each path is calculated, specifically:

[0033] Each path is scored based on three indicators, including: the minimum distance between the AGV and the robot trajectory, the synchronization difference between the AGV speed and the reference speed, and the thermal risk cost;

[0034] Based on the index scoring multi-objective scoring function, a comprehensive score is calculated for each path.

[0035] Furthermore, the synchronization difference is calculated based on the average estimated speed of the AGV on the current path; and the thermal risk cost is calculated based on the average value of the thermal map on the current path.

[0036] Furthermore, the S4 is specifically:

[0037] Obtain AGV scheduling output and record AGV task scheduling and path optimization results;

[0038] PLC processes collected data and calculates key performance indicators for feedback data analysis;

[0039] Adjust system parameters based on analysis results, perform periodic optimization, and generate optimized control parameters;

[0040] Based on the optimized control parameters, the loss function is minimized to generate adjustment instructions with the goal of minimizing the scheduling score deviation.

[0041] Output adjustment instructions and apply them to PLC registers.

[0042] In a second aspect, an embodiment of the present invention provides an MES collaborative scheduling system based on PLC and visual positioning, the method comprising:

[0043] A visual acquisition module is used to acquire images of production materials through a dual-modal visual sensor and generate visual positioning of the materials; the visual positioning includes material position coordinates and posture information;

[0044] A scheduling analysis module is configured to receive the visual positioning information from the PLC system, synchronously collect the current joint status of the robotic arm, dynamically plan the initial path using an optimized A* algorithm, and perform trajectory correction through real-time PID control to obtain robotic arm trajectory data; wherein the robotic arm trajectory data includes the robotic arm motion trajectory and correction instructions;

[0045] A scheduling output module is used to input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output;

[0046] The scheduling optimization module is used to execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

[0047] The beneficial technical effects of the present invention are at least as follows:

[0048] The present invention, by introducing the collaborative work of PLC and visual positioning technology, combined with advanced sensing technology and automated scheduling systems, can achieve more accurate material stacking and handling, and eliminate the unstable factors caused by manual operation. Specifically, the present invention obtains material position data from the visual system in real time, combines it with the control algorithm of the PLC system, and dynamically corrects the grasping path and operation trajectory of the robotic arm, thereby ensuring that the material can be accurately placed in the designated position throughout the production process, and the accuracy meets the required requirements. At the same time, the present invention is also combined with the intelligent scheduling function of the MES system, which can optimize the scheduling of the AGV carts in the production line based on real-time production data, ensuring the efficient flow of materials and the continuous operation of the production line. Compared with traditional automation systems, the advantage of the present invention is that it can be flexibly adjusted according to real-time data in the production process, and greatly improves the automation level and safety of the production line, reduces the impact of human errors and environmental interference, and thus improves the stability and efficiency of the entire production process. BRIEF DESCRIPTION OF THE DRAWINGS

[0049] The present invention is further described with reference to the accompanying drawings. However, the embodiments in the accompanying drawings do not constitute any limitation to the present invention. A person skilled in the art can obtain other drawings based on the following drawings without creative effort.

[0050] Figure 1 This is a flow chart of the MES collaborative scheduling method based on PLC and visual positioning of the present invention. DETAILED DESCRIPTION

[0051] The following describes embodiments of the present invention in detail. Examples of the embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended only to explain the present invention and are not to be construed as limiting the present invention.

[0052] In one embodiment, Figure 1 As shown, a MES collaborative scheduling method based on PLC and visual positioning is provided, including the following steps:

[0053] S1. Capture production material images through a dual-modal vision sensor to generate visual positioning of the material; the visual positioning includes material position coordinates and posture information.

[0054] First, a dual-modal vision sensor system is deployed to acquire material position and angle data on the production line. This system consists of a high-resolution industrial RGB camera (resolution at least 1920x1080 pixels) and an infrared thermal imaging camera (wavelength range 8-14μm, sensitivity <50mK). These sensors simultaneously capture images of the kiln sintering area at a rate of at least 15 frames per second, ensuring a panoramic view of the material stack. Designed for the high-temperature environment of the kiln sintering line, the RGB camera is mounted on a fixed bracket 2 meters above the material stack, capturing the material's outline, color, and texture characteristics in visible light. The thermal imaging camera, mounted side by side, focuses on capturing the thermal radiation distribution and temperature gradient of the hot material. This configuration uses complementary data acquisition mechanisms to address the failure of existing single-modality systems due to shape distortion caused by thermal expansion and steam obscuration, improving recognition reliability. For example, when the material temperature rises, the thermal imaging compensates for RGB image blur, ensuring a 20% improvement in overall positioning accuracy. Data is sourced from the kiln's real-time monitoring equipment and transmitted directly to the processing unit via USB or Ethernet, eliminating transmission delays that could affect subsequent calculations.

[0055] Next, image alignment and fusion preprocessing are performed. Geometric calibration is performed on the RGB and thermal imaging data. A pre-calibrated checkerboard (at least 20 images) is used to calculate the camera extrinsic parameter matrix. For example, the checkerboard is placed at the entrance of the kiln. After taking multiple angle views, the matrix is ​​iteratively calculated using the calibrateCamera function of OpenCV to ensure that the pixel alignment error is less than 0.5 pixels. This calibration improves data consistency and reduces positioning offsets caused by differences in camera positions. Subsequently, pixel-level weighted fusion is applied to convert the temperature matrix of the thermal imaging image into a pixel-level matrix. (Temperature value of each pixel, range 0-1000°C) superimposed on the RGB pixel point On the fusion image , specifically calculated as the fusion value of each pixel = ,in Emphasize visible details such as material color, It emphasizes thermal features such as temperature gradients and adds thermal distortion corrections, such as a 0.2-pixel offset when the temperature exceeds a reference value. This fusion mechanism addresses interference from kiln dust and high-temperature steam, enhancing material edge contrast. By superimposing temperature information, it reduces noise interference by 35%. Compared to existing single-modality systems, this improves image clarity at high temperatures and the accuracy of subsequent edge extraction.

[0056] Then, edge detection and noise removal are performed, and a modified Canny edge detection algorithm is applied to the fused image, which optimizes the kiln environment in four sub-steps: First, Gaussian filtering (standard deviation ) Smooth the image to suppress dust noise; secondly, use the Sobel operator to calculate the gradient amplitude and direction; third, perform non-maximum suppression to retain the local maximum edge; fourth, dual thresholds are used to connect the edges (low threshold 0.05, high threshold 0.12), and introduce adaptive adjustment. When the local temperature gradient increases during thermal imaging detection, the high threshold is dynamically increased to 0.15 to retain the true edge and filter out thermal noise artifacts. This detection improves edge continuity. For example, after processing the surface of a curved material, the edge points are connected from discontinuous to continuous. Subsequently, a median filter (kernel size 5x5 pixels) is applied to remove isolated noise points to ensure that the noise level is less than 5%. This removal mechanism improves the stability of feature extraction for the irregular surface of sintered materials and reduces the false edge rate by 15% compared to the existing Canny algorithm when applied at high temperatures.

[0057] Finally, the position and attitude are calculated, and the three-dimensional coordinates are calculated based on the processed fusion image using the enhanced stereo vision method. First, the camera intrinsic parameter matrix (focal length Pixel, principal point pixels, obtained through at least 10 calibration iterations) and depth disparity (Calculated from thermal imaging block matching algorithm, block size 21x21 pixels, matching threshold 0.8) Convert to , the specific process is to calculate the depth , then apply the formula:

[0058]

[0059] in, Taken from the inverse of the calibration matrix, As a correction for thermal expansion, is the pixel temperature, This correction compensates for thermal distortion by temperature difference, solving the depth offset problem of existing stereo vision at high temperature. , use weighted least squares method to fit the edge point set (extract at least 80 points), the formula is:

[0060]

[0061] in, Obtained from edge detection, , , As the thermal gradient regularization, Calculated from Sobel; Refers to the edge point The temperature value of the corresponding pixel, Refers to the lowest pixel temperature in the current edge point set.

[0062] S2. The PLC system receives the visual positioning, synchronously collects the current joint status of the robotic arm, applies the optimized A* algorithm to dynamically plan the initial path, and performs trajectory correction through real-time PID control to obtain robotic arm trajectory data; wherein the robotic arm trajectory data includes the robotic arm motion trajectory and correction instructions.

[0063] Among them, the optimized A* path planning algorithm (which is the optimized A path planning algorithm, a heuristic algorithm for graph search and path finding) generates an initial path on the work space grid obtained by PLC discretization; the cost function of A* is is the cumulative step cost and contains the current trajectory of the robot The safety distance penalty term, is a geometric heuristic to the target and can be weighted by the environmental cost; the initial path is used as the input for the same-period PID trajectory correction.

[0064] Specifically, this step transforms the visual positioning output from step 1 into a real-time robot arm trajectory through a dynamic path planning and correction mechanism, ensuring precise material grabbing and placement in high-temperature environments on the kiln sintering line. This mechanism addresses the inflexibility and error accumulation issues of existing path planning systems in dynamic, high-temperature scenarios. By deeply integrating material position coordinates (x, y, z) and posture information (angle θ) into the PLC algorithm, it improves the robot arm's operational accuracy and provides optimized instructions for MES scheduling in step 3, thereby enhancing overall production continuity and safety and overcoming the limitations of traditional static systems.

[0065] First, the PLC system receives the output data from step 1 and loads the material's position coordinates (x, y, z) and posture information (angle θ) into the control logic via a real-time communication protocol (such as Ethernet / IP). This reception process ensures that the data update frequency is synchronized with visual capture, avoiding positioning drift caused by latency. In kiln sintering scenarios, this process is designed to address high-temperature interference, such as using shielded cables to transmit data to resist electromagnetic noise. The coordinates are immediately set as the trajectory target endpoint, and the posture is used as a rotation reference for the end effector, laying the foundation for initial path calculation and improving the system's response stability in thermal environments.

[0066] Next, the initial path calculation is performed. The PLC combines the current joint states of the robot arm (obtained from built-in encoder feedback, such as reading joint angles via position sensors) with the input from step 1 and applies an optimized A* algorithm to generate a preliminary trajectory within a three-dimensional spatial grid (resolution 0.005m). This algorithm evaluates costs, including Euclidean distance and kiln-specific thermal risks. For example, it calculates the node sequence from the starting point to (x, y, z), prioritizing avoidance of areas with temperatures > 600°C. This is achieved through a pre-set weighted cost function for the thermal map. Simultaneously, posture information θ is integrated into the end-effector guidance, for example, by adjusting the joint rotation at the end of the trajectory to match θ to avoid grasping angle deviation. This integrated mechanism improves the applicability of the initial path to the irregular surfaces of sintered materials and reduces the risk of collision in high-temperature areas by 15% compared to existing static planning.

[0067] Next, dynamic path correction is performed, monitoring deviations from the initial trajectory using visual feedback. For example, by comparing the current coordinates with (x, y, z), a micro-shift of 0.01m is calculated. The PLC then activates the proportional-integral-derivative (PID) controller to adjust the trajectory. The PID parameters (Kp=2.0, Ki=0.3, Kd=0.6) are tuned for kiln vibration and heat flow. For example, Kp controls the proportional response to quickly correct deviations, the Ki integral term eliminates steady-state errors, and the Kd derivative term suppresses overshoot and ensures a smooth response. The posture θ is used as a rotation compensation factor, for example, by calculating the joint torque increment θ*a constant factor and dynamically applying it to the controller output. This correction mechanism, which iterates every 0.03 seconds to address thermal dynamics, reduces accumulated errors through a feedback loop, improving the path stability of the existing system at high temperatures.

[0068] Finally, the final optimized trajectory and instructions are generated. The initial path, correction feedback, and input data are integrated to calculate the smooth motion trajectory. This is parameterized using a modified Bezier curve, for example, generating a sequence of curve points from the starting point P_0 to the target (x, y, z). The specific calculation process is as follows: First, control points P_1 and P_2 are determined as intermediate nodes of the A* path, for example, P_1 is set to the coordinates of the 1 / 3 of the path; then the parameter t is incremented from 0 to 1, and each P(t) value is calculated in steps of 0.01; the formula is applied:

[0069]

[0070] in, Output path coordinates (unit: m), Get the starting point from the PLC, and From A* calculation, For step 1, As thermal index compensation, it reduces the impact of high temperature deformation through the exponential decay mechanism. Regularized posture deviations prevent overfitting. Corrective instructions are also generated based on trajectory deviations. For example, velocity v = 0.8 m / s is calculated using PID output, and the joint angle Δq is used as the torque increment. This is then translated into signals in the PLC ladder logic to ensure matching with the kiln cycle and improve practical efficiency. The generated robot arm trajectory P(t) and correction instructions (including velocity v) are derived from the Bessel formula and PID adjustment. The trajectory is calculated using a curve formula, and the correction instructions are generated through deviation feedback and directly passed to step S3.

[0071] S3. Input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output.

[0072] Among them, high temperature environment constraints refer to the Two-level restrictions: When the corresponding grid cost is increased by the weight increment, Marked as prohibited.

[0073] In this step, the MES system is used to optimize the scheduling of the AGV so that it can coordinate with the robot arm's operation path and achieve efficient material transportation. The system input is the robot arm trajectory function output by the PLC module in step 2. and the corresponding speed command , the output is the AGV path , speed adjustment command and scheduling path scoring , used to perform collaborative transportation tasks.

[0074] First, the MES system receives the robot arm trajectory function from the PLC through a database interface (such as ODBC) , indicating that the end of the robotic arm is Spatial position under time, and speed reference , which is the average moving speed of the expected trajectory calculated by the PLC. The trajectory data is redundantly managed through the cache queue to reduce the synchronization error caused by environmental interference and is mapped into time periods. The "robotic arm occupation window" in the controller is used by the scheduling module.

[0075] MES system reads production task queue , issued by ERP, including the material type, destination site and priority label of the task to be completed. , the system calculates the time required for the robot arm task , as a path avoidance window. By the length of the track and speed The ratio of is the trajectory length, estimated by the following formula:

[0076]

[0077] in, Represents trajectory The time derivative of corresponds to the real-time velocity vector of the manipulator. This integral is used to estimate the total motion path length of the manipulator in the current task.

[0078] Will and Combine and estimate the robot arm task occupancy window:

[0079]

[0080] in, is the trajectory reference speed output in step 2, It is used as a scheduling prohibited area for subsequent AGV path scheduling.

[0081] Scheduling module access to AGV location , provided by the navigation system, combined with the factory map (Grid map) and heat risk map , build the path search space. Generate the AGV path candidate set using genetic algorithm , each path Represents a grid sequence from the current position of the AGV to the target task point. Each path will be scored according to three types of indicators:

[0082] and robotic arm trajectory The minimum distance ;

[0083] With reference speed The synchronization difference ,in For path The average estimated speed of

[0084] Heat risk cost , is the path Average value on heat map.

[0085] Based on the above indicators, the system constructs a multi-objective scoring function for each path. Calculate its comprehensive score :

[0086]

[0087] in, Indicates the path The shortest distance between the robot arm trajectory and the path point The minimum Euclidean distance of the trajectory point set is calculated; Rhythm matching weight, used to adjust the weight of speed synchronization item in the overall score; is the thermal regularization term coefficient, which is used to suppress the high thermal path priority; For path The average of the heat map values ​​passed above, the heat map Derived from the workshop temperature sensor array and generated using interpolation filling.

[0088] The system selects the path with the lowest score:

[0089]

[0090] Selected Path The corresponding speed is , the system calculates the speed adjustment value:

[0091]

[0092] Output include paths Coordinate sequence and speed command and scheduling windows ,in It is the redundant time constant for system safety.

[0093] This mechanism realizes the dynamic coordination of the robot arm path and AGV scheduling in three dimensions: space, rhythm and heat load, significantly improving the stability and safety of material scheduling in the kiln environment.

[0094] S4. Execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

[0095] This step converts the AGV scheduling output of step 3 into long-term system parameter tuning through feedback optimization and periodic adjustment mechanism, ensuring that the kiln sintering line maintains equipment adaptability and stability under high temperature environment. This mechanism solves the performance degradation, error accumulation and efficiency reduction problems of the existing system under continuous high temperature operation. By combining the scheduling instruction S and the path optimization result (speed adjustment ) is integrated with historical production data and looped back to the aforementioned steps to improve overall robustness and sustainability and overcome the limitations of traditional static adjustment.

[0096] Step 3 provides the AGV scheduling instructions S (including path A) and path optimization results (speed adjustment ), these data are directly derived from the output of MES scheduling and are used as the feedback basis for this step, ensuring that scheduling instructions as performance benchmarks and speed adjustments as deviation indicators are all applied to the analysis and optimization process.

[0097] First, the PLC system collects the output data of step 3 and records the AGV scheduling instructions S and path optimization results (speed adjustment ), for example, set path A as the actual execution trajectory reference, As a speed deviation indicator, it also aggregates environmental data such as temperature changes during the production cycle and stores them in an embedded database on an hourly basis to ensure data integrity. This collection process targets equipment vibration and data noise caused by high temperature of the kiln, and records the value of S and Adjust the amplitude to avoid loss caused by thermal interference, improve the reliability of historical sequences, and increase data availability compared to existing log systems.

[0098] Next, feedback data analysis is performed, PLC processes the collected data, and calculates key performance indicators. For example, the deviation between the actual path of the AGV and instruction A is obtained by summing the point-by-point coordinate differences to obtain the total error value, and the speed execution error is obtained by The average deviation is calculated from the successive comparisons with the target v and the thermal influence is evaluated in conjunction with the furnace temperature log, e.g. when When the deviation exceeds a set threshold, the path segment temperature record is traced back to quantify the impact. This analysis mechanism targets thermal gradient characteristics and runs once per cycle, smoothing the deviation curve using a moving average method. For example, the average value is calculated using a 5-point window. This ensures that operators can manually verify the results, avoiding complex calculations, improving analysis accuracy, and addressing the inaccuracy of existing feedback at high temperatures.

[0099] Then, periodic optimization is performed to adjust system parameters based on the analysis results. For example, the thermal risk regularization coefficient in the MES scheduling formula is updated, and the weight of the high-temperature path is dynamically reduced by comparing the S score with the actual throughput (for example, throughput = number of completed tasks / cycle time). This optimization mechanism targets the long-term heat accumulation of the kiln and achieves gradual adaptation through adaptive thresholds. For example, if three consecutive cycles If the average deviation is >0.2m / s, the weight of the hot zone of path A is reduced to ensure that the system gradually adapts to environmental changes, improves environmental adaptability, and reduces cumulative error accumulation.

[0100] Finally, the adjustment instructions are generated, and the optimized control parameters are calculated based on the feedback data, using the minimization loss function. The specific calculation process is as follows: initialize the loss term; calculate the scheduling score deviation ∑|S-S_a| by summing the absolute difference between the log S and the actual S_a (obtained from the execution record); add the speed term ϕ*( )^2 by squaring The value is multiplied by the coefficient; the thermal term ψ*∫Tdt is integrated, and the temperature T series accumulation within the period is calculated by the numerical summation method (for example, the sum of T values ​​sampled at 1 minute intervals is calculated). The formula is:

[0101]

[0102] in, Output updated parameters such as thermal coefficient, Sum the deviations (S from step 3, S_a from the log), The nonlinear penalty is ensured by the square term, is the square of the velocity (input in step 3, unit ))、 The cumulative effect of temperature T (T from sensor, in °C) is quantified by the integral regularization, This function takes into account the effect of continuous high temperature by optimizing the thermal integral term, improving parameter stability and solving the problem that the existing loss function ignores the cumulative effect.

[0103] At the same time, the output instruction applies O to the PLC register such as the path threshold update. The whole process is executed at the end of the cycle with a response time of <1 minute, ensuring practical integration.

[0104] In one embodiment, a MES collaborative scheduling system based on PLC and visual positioning is provided, wherein the method includes:

[0105] A visual acquisition module is used to acquire images of production materials through a dual-modal visual sensor and generate visual positioning of the materials; the visual positioning includes material position coordinates and posture information;

[0106] A scheduling analysis module is configured to receive the visual positioning information from the PLC system, synchronously collect the current joint status of the robotic arm, dynamically plan the initial path using an optimized A* algorithm, and perform trajectory correction through real-time PID control to obtain robotic arm trajectory data; wherein the robotic arm trajectory data includes the robotic arm motion trajectory and correction instructions;

[0107] A scheduling output module is used to input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output;

[0108] The scheduling optimization module is used to execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

[0109] Unless otherwise specifically stated, the relative steps, numerical expressions and values ​​of the components and steps set forth in these embodiments do not limit the scope of the present application.

[0110] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a non-volatile computer-readable storage medium that is executable by a processor. Based on this understanding, the technical solution of the present application, or the part that contributes to the prior art, or the part of the technical solution, can be embodied in the form of a software product. The computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the system described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program code, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, or an optical disk.

[0111] In the description of this application, it should be noted that the terms "upper" and "lower" etc. indicate orientations or positional relationships based on the orientations or positional relationships shown in the accompanying drawings, or the orientations or positional relationships in which the invented product is usually placed when in use. These are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, they should not be understood as limitations on this application.

[0112] It should also be noted that, in the description of this application, unless otherwise expressly specified or limited, the terms "disposed," "installed," "connected," and "connected" should be understood in a broad sense. For example, they can refer to fixed connections, detachable connections, or integral connections; they can refer to mechanical connections or electrical connections; they can refer to direct connections or indirect connections through an intermediate medium; and they can refer to internal connections between two components. Those skilled in the art will understand the specific meanings of the above terms in this application based on the specific circumstances.

[0113] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some or all of the technical features therein. These modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. The MES collaborative scheduling method based on PLC and visual positioning is characterized by: The method comprises: S1. Capturing images of production materials through a dual-modal vision sensor to generate visual positioning of the materials; the visual positioning includes material position coordinates and posture information; S2, the PLC system receives the visual positioning, synchronously collects the current joint state of the robot arm, applies the optimized A* algorithm to dynamically plan the initial path, and performs trajectory correction through real-time PID control to obtain robot arm trajectory data; wherein the robot arm trajectory data includes the robot arm motion trajectory and correction instructions; S3. Input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output; S4. Execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

2. The MES collaborative scheduling method based on PLC and visual positioning according to claim 1 is characterized in that: The dual-modal vision sensor includes a high-resolution industrial RGB camera and an infrared thermal imaging camera; Said S1 further includes: S101: Perform image alignment and fusion preprocessing on the production material images, specifically: Performing geometric calibration on the production material images captured by the high-resolution industrial RGB camera and the infrared thermal imaging camera, and calculating the camera extrinsic parameter matrix using a pre-calibrated checkerboard; Apply pixel-level weighted fusion to superimpose the temperature matrix of thermal imaging onto the RGB pixels to form a fused image; S102, performing edge detection and noise removal, applying an improved Canny edge detection algorithm to the fused image to generate a processed fused image; S103 , calculating three-dimensional coordinates based on the processed fused image using an enhanced stereo vision method, and fitting edge point sets using a weighted least squares method to calculate posture information.

3. The MES collaborative scheduling method based on PLC and visual positioning according to claim 2 is characterized in that: The S102 is specifically as follows: First, use Gaussian filtering to smooth the image to suppress dust noise; second, use the Sobel operator to calculate the gradient magnitude and direction; third, perform non-maximum suppression to retain the local maximum edge; Fourth, dual thresholds are used to connect edges, and adaptive adjustment is introduced. When the thermal imaging detects an increase in the local temperature gradient, the high threshold is dynamically increased to 0.15 to retain the true edges and filter the thermal noise artifacts.

4. The MES collaborative scheduling method based on PLC and visual positioning according to claim 1 is characterized in that: Said S2 is specifically: The PLC system receives the visual positioning and loads the material position coordinates and posture information into the control logic through a real-time communication protocol; Apply the optimized A* algorithm to generate the initial path in the three-dimensional space grid; monitoring the deviation between the visual feedback result and the initial path, performing dynamic path correction, and generating correction feedback; According to the visual positioning, initial path and correction feedback, a smooth motion trajectory is calculated, parameterized using an improved Bezier curve, and the robot arm trajectory data is generated.

5. The MES collaborative scheduling method based on PLC and visual positioning according to claim 4 is characterized in that: The current joint state of the robotic arm is obtained from the built-in encoder feedback.

6. The MES collaborative scheduling method based on PLC and visual positioning according to claim 1 is characterized in that: The S3 is specifically: The MES system loads the robot arm motion trajectory and correction instructions through the database interface; The MES system analyzes the production task queue, calculates the robot's busy window based on the robot's motion trajectory, and assigns the AGV's restocking task to a non-conflicting time period. The correction instructions are used to predict the robot's cycle time to prevent the risk of material supply interruption. Monitor the robot arm's status and use a genetic algorithm to generate a set of candidate AGV paths in the factory map. Each path represents a grid sequence from the AGV's current position to the target task point. Scoring the paths, calculating a comprehensive score for each path, and selecting the path with the smallest comprehensive score as the selected path; The corresponding speed of the AGV is calculated based on the selected path to obtain a speed adjustment value and a corresponding scheduling window.

7. The MES collaborative scheduling method based on PLC and visual positioning according to claim 6 is characterized in that: The paths are scored and the comprehensive score of each path is calculated, specifically: Each path is scored based on three indicators, including: the minimum distance between the AGV and the robot trajectory, the synchronization difference between the AGV speed and the reference speed, and the thermal risk cost; Based on the index scoring multi-objective scoring function, a comprehensive score is calculated for each path.

8. The MES collaborative scheduling method based on PLC and visual positioning according to claim 7 is characterized in that: The synchronization difference is calculated based on the average estimated speed of the AGV on the current path; and the thermal risk cost is calculated based on the average value of the thermal map on the current path.

9. The MES collaborative scheduling method based on PLC and visual positioning according to claim 1 is characterized in that: Said S4 is specifically: Obtain AGV scheduling output and record AGV task scheduling and path optimization results; PLC processes collected data and calculates key performance indicators for feedback data analysis; Adjust system parameters based on analysis results, perform periodic optimization, and generate optimized control parameters; Based on the optimized control parameters, the loss function is minimized to generate adjustment instructions with the goal of minimizing the scheduling score deviation. Output adjustment instructions and apply them to PLC registers.

10. The MES collaborative scheduling system based on PLC and visual positioning is characterized by: The system comprises: A visual acquisition module is used to acquire images of production materials through a dual-modal visual sensor and generate visual positioning of the materials; the visual positioning includes material position coordinates and posture information; A scheduling analysis module is configured to receive the visual positioning information from the PLC system, synchronously collect the current joint status of the robotic arm, dynamically plan the initial path using an optimized A* algorithm, and perform trajectory correction through real-time PID control to obtain robotic arm trajectory data; wherein the robotic arm trajectory data includes the robotic arm motion trajectory and correction instructions; A scheduling output module is used to input the robot arm trajectory data into the MES system, optimize the AGV task scheduling instructions and path planning with the optimization goal of minimizing the robot arm busy window and high temperature environment constraints, and generate AGV scheduling output; The scheduling optimization module is used to execute the AGV scheduling output, collect execution process data and analyze performance deviations, and periodically update thermal risk parameters and control thresholds to achieve system adaptive optimization.

Images