PC-based integrated vision-motion logic control method, device and storage medium

Abstract

This invention discloses a PC-based integrated vision-motion logic control method, device, and storage medium, relating to the field of industrial automation control technology. The method includes the following steps: establishing a three-buffer ring array and a global atomic index in the physical memory of a multi-core PC; utilizing CPU atomic instructions to achieve data interaction between the vision process and the PLC core; the PLC periodically comparing frame tokens, and when visual data lag is detected, automatically entering inertia compensation mode to generate virtual target coordinates and calculate inertial drift to ensure motion continuity; real-time calculation of crash warning values ​​and jamming risk indicators, based on which a multi-level linkage multi-dimensional adjustment algorithm dynamically adjusts the PLC control cycle and process parameters to achieve proactive defense and adaptive adjustment of the system. This invention solves the problem of control cycle breakage and discontinuous mechanical motion oscillation caused by significant fluctuations in the visual recognition time of non-standard agricultural products.

Description

Technical Field

[0001] This invention relates to the field of industrial automation control technology, specifically to a PC-based integrated visual-motion logic control method, device, and storage medium. Background Technology

[0002] With the widespread application of machine vision and robotics, many intelligent equipment and production lines now feature multiple physical controllers operating simultaneously. This not only increases equipment costs but also complicates and inefficients collaboration between controllers. The emergence of various open-source software based on PC technology offers an opportunity to integrate multiple controllers. However, in traditional industrial automation control systems, visual inspection, motion control, and logic control are typically implemented by independent devices, such as industrial PCs, vision controllers, and PLC controllers, combined through communication protocols. This approach suffers from drawbacks including complex system integration, poor real-time performance, high cost, and low resource utilization.

[0003] Existing technologies typically employ distributed network control schemes to address this issue. The mainstream approach involves building a subordinate architecture with an IPC (Industrial PC) as the host computer and a PLC (Programmable Logic Controller) or motion controller as the slave computer, interconnecting the physical units via industrial Ethernet or fieldbus. In this architecture, the IPC processes the computationally demanding image data and encapsulates the recognition results into communication protocol messages, which are then sent to the PLC. After parsing the messages, the PLC generates pulses or bus commands based on preset logic to drive the actuators. Additionally, some solutions integrate an industrial PC with a motion control board, attempting to shorten the physical communication path and improve the ease of data exchange to some extent.

[0004] Although existing technologies are widely used in industrial applications, they still have the following limitations when dealing with high-speed vision-motion synchronization scenarios at the sub-millisecond level: First, when the vision system transmits large amounts of 3D pose or point cloud data to the control system, existing solutions lack hardware-level atomic operation protection. This makes them highly susceptible to data tearing due to read / write timing conflicts between the non-real-time system and the real-time kernel, leading to the control end obtaining incorrect motion coordinates and causing overcutting of the grinding head or mechanical collisions. Second, cross-device data interaction is limited by the encapsulation and parsing process of the network protocol stack, introducing non-negligible communication latency and jitter. Furthermore, traditional PLCs use fixed scanning cycles and cannot perceive the fluctuations in computation time caused by upstream vision algorithms processing complex defects (such as irregular cracks), resulting in a mismatch between control cycle and vision refresh rate, causing discontinuous oscillations in mechanical movements. Traditional PLC logic is usually based on deterministic binary logic (normal / alarm) design, lacking the ability to quantitatively perceive system scheduling jitter and algorithm performance degradation trends. When visual data lags or the system load is too high, the controller cannot dynamically adjust the control cycle or process parameters (such as cutting depth) and can only passively wait for communication timeout or execute an emergency stop.

[0005] Therefore, the present invention provides a PC-based integrated visual-motion logic control method, device and storage medium. Summary of the Invention

[0006] The purpose of this invention is to provide a PC-based integrated vision and motion logic control method to solve the existing problems mentioned in the background art.

[0007] To achieve the above objectives, the present invention provides the following technical solution: a PC-based integrated vision-motion logic control method, comprising the following steps: S1. Construct a three-buffered ring array and a global atomic index in the physical memory of a multi-core PC as a data interaction channel between the vision process running on a non-real-time operating system and the PLC logic running on a real-time control kernel. S2. The vision process writes visual pose data and frame tokens to the free slots in the three buffers, and updates the global atomic index using CPU atomic instructions; S3. The real-time control kernel periodically reads the target slot pointed to by the global atomic index and compares the current frame token with the last execution token cached locally. S4. When the frame token is not updated, the real-time control kernel enters the inertial compensation mode, generates virtual target coordinates based on the motion state of the previous cycle, and synchronously accumulates and calculates the cumulative amount of inertial drift. S5. The real-time control kernel calculates the crash warning value representing system scheduling jitter and the dead risk index representing algorithm performance degradation in real time based on the historical interaction data of shared memory. S6. Based on the output of step S5, the real-time control kernel creates a multi-level linkage and multi-dimensional adjustment algorithm to dynamically adjust the PLC control cycle and the process parameters of the end effector.

[0008] A further improvement of the present invention is that, in step S2, the vision process first reads the global atomic index and identifies the active slot currently occupied by the real-time control kernel; selects an inactive slot from the three-buffer ring array as a write target point and writes the data packet to be loaded, including the 6-DOF pose, computation time, and auto-incrementing frame token; then compares and exchanges atomic instructions, and modifies the value of the global atomic index to the index value of the write target point within a single instruction cycle.

[0009] A further improvement of this invention is that, when the current frame token in step S3 is greater than the execution token in the local cache, the execution token in the local cache is updated to the current frame token, and the accumulated inertial drift is also increased. Zero out the current slot and use the visual pose data in the current slot for motion planning. When the current frame token equals the locally cached execution token, visual data lag is determined, the locally cached execution token remains unchanged, and the end-of-cycle instruction speed of the previous control cycle is obtained. and acceleration Using kinematic formulas Calculate the virtual target coordinates for the current cycle to drive the servo mechanism to maintain smooth motion; simultaneously, update the accumulated inertial drift. The string is kept cleared until a new frame token is received. This represents the amount of motion accumulated by the robot while it is blindly walking during the period when its vision is not updated. For the current control cycle, These are the position coordinates of the previous cycle.

[0010] A further improvement of this invention is that the multi-level linkage multi-dimensional adjustment algorithm includes: Level 1 adjustment is configured to use a dynamic computing power balancing strategy, based on the aforementioned crash warning value. Adjust the control cycle to smooth out system scheduling jitter; The secondary adjustment is configured to utilize a process flexibility degradation strategy based on the accumulated inertial drift. Adjust process parameters to maintain safe operation during visual lag; Level 3 adjustment: Configured to use a preemptive self-healing strategy based on the aforementioned deadlock risk indicators. Execute the algorithm reset to avoid system crashes; The response sensitivity of the secondary regulation is corrected by the state parameters of the first-level regulation, and the triggering of the tertiary regulation has the highest priority. When triggered, the first-level and second-level regulation are forcibly linked to enter a safe holding state.

[0011] A further improvement of this invention is that the computing power dynamic balance adjustment strategy sets a basic control cycle. and safety jitter threshold Implementation, specifically including: When the crash warning value Less than or equal to the safe jitter threshold At the same time, maintain the target control cycle. Equal to the basic control cycle ; When the crash warning value Greater than the safe jitter threshold At the same time, the target control cycle is calculated in real time. ,in The proportional gain coefficient; the real-time control kernel dynamically extends its bus scan cycle to the target control cycle. By actively reducing the CPU time slice usage, a computing power recovery window is provided for non-real-time vision processes.

[0012] A further improvement of this invention is that the process flexibility degradation strategy includes defining a process parameter attenuation model. ,in To set parameters, For actual execution parameters, This is the attenuation coefficient; subsequently, the collapse warning value is... The attenuation coefficient is introduced as a correction factor. Construct dynamic attenuation coefficient ,in This is a weighted linkage.

[0013] A further improvement of this invention is that the preemptive self-healing strategy has the right to globally block the linkage of preceding strategies, specifically manifested as follows: When the aforementioned jamming danger indicator When the slope exceeds a preset threshold, the visual algorithm is determined to be about to deadlock. The real-time control kernel immediately writes a forced reset instruction to the shared memory and locks the dynamic balance adjustment strategy for computing power, forcing the target control cycle. Maintain at the maximum allowable value; simultaneously cover the process flexible degradation strategy, forcibly accumulating the inertial drift amount. Set to infinity, directly drive the end effector to enter a safe pose of full lift or return to zero until a new frame token is received after the vision process restarts. Then, release the above lock and cover, and restore normal closed-loop control.

[0014] A further improvement of this invention is that the visual process executes a deep learning defect recognition algorithm based on RGB-D data acquired by a 3D structured light camera; and in step S2, a data packet with three buffers is written, specifically including the three-dimensional coordinates of the identified defect, the defect type code, and the computation time data of this algorithm processing; thus, the jamming risk indicator... Read visual computation time data from multiple consecutive frames in shared memory and calculate its slope over time; The crash warning value By extracting the jitter of the time interval when the data packet is written, a sliding window is constructed to record the time interval sequence of the most recent N frame token updates, and the standard deviation of the updated time interval sequence is calculated and normalized.

[0015] An electronic device includes a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the aforementioned PC-based integrated vision-motion logic control method.

[0016] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned PC-based integrated vision-motion logic control method.

[0017] Compared with the prior art, the beneficial effects of the present invention are: 1. This invention first solves the problem of data tearing and resource contention between non-real-time vision processes and real-time control kernels under high-frequency concurrent read and write timing by constructing a three-buffer ring array in physical memory and cooperating with a global atomic index switching mechanism. It realizes low-latency, atomic-level integrity transmission of large-capacity 3D vision data across heterogeneous operating systems under the condition of no mutex lock.

[0018] 2. By using an inertial compensation mode based on time token comparison and a second-order kinematic virtual target generation algorithm, the problems of control cycle breakage and discontinuous oscillation of mechanical movements caused by large fluctuations in the visual recognition time of non-standard agricultural products are solved. It enables precise motion control of the robot end effector to maintain a smooth trajectory based on physical inertia during blind periods when visual data is delayed or communication jitter occurs.

[0019] 3. By calculating and characterizing the system scheduling jitter collapse warning value and the algorithm degradation deadlock risk index in real time, and constructing a multi-level linkage adaptive adjustment model that includes dynamic transfer of computing power, flexible process degradation and preemptive self-healing, the problem of blind execution or passive shutdown caused by the inability of traditional PLC logic to perceive the upstream computing status is solved. It realizes flexible degradation operation of equipment in sub-health state and automatic reset in milliseconds before deadlock, which significantly improves the product yield of the production line. Attached Figure Description

[0020] Figure 1 This is a flowchart of the PC-based integrated vision and motion logic control method of the present invention. Detailed Implementation

[0021] The technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments. It should be understood that the embodiments of the present invention and the specific features in the embodiments are detailed descriptions of the technical solution of the present invention, rather than limitations thereof. In the absence of conflict, the embodiments of the present invention and the technical features in the embodiments can be combined with each other.

[0022] The term "and / or" is merely a description of the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three cases: A exists alone, A and B exist simultaneously, and B exists alone.

[0023] Example 1 Figure 1The flowchart of the PC-based integrated view-motion logic control method disclosed in this embodiment is shown, and the steps are as follows: Step S1: Construct a three-buffered ring array and a global atomic index in the physical memory of a multi-core PC as a data interaction channel between the vision process running on a non-real-time operating system and the PLC logic running on a real-time control kernel. Step S2: The vision process writes visual pose data and frame tokens to the free slots in the three buffers, and updates the global atomic index using CPU atomic instructions to resolve the data tearing problem caused by read-write conflicts. Step S3: The real-time control kernel periodically reads the target slot pointed to by the global atomic index and compares the current frame token with the last execution token in the local cache. Step S4: When it is determined that the frame token has not been updated, the real-time control kernel enters the inertial compensation mode, generates virtual target coordinates based on the motion state of the previous cycle, and synchronously accumulates and calculates the cumulative amount of inertial drift. Step S5: The real-time control kernel calculates in real time the crash warning value representing system scheduling jitter and the deadlock risk index representing algorithm performance degradation based on the historical interaction data of shared memory. Step S6: Based on the output of step S5, the real-time control kernel creates a multi-level linkage and multi-dimensional adjustment algorithm to dynamically adjust the PLC control cycle and the process parameters of the end effector, so as to overcome the lag and binary nature of traditional PLC logic.

[0024] This embodiment discloses a PC-based integrated vision-motion-logic control method, specifically applied to an automated potato defect removal device. It utilizes a single multi-core computer to simultaneously perform 3D visual defect recognition, logical judgment, and motion control of a multi-axis robot. At the hardware architecture level, traditional potato defect removal production lines typically employ a stacked hardware configuration of an industrial PC, PLC, and motion control card. This architecture is costly, and the communication latency between devices via TCP / IP or Modbus is in the millisecond range, making it difficult to meet the real-time requirements of high-speed cutting, such as processing multiple potatoes per second. Therefore, this embodiment selects a central processing unit with four or more processing cores. Through the basic input / output system or operating system startup parameters, a hardware storage area, invisible to the operating system kernel and non-pageable, is reserved in the address space of random access memory. The preferred size of this area is 16 megabytes, used to construct a shared communication channel between the non-real-time domain and the real-time domain. The non-real-time visual processes running the Windows operating system are bound to the zeroth and first cores of the central processing unit (CPU), while the logic control processes running the real-time control kernel are bound to the second and third cores of the CPU. Kernel patches are used to block interrupt requests from the non-real-time system to the real-time core. For example, by modifying the Windows HAL (Hardware Abstraction Layer) or using real-time extensions such as RTX and the real-time drivers that come with the EtherCATMaster Stack, core exclusivity and interrupt routing redirection are achieved, thereby realizing hardware-level isolation of computing resources.

[0025] In the process of establishing communication links and data interaction, to overcome the transmission delays and data inconsistencies caused by shared memory read / write conflicts in traditional network communication, this embodiment constructs a three-buffer ring array and a global atomic index in the reserved hardware storage area. This array contains three identical data slots, labeled Slot A, Slot B, and Slot C, respectively. The global atomic index is an integer variable whose value points only to the index of the slot currently containing the latest complete data. During the data writing phase, the non-real-time vision process first reads the global atomic index to identify the active slot currently occupied by the real-time control kernel, and then selects one of the remaining two inactive slots as the writing target.

[0026] The processing time for standard industrial parts is usually fixed, but potatoes have extremely irregular shapes and defects. Processing a smooth potato might only take 20ms, while processing a potato full of dirt and cracks might take 50ms. Therefore, the large fluctuations in processing time during potato waste removal can directly lead to subsequent calculations of jamming risk indicators. Therefore, after the vision process acquires the 3D point cloud and RGB images of the potato surface using a 3D structured light camera, a deep learning algorithm is used to identify defects such as insect damage, greening, or cracks. The six-degree-of-freedom pose data of the defect center in the robot's base coordinate system is calculated, and this pose data, the actual processing time of the algorithm, and a monotonically increasing frame token are encapsulated and written to the selected writing target point. After writing, the vision process calls the interlocked atomic instruction of the central processing unit to modify the value of the global atomic index to the index value of the writing target point within a single instruction cycle. This design principle utilizes the atomicity of the central processing unit hardware level to ensure that the index switching action cannot be interrupted, thereby ensuring that the real-time control kernel reads a complete frame of data at any time, eliminating the risk of data tearing.

[0027] In the synchronization and logic processing phase of motion control, the real-time control kernel undertakes the core scheduling function. To address the issue of control cycle breaks caused by fluctuations in processing time in non-real-time vision systems, the real-time control kernel uses a fixed basic control cycle. (Preferably 250 microseconds) Periodically read the target slot pointed to by the global atomic index and obtain the current frame token therein. .

[0028] The real-time control kernel periodically reads the target slot pointed to by the global atomic index and compares the current frame token with the last execution token cached locally. The frame token is defined as an auto-incrementing frame token. This means that the kernel executes this process every time the vision process completes a calculation and writes new data to shared memory (slot). The operation involves tokens that are strictly monotonically increasing over time, as the visual process is the sole data producer. If the comparison shows that the current frame token is greater than the locally cached execution token, the system will update the locally cached execution token to the current frame token and will also include the accumulated inertial drift mentioned later. The system clears the cache and uses the visual pose data in the current slot for normal motion planning. However, when it determines that the frame token has not been updated (i.e., the current frame token equals the locally cached execution token), the system determines that the visual data is lagging. In this case, motion should not stop, but rather, as described in step S4, the real-time control kernel enters inertial compensation mode. In this mode, the system keeps the locally cached execution token unchanged and generates virtual target coordinates based on the motion state of the previous control cycle. Specifically: it obtains the end-of-line command velocity of the previous control cycle. and acceleration Using kinematic formulas Calculate the virtual target coordinates for the current cycle to drive the servo mechanism to maintain smooth motion. In this formula... For virtual target coordinates, The initial coordinates (position at the previous moment). For the current control cycle, an acceleration term is introduced. The design principle lies in using Taylor series expansion to achieve second-order position interpolation, ensuring that the robot's trajectory remains second-order continuous within the visual blind zone, thus avoiding mechanical vibrations caused by sudden velocity changes due to uneven potato surfaces. Simultaneously, the system synchronously accumulates and calculates the cumulative inertial drift, updating the formula as follows: This process continues until a new frame token is received, at which point the token is reset to zero. This parameter represents the amount of motion accumulated by the robot while it is blindly walking during the period when the vision is not updated. It intuitively quantifies the degree of risk that the current control action deviates from the actual visual feedback.

[0029] To enable the potato waste removal control system to sense its own state and make adaptive adjustments, a multi-dimensional feature analysis mechanism is introduced. The real-time control kernel, based on historical interaction data from shared memory, calculates in real-time a crash warning value characterizing system scheduling jitter. And indicators that characterize the risk of algorithm performance degradation and crashing. The specific calculation process is as follows: For the risk of being stuck The system reads visual computation time data from shared memory for multiple consecutive frames. The data is written by the vision process in step S2, and its slope over time is calculated. If the slope remains positive, it indicates that the algorithm's time consumption is increasing, suggesting a risk of memory leaks or computational backlog.

[0030] For crash warning values The system extracts the jitter of the time interval when the data packets are written and constructs a sliding window to record the time interval sequence of the most recent N frame token updates. (N is preferably 50), let the average value of this sequence be... The standard deviation of the updated time interval sequence is calculated and normalized to obtain the crash warning value. The calculation formula is as follows: ,in This is the normalization coefficient, and its value is chosen to map the standard deviation to between 0 and 1 within the normal fluctuation range, preferably 0.1. The higher this value, the more unstable the scheduling of the non-real-time operating system is, and the greater the probability of system crash.

[0031] Based on the aforementioned characteristic indicators, this embodiment creates a multi-level linkage and multi-dimensional adjustment algorithm. This algorithm is used to dynamically adjust the PLC control cycle and the process parameters of the end effector, aiming to sacrifice a small amount of real-time performance to improve system stability.

[0032] First, the primary adjustment is configured to use a dynamic computing power balancing adjustment strategy, based on the aforementioned crash warning value. Adjusting the control cycle helps to smooth out system scheduling fluctuations. This is achieved by setting a basic control cycle. (Preferred to be 250 microseconds) and safety jitter threshold (Preferably 15%) is achieved. When the crash warning value... Less than or equal to the safe jitter threshold At the same time, maintain the target control cycle. Equal to the basic control cycle When the aforementioned crash warning value Greater than the safe jitter threshold At the same time, the target control cycle is calculated in real time. ,in The proportional gain coefficient is (preferably 2.0). Subsequently, the real-time control kernel dynamically extends its bus scan cycle to the target control cycle. The design principle behind introducing the squared term in this formula is to construct a variable gain characteristic, which makes the adjustment force increase exponentially when jitter is severe. By actively reducing the CPU time slice occupancy, it provides a computing power recovery window for non-real-time vision processes, thereby suppressing further deterioration of jitter at the physical level.

[0033] Secondly, the secondary regulation is configured to use a process flexibility degradation strategy, based on the accumulated inertial drift. Adjusting process parameters to maintain safe operation during visual lag. This strategy defines a process parameter decay model. ,in To set parameters (such as cutting depth). These are the actual execution parameters. In this embodiment, the crash warning value is... The basic attenuation coefficient is introduced as a correction factor. Construct dynamic attenuation coefficient ,in The linkage weight is set to 1.0 (preferably). At this point, when the system scheduling jitter... A large value indicates that the system as a whole is in an unstable state. In this case, increasing the dynamic attenuation coefficient can help. This makes the actual process parameters With drift amount The increase in the rate of change leads to a faster decay (i.e., a more sensitive lifting of the cutter head), thus providing more aggressive safety protection under conditions of overlapping risks. The response sensitivity of the secondary regulation is corrected by the state parameters of the primary regulation, reflecting the synergy of multi-dimensional regulation.

[0034] Finally, the three-level adjustment is configured to use a preemptive self-healing strategy based on the aforementioned jamming risk indicators. The algorithm is reset to prevent system crashes. This strategy has global blocking authority over preceding strategies. Specifically, when the aforementioned deadlock risk indicator... When the slope exceeds a preset threshold (preferably 5.0 ms / s), the visual algorithm is determined to be about to deadlock. The real-time control kernel immediately writes a forced reset instruction to the shared memory and simultaneously triggers the following linked actions: locking the dynamic balance adjustment strategy of computing power and forcing the target control cycle. Maintaining the value at the maximum allowable value (e.g., 10 milliseconds) ensures communication stability during the reset process; simultaneously, it covers the aforementioned process flexible degradation strategy, forcibly accumulating the inertial drift amount. Setting the value to infinity directly drives the end effector to a safe position of full lift or return to zero. The system maintains this state until it receives a new frame token sent after the vision process restarts, at which point the locking and overwriting are released, and normal closed-loop control is restored. This design transforms traditional manual shutdown maintenance into millisecond-level automatic reset within the system, significantly improving the continuous operation efficiency of the equipment.

[0035] It is important to note that the real-time control kernel described in this embodiment, in the specific form of industrial control, is a soft logic controller (Soft-PLC) running on a specific core of a general-purpose computer. This soft logic controller is not a traditional independent hardware device, but rather, through the scheduling mechanism of a real-time operating system, virtualizes the second and third cores of the central processing unit into logic execution units with microsecond-level deterministic response capabilities. In the control flow of this embodiment, this soft logic controller undertakes all the core functions of a traditional PLC: its periodic reading of the global atomic index corresponds to the input sampling stage of a traditional PLC; its inertial compensation calculation based on frame token comparison and its... , The adaptive algorithm derivation based on characteristic indicators corresponds to the user program execution stage of a traditional PLC, and its final result is based on the dynamically adjusted control cycle. and process parameters The actions of the driving servo mechanism correspond to the output refresh stage of a traditional PLC. Unlike the simple calculations based on ladder diagrams or Boolean logic in traditional PLCs, the soft logic controller in this embodiment utilizes the powerful floating-point computing capabilities of a personal computer to directly solve complex mathematical models containing exponential decay terms and standard deviation statistics in the real-time domain, thereby achieving high-order adaptive control tasks that traditional hardware PLCs cannot accomplish. The non-real-time operating system and vision process are bound to the 0th and 1st cores of the CPU; the real-time control kernel is bound to the 2nd and 3rd cores of the CPU, and interrupt requests from the non-real-time operating system are masked through kernel patches to ensure that the cycle jitter of the real-time control kernel is less than 10 microseconds.

[0036] The threshold and weight settings involved in this embodiment can be set by default according to the present invention, or can be set by those skilled in the art.

[0037] Example 2 This embodiment provides an electronic device, including: a processor and a memory, wherein the memory stores a computer program that can be called by the processor; The processor executes the aforementioned PC-based integrated vision and motion logic control method by calling computer programs stored in memory.

[0038] The electronic device can vary considerably depending on its configuration or performance. It may include one or more Central Processing Units (CPUs) and one or more memories, wherein the memory stores at least one computer program, which is loaded and executed by the processor to implement the PC-based integrated vision-motion logic control method provided in the above-described embodiment. The electronic device may also include other components for implementing device functions; for example, it may have wired or wireless network interfaces and input / output interfaces for data input and output. Further details are omitted in this embodiment.

[0039] Example 3 This embodiment proposes a computer-readable storage medium on which an erasable and rewritable computer program is stored. When the computer program runs on the computer device, it causes the computer device to execute the aforementioned PC-based integrated visual-motion logic control method.

[0040] For example, computer-readable storage media can be read-only memory (ROM), random access memory (RAM), compact disc read-only memory (CD-ROM), magnetic tape, floppy disk, and optical data storage devices.

[0041] Those skilled in the art will understand that embodiments of the present invention can be provided as methods, systems, or computer program products. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, the present invention can take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0042] This invention is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0043] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0044] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0045] The embodiments of the present invention have been described above with reference to the accompanying drawings. However, the present invention is not limited to the specific embodiments described above. The specific embodiments described above are merely illustrative and not restrictive. Those skilled in the art can make many other forms under the guidance of the present invention without departing from the spirit and scope of the claims. All of these forms are within the protection scope of the present invention.

Claims

1. A PC-based integrated vision-motion logic control method, characterized in that: Includes the following steps: S1. Construct a three-buffered ring array and a global atomic index in the physical memory of a multi-core PC as a data interaction channel between the vision process running on a non-real-time operating system and the PLC logic running on a real-time control kernel. S2. The vision process writes visual pose data and frame tokens to the free slots in the three buffers, and updates the global atomic index using CPU atomic instructions; S3. The real-time control kernel periodically reads the target slot pointed to by the global atomic index and compares the current frame token with the last execution token cached locally. S4. When the frame token is not updated, the real-time control kernel enters the inertial compensation mode, generates virtual target coordinates based on the motion state of the previous cycle, and synchronously accumulates and calculates the cumulative amount of inertial drift. S5. The real-time control kernel calculates the crash warning value representing system scheduling jitter and the deadlock risk index representing algorithm performance degradation in real time based on the historical interaction data of shared memory. S6. Based on the output of step S5, the real-time control kernel creates a multi-level linkage and multi-dimensional adjustment algorithm to dynamically adjust the PLC control cycle and the process parameters of the end effector.

2. The PC-based integrated vision-motion logic control method according to claim 1, characterized in that: In step S2, the vision process first reads the global atomic index to identify the active slot currently occupied by the real-time control kernel; An inactive slot is selected from the three-buffered circular array as a write target, and a data packet to be loaded, including 6-DOF pose, computation time, and auto-incrementing frame token, is written. Then, atomic instructions are compared and exchanged, and the value of the global atomic index is modified to the index value of the write target within a single instruction cycle.

3. The PC-based integrated vision-motion logic control method according to claim 2, characterized in that: When the current frame token mentioned in step S3 is greater than the execution token in the local cache, the execution token in the local cache is updated to the current frame token, and the accumulated inertial drift is also increased. Zero out the current slot and use the visual pose data in the current slot for motion planning. When the current frame token equals the locally cached execution token, visual data lag is determined, the locally cached execution token remains unchanged, and the end-of-cycle instruction speed of the previous control cycle is obtained. and acceleration Using kinematic formulas Calculate the virtual target coordinates for the current cycle to drive the servo mechanism to maintain smooth motion; simultaneously, update the accumulated inertial drift. The string is kept cleared until a new frame token is received. This represents the amount of motion accumulated by the robot while it is blindly walking during the period when its vision is not updated. For the current control cycle, These are the position coordinates of the previous cycle.

4. The PC-based integrated vision-motion logic control method according to claim 1, characterized in that: The multi-level linkage and multi-dimensional adjustment algorithm includes: Level 1 adjustment is configured to use a dynamic computing power balancing strategy, based on the aforementioned crash warning value. Adjust the control cycle to smooth out system scheduling jitter; The secondary adjustment is configured to utilize a process flexibility degradation strategy based on the accumulated inertial drift. Adjust process parameters to maintain safe operation during visual lag; Level 3 adjustment: Configured to use a preemptive self-healing strategy based on the aforementioned deadlock risk indicators. Perform an algorithm reset to prevent system crashes; The response sensitivity of the secondary regulation is corrected by the state parameters of the primary regulation, and the triggering of the tertiary regulation has the highest priority. When triggered, the primary and secondary regulation are forcibly linked to enter a safe holding state.

5. The PC-based integrated vision-motion logic control method according to claim 4, characterized in that: The computing power dynamic balance adjustment strategy sets a basic control cycle. and safety jitter threshold Implementation, specifically including: When the crash warning value Less than or equal to the safe jitter threshold At the same time, maintain the target control cycle. Equal to the basic control cycle ; When the crash warning value Greater than the safe jitter threshold At the same time, the target control cycle is calculated in real time. ,in The proportional gain coefficient; the real-time control kernel dynamically extends its bus scan cycle to the target control cycle. By actively reducing the CPU time slice usage, a computing power recovery window is provided for non-real-time vision processes.

6. The PC-based integrated vision-motion logic control method according to claim 4, characterized in that: The process flexibility degradation strategy includes defining a process parameter attenuation model. ,in To set parameters, For actual execution parameters, This is the attenuation coefficient; subsequently, the collapse warning value is... The attenuation coefficient is introduced as a correction factor. Construct dynamic attenuation coefficient ,in This is a weighted linkage.

7. The PC-based integrated vision-motion logic control method according to claim 6, characterized in that: The preemptive self-healing strategy has the global right to block the linkage of preceding strategies, specifically manifested as follows: When the aforementioned jamming danger indicator When the slope exceeds a preset threshold, the visual algorithm is determined to be about to deadlock. The real-time control kernel immediately writes a forced reset instruction to the shared memory and locks the dynamic balance adjustment strategy for computing power, forcing the target control cycle. Maintain at the maximum allowable value; simultaneously cover the process flexible degradation strategy, forcibly accumulating the inertial drift amount. Set to infinity, directly drive the end effector to enter a safe pose of full lift or return to zero until a new frame token is received after the vision process restarts. Then, release the above lock and cover, and restore normal closed-loop control.

8. The PC-based integrated vision-motion logic control method according to claim 1, characterized in that: The visual process executes a deep learning defect recognition algorithm based on RGB-D data acquired by a 3D structured light camera; and in step S2, a data packet with three buffers is written, specifically containing the three-dimensional coordinates of the identified defect, the defect type code, and the computation time data of this algorithm processing; then the jamming risk indicator... Read visual computation time data from multiple consecutive frames in shared memory and calculate its slope over time; The crash warning value By extracting the jitter of the time interval when the data packet is written, a sliding window is constructed to record the time interval sequence of the most recent N frame token updates, and the standard deviation of the updated time interval sequence is calculated and normalized.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, it implements any one of the PC-based integrated vision-motion logic control methods of claims 1-8.

10. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a computer program, which, when executed by a processor, implements the PC-based integrated vision-motion logic control method of any one of claims 1-8.

Images