An industrial robot quality detection system and method based on industrial vision

Abstract

The application discloses an industrial robot quality detection system and method based on industrial vision, and relates to the technical field of image recognition. The steps of the method comprise: alternately triggering a plurality of polarized light sources to collect polarized images of a to-be-detected die-cutting piece; determining the relative displacement vector of a plurality of die-cutting stations through real-time triggered pulse counting to construct a parameter vector matrix; introducing a constraint rule engine, combining a robot configuration state, and jointly inputting into a preset mask analysis model to identify the defect level and the target conflict area of the to-be-detected die-cutting piece through cross attention matching; analyzing the hardware coding of a suction cup and the spatial position of the to-be-sorted die-cutting piece to construct a sorting scheduling topology, reserving a given sorting path segment as a background super node, and generating a sorting action track; and optimizing the sorting action track to generate a track sub-solution through an asynchronous double-time window judgment mechanism. The application ensures ultra-high real-time performance and realizes high fault-tolerant collaborative control at the system level.

Description

Technical Field

[0001] This invention relates to the field of image recognition technology, specifically to an industrial robot quality inspection system and method based on industrial vision. Background Technology

[0002] With the continuous improvement of industrial automation and production efficiency, die-cutting machines, as indispensable equipment in industries such as packaging and printing, are also facing increasing demands for performance and functionality. High-speed sorting and quality inspection on automated production lines are core components of modern intelligent manufacturing.

[0003] Traditional industrial vision inspection and robotic sorting systems are increasingly revealing a series of insurmountable technical limitations when dealing with high reflectivity, dynamic multi-station collaboration, and concurrent control of multiple actuators (such as array-type multi-suction cups). Firstly, most die-cutting machines use single-unit die-cutting devices. While these devices can meet basic production needs in certain applications, their efficiency limitations become increasingly apparent when facing large-volume production tasks. Secondly, because die-cut parts are typically complex multi-layered materials such as metal foil, release film, or high-gloss polymers, image recognition technology struggles to effectively distinguish the contour features and minute defects of die-cut parts on high-speed conveyor belts, easily leading to missed or false detections. This is especially true in high-speed conveyor belts. In the event of a sudden conflict, the robot's configuration status cannot be perceived. Due to the lack of coordination between the sorting motion trajectory and the timing data of end effectors such as suction cups and solenoid valves, not only is the computational cost enormous and the communication response latency high, but the system is also prone to deadlock due to the physical slippage of the pneumatic actuators and the disconnection of data from the master control log. This can easily lead to mechanical cross-collisions between multiple suction cups, resulting in mis-sampling and missed sorting. On the other hand, on high-speed, continuously operating die-cutting lines, a global A* algorithm is usually used to optimize the sorting motion trajectory. This global recalculation brings huge computational costs and system communication delays. The robotic arm often misses the optimal sorting window because it cannot respond in time. To a certain extent, this will greatly reduce the accuracy and precision of sorting. Summary of the Invention

[0004] (a) Technical problems to be solved To address the shortcomings of existing technologies, this invention provides an industrial robot quality inspection system and method based on industrial vision. It integrates polarized images acquired by alternately triggering dual-polarization light sources with the relative displacement vector of the workstation determined by real-time pulse counting to construct a parameter vector matrix. This matrix, combined with a remainder rule engine and robot configuration status, is input as a unified data stream into a mask analysis model. Utilizing a cross-attention matching mechanism, it accurately identifies defect levels while directly calculating the target conflict area. When constructing the sorting scheduling topology based on the target conflict area, it reduces and solidifies the data of unaffected predetermined sorting path segments into spatiotemporal background supernodes. Through an asynchronous dual-time-window judgment mechanism, it searches the sorting action trajectory, outputs the corresponding trajectory sub-solution, and securely backfills the fault-tolerant execution result data into the main control log, thus solving the problems raised in the background technology.

[0005] (II) Technical Solution To achieve the above objectives, the present invention provides the following technical solution: In a first aspect, this application provides a quality inspection method for industrial robots based on industrial vision. The method includes: alternately triggering a first polarization light source and a second polarization light source to acquire a first polarization image and a second polarization image of the die-cut part to be tested; determining the relative displacement vector of the first die-cutting station and the second die-cutting station by counting pulses triggered in real time, and constructing a parameter vector matrix by combining the first polarization image and the second polarization image. Based on the parameter vector matrix, a constraint rule engine is introduced, and the robot configuration state is jointly input into the preset mask analysis model. The defect level and target conflict area of ​​the die-cut part to be tested are identified through cross-attention matching. According to the target conflict area, the sorting action is executed. By analyzing the suction cup hardware code and the spatial position of the die-cut part to be sorted, the sorting scheduling topology is constructed, and the predetermined sorting path segment is retained as the background super node to generate the sorting action trajectory. A sorting conflict event is marked based on the target conflict area, and the sorting action trajectory is optimized. The sorting conflict event is optimized synchronously, and a trajectory sub-solution is generated through an asynchronous dual time window judgment mechanism. Only the fault-tolerant execution result data that has passed the verification is backfilled to the main control log.

[0006] Further, first and second polarization images of the die-cut part under test are acquired, including: In response to the first trigger pulse, the polarization CMOS photosensitive array is driven to be exposed for the first time, and the first polarization components of the die-cut part under test under the first polarization light source (horizontal, vertical, 45° oblique and 135° oblique) are captured and acquired simultaneously, and packaged to construct the first polarization image; wherein, the first polarization light source includes three bands of red, green and blue visible light. In response to the second trigger pulse, the polarization CMOS photosensitive array is driven to undergo a second exposure, simultaneously capturing and acquiring the second polarization components of the die-cut part under test under the second polarization light source, including horizontal, vertical, 45° oblique, and 135° oblique components, and packaging them to construct a second polarization image; wherein, the first polarization light source includes the near-infrared band.

[0007] Further, construct the parameter vector matrix, including: Based on the first polarization image, the first feature component is separated and embedded into the first channel; Based on the second polarization image, the second feature vector is separated and embedded into the second channel; The relative displacement vector is embedded into the third channel, and the eigenvalues ​​of each channel are linearly mapped to the feature tensor space. Using the relative displacement vector, the first and second feature components corresponding to the overlapping areas of multiple die-cutting stations are affinely translated and recombined to construct the parameter vector matrix.

[0008] Furthermore, the constraint rule engine includes first constraint rules for visual analysis and second constraint rules for sorting execution: The first constraint rules include: ROI physical boundary, pixel grayscale gradient threshold, and lower limit of single trigger pulse window; The second set of constraints includes: suction cup geometric boundaries, throughput limits, parameter boundaries for multispectral alternation, and solenoid valve action response delay; The first and second constraint rules are converted into semantic constraint vectors, which are then combined with the robot configuration state to perform mask reconstruction of constraint boundary vectors. The parameter vector matrix is ​​used as the query matrix, and the constraint boundary vector is used as the key matrix. The dot product of the query matrix and the matrix is ​​performed to obtain feature activation values. After normalization, several sorting action mask matrices containing the suction cup opening and closing sequence are output. Among them, the robot configuration state includes at least the suction cup opening and closing sequence. The feature activation values ​​are compared with the preset activation value threshold to determine the defect level, including mild, moderate and severe defect levels, and the connected components of defective pixels are extracted from the sorting action mask matrix. Under the condition of triggering a mild defect level, the maximum Euclidean distance of the connected domain of the defective pixel is identified by rectangular meshing to determine the envelope half-width and envelope height of the sorting action, generate a rectangular boundary, and perform equal-interval interpolation sampling on the horizontal and vertical boundary axes of the rectangular boundary to obtain a rectangular mesh cell feature set with a total number of M0. Under the condition of triggering a medium or high defect level, the geometric centroid of the connected domain of the defect pixel is identified by circular meshing. The envelope radius of the sorting action is determined based on the solenoid valve action response delay, a circular boundary is generated, and sampling is performed according to a preset step size to obtain a circular unit feature set with a total number of M1; where M0=M1. Extract the suction cup opening and closing sequence of two adjacent grid cells, and reconstruct it through connected component partitioning. Close the beginning and end of the grid corresponding to the sorting action to form the target conflict area.

[0009] Furthermore, a sorting scheduling topology is constructed, including: Based on the target conflict area, discrete feature extraction is performed on the mask matrix of each sorting action to obtain the suction cup code and the spatial position of the die-cut parts to be sorted. These are then combined in pairs to construct independent task nodes, and topological edges with time delay constraints are established to generate the sorting scheduling topology. Simultaneously, standard sorting trajectory sequences from non-target conflict areas are retrieved. Through discretization sampling, standard sorting trajectory sequences containing multi-dimensional spatial locations are obtained, mapped to the target conflict area, and spatiotemporal intersection operations are performed. If the spatiotemporal intersection is empty, the sorting trajectory sequence corresponding to the target conflict area is determined as the predetermined sorting segment, and the corresponding independent task node and topological edge are locked and clustered and encapsulated as a spatiotemporal background super node.

[0010] Furthermore, the time delay constraint includes: determining multiple time-space interval partitions based on the suction cup opening and closing timing of the parameter vector matrix, establishing a control index based on each time interval partition, and recording the control index of a sorting conflict event through the sorting scheduling topology.

[0011] Furthermore, the optimization of the sorting scheduling topology trajectory includes: Construct an objective function that minimizes the total time spent on sorting tasks and the variance of suction cup load fluctuation. Use particle swarm optimization to perform sorting action trajectory planning in a preset first time window and a preset second time window respectively. Iterate the objective function to obtain the first trajectory sub-solution and the second trajectory sub-solution respectively. The predetermined sorting segment in the first trajectory sub-solution is extracted, and the sorting segment to be optimized corresponding to the target conflict area in the second trajectory subset is extracted. By applying particle swarm velocity constraints, a smooth transition segment is generated by B-spline. The extracted sorting segments are then spliced ​​and merged into the sorting action trajectory.

[0012] Furthermore, the preset first time window is a coarse-grained time window, and the preset second time window is a fine-grained time window.

[0013] Furthermore, after completing a single sorting action, the process includes: Air pressure feedback data is collected by an air pressure sensor and compared with the robot's configuration status for analysis. If the air pressure feedback data indicates effective negative pressure, the sorting is determined to have failed and a secondary reverse air blowing command is triggered. The backfill fault tolerance execution result is updated, and the vibration acceleration of the current die-cutting station is retrieved. The structured encrypted log data is then output to the server through encrypted private key signature.

[0014] Secondly, this application provides an industrial robot quality inspection system based on industrial vision, the system comprising: The parameter construction module alternately triggers the first polarization light source and the second polarization light source to acquire the first polarization image and the second polarization image of the die-cut part under test; by counting the pulses triggered in real time, the relative displacement vector of the first die-cutting station and the second die-cutting station is determined, and a parameter vector matrix is ​​constructed by combining the first polarization image and the second polarization image; The mask analysis module, based on the parameter vector matrix, introduces a constraint rule engine and combines the robot configuration state with the input to the preset mask analysis model. It identifies the defect level and target conflict area of ​​the die-cut part to be tested through cross-attention matching. Based on the target conflict area, it performs sorting actions. By analyzing the suction cup hardware code and the spatial position of the die-cut part to be sorted, it constructs a sorting scheduling topology and retains the predetermined sorting path segment as background super nodes to generate sorting action trajectories. The sorting module is optimized by marking a sorting conflict event based on the target conflict area and optimizing the sorting action trajectory. The sorting conflict event is optimized synchronously, and a trajectory sub-solution is generated through an asynchronous dual-time window judgment mechanism. Only the fault-tolerant execution result data that has passed the verification is backfilled to the main control log.

[0015] (III) Beneficial Effects This invention provides an industrial robot quality inspection system and method based on industrial vision, which has the following beneficial effects: This invention acquires multimodal polarization images by alternately triggering the first and second polarization light sources, and organically combines the relative displacement vector between the two die-cutting stations calculated by real-time pulse counting to construct a parameter vector matrix. Based on this, the matrix, along with the constraint rule engine and the robot's real-time configuration status, is linked and input into a mask analysis model based on cross-attention matching to achieve deep alignment and cross-domain fusion of multimodal data. This enables accurate identification of multi-level defect levels in die-cut parts and precise calculation of target conflict areas where sorting actions interfere. Combining the suction cup hardware code and the spatial position data of the die-cut parts to be sorted, a sorting scheduling topology network is abstractly constructed. By reducing and solidifying the predetermined sorting path segments unaffected by conflict areas into spatiotemporal background supernodes that do not participate in dynamic optimization, the computational power of the safe operation boundary is locked, reducing the computational power of path optimization. This allows the trajectory generation optimization step to accurately focus on the target conflict area, significantly improving the quality inspection accuracy and timeliness under high-speed dynamic production lines. This invention employs an asynchronous dual-time-window decision mechanism. It uses a coarse-grained time window for global optimization and a fine-grained time window for local optimization. By performing asynchronous solutions within different parallel time windows, the system can generate trajectory sub-solutions for specific conflict events with extremely high response speed. It extracts a predetermined sorting segment from the first trajectory sub-solution and a sorting segment corresponding to the target conflict area from the second trajectory subset. By applying particle swarm velocity constraints and generating smooth transition segments using B-splines, the extracted sorting segments are spliced ​​and merged into the sorting action trajectory. Finally, a rigorous fault-tolerant verification logic is introduced in the backfilling stage to ensure that only fault-tolerant execution results verified by hardware security are securely written to the main control log. This not only guarantees the system's extremely fast response capability but also fundamentally eliminates chain physical collisions and data deadlocks caused by aerodynamic response delays, significantly improving the overall robustness and production continuity of industrial robots under harsh working conditions. Attached Figure Description

[0016] Figure 1 This is a flowchart illustrating the present invention. Detailed Implementation

[0017] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0018] By alternately triggering polarization images acquired by dual polarization light sources and fusing them with the relative displacement vector of the workstation determined by real-time pulse counting, a parameter vector matrix is ​​constructed. Combined with the remainder rule engine and robot configuration status, this matrix is ​​input into the mask analysis model as a unified data stream. Using a cross-attention matching mechanism, the target conflict area is directly calculated while accurately identifying the defect level. When constructing the sorting scheduling topology based on the target conflict area, the data of the unaffected predetermined sorting path segments is reduced in dimension and solidified into spatiotemporal background supernodes. Through an asynchronous dual-time-window judgment mechanism, the high-frequency alignment and parallel feedback on the coarse and fine-grained time axes of the sorting action trajectory are calculated and optimized for conflict events. The corresponding trajectory sub-solution is output and finally closely coordinated with the fault-tolerant verification data of real-time feedback. Only the fault-tolerant execution result data that has passed the hardware safety verification is safely backfilled into the main control log, achieving high fault-tolerant collaborative control at the system level while ensuring ultra-high real-time performance.

[0019] Example 1: This embodiment of the invention provides a quality inspection method for industrial robots based on industrial vision; Figure 1 This is a flowchart illustrating the present invention; please refer to [link / reference]. Figure 1 The method includes the following steps: S1: Alternately trigger the first polarization light source and the second polarization light source to acquire the first polarization image and the second polarization image of the die-cut part under test; determine the displacement vector of the first die-cutting station and the second die-cutting station by real-time pulse counting, and construct a parameter vector matrix by combining the first polarization image and the second polarization image; Acquiring the first and second polarization images of the die-cut part under test includes: responding to a first trigger pulse to illuminate the red, green, and blue visible light bands contained within the first polarization light source. For example, the first polarization light source uses 620nm red, 520nm green, and 450nm blue LEDs, with each band of LED integrating a linear polarizer. The polarization direction corresponds one-to-one with the micro-polarizers in the photosensitive array, ensuring that the polarization components of each band can be accurately acquired; under the condition that the first polarization light source reaches a preset extinction ratio, driving the polarization CMOS photosensitive array for the first exposure, through the four different polarization components integrated in each pixel unit of the polarization CMOS photosensitive array... The directional micro-polarizers simultaneously capture and decompose the first polarization component data of the die-cut workpiece under test under the first polarization light source in the horizontal, vertical, 45° oblique, and 135° oblique directions, and package them to construct a first polarization image. For example, the first trigger pulse is generated by the incremental encoder of the production line. When the conveyor belt moves 50mm, it outputs a pulse, which corresponds to a trigger frequency of 10Hz when the production line speed is 0.5m / s. Each pixel of the polarization CMOS integrates micro-polarizers in four directions: 0°, 90°, 45°, and 135°. A single exposure can simultaneously acquire four polarization components and package them into a first polarization image with a resolution of 1280×1024. After a preset time interval, the first trigger pulse is turned off and a second trigger pulse is sent to the second polarization light source. The preset time interval is used to isolate the residual wavelength data of the three visible light bands. In response to the second trigger pulse, the near-infrared light band inside the second polarization light source is illuminated. For example, the second polarization light source uses an 850nm near-infrared LED and integrates a linear polarizer, with the polarization direction corresponding one-to-one with the micro-polarizer of the photosensitive array. When the second polarization light source reaches a preset polarization phase angle, the polarization CMOS photosensitive array is driven to undergo a second exposure. Through the four micro-polarizers with different directions integrated in each pixel unit of the polarization CMOS photosensitive array, the horizontal, vertical, 45° oblique, and 135° oblique second polarization component data of the die-cut part under test under the second polarization light source are simultaneously captured and separated, and packaged into a second polarization image. For example, the second trigger pulse is generated alternately with the first trigger pulse at an interval of 50ms, corresponding to a dual-light source alternation frequency of 10Hz. The polarization CMOS has a good response to 850nm near-infrared light. The second exposure acquires the four polarization components of the near-infrared band and packages them into a second polarization image with the same resolution as the first polarization image. By collecting the full polarization information of the die-cut parts in the near-infrared band, near-infrared light has strong penetrability and can penetrate the surface coating or transparent substrate of the die-cut parts to detect internal defects such as delamination and bubbles, thus making up for the shortcomings of visible light detection. A parameter vector matrix is ​​constructed, including: obtaining the pulse count of the die-cut parts under test flowing through the first and second die-cutting stations through the conveyor line encoder, multiplying it by the pre-stored single-pulse physical constant, and calculating the relative displacement vector used to characterize the real-time movement distance of the die-cut parts under test between the two stations. The single-pulse physical constant is defined as the actual physical distance the die-cut part moves with the conveyor belt when the encoder outputs one trigger pulse. It converts the abstract, meaningless pulse count output by the encoder into a displacement value with clear spatial meaning. In high-speed continuous production, the die-cut part will appear simultaneously in the field of view of the first and second die-cutting stations, forming an image overlap. The features of the overlapping area are extracted and merged from the two images through affine translation and reconstruction. This process relies entirely on accurate relative displacement vectors. It should be noted that due to wear of conveyor rollers, belt slippage, etc., the single-pulse physical constant will change over time and requires periodic calibration. The calibration method is as follows: move the conveyor line a known precise distance, count the pulses output by the encoder, and obtain the actual single-pulse physical constant by dividing the known distance by the pulse count. The higher the encoder resolution, the smaller the single-pulse physical constant, and the higher the displacement measurement accuracy. Channel separation and spatial resolution alignment are performed on the acquired first and second polarization images to obtain standardized one-dimensional surface feature components and one-dimensional internal interlayer feature components. Specifically, based on the first polarization image, the first feature component is separated and embedded into the first channel, providing a unified data format for subsequent matrix operations and model processing. The first feature component contains the polarization degree, polarization angle, and grayscale information of the die-cut surface, which can effectively characterize the features of surface defects. For example, the polarization degree, polarization angle, and grayscale value of each pixel are calculated from the first polarization image, combined into a 3D first feature component, and embedded into the first channel of the parameter vector matrix to form a 1280×1024×3 first channel feature map. Based on the second polarization image, a second feature vector is extracted and embedded into the second channel. This is used to extract key features from the near-infrared polarization image and map them to the second channel of the feature tensor, achieving the fusion of surface and internal features. The second feature component contains the polarization degree, polarization angle, and grayscale information of the die-cut part, which can effectively characterize the features of internal defects. For example, the polarization degree, polarization angle, and grayscale value of each pixel are calculated from the second polarization image, combined into a 3D second feature component, and embedded into the second channel of the parameter vector matrix to form a second channel feature map of the same size as the first channel. The relative displacement vector is embedded into the third channel, and the feature values ​​of each channel are linearly mapped to the feature tensor space. Using the relative displacement vector as a boundary constraint factor, the first and second feature components corresponding to the overlapping areas of multiple die-cutting stations are affinely translated and recombined. The feature coordinates of the overlapping areas are adjusted according to the relative displacement to ensure that the feature at each position corresponds to the same area of ​​the same die-cut part. For example, the relative displacement vector of the first and second die-cutting stations is (100mm, 0mm), which corresponds to an image pixel displacement of (200 pixels, 0 pixels). The x and y components of the relative displacement vector are embedded into the third channel to form a 1280×1024×2 third channel feature map. The first and second feature components of the overlapping areas are subjected to an affine translation transformation, and the feature of the second station is translated 200 pixels to the right to align with the first station. Finally, a 1280×1024×8 parameter vector matrix is ​​constructed.

[0020] S2: Based on the parameter vector matrix, a constraint rule engine is introduced and combined with the robot configuration state to input into the preset mask analysis model. The target conflict area to be sorted is identified through cross-attention matching. Based on the target conflict area, the suction cup hardware code and the spatial position of the die-cut parts to be sorted are analyzed to construct the sorting scheduling topology and retain non-target conflict areas as background super nodes. The constraint rule engine includes first constraint rules for visual analysis and second constraint rules for sorting execution. The first constraint rules include: ROI physical boundary, pixel grayscale gradient threshold, and single-trigger pulse window lower limit. The ROI physical boundary defines the detection area to avoid background interference; the pixel grayscale gradient threshold is used to detect edges and defects, filtering out normal areas with smooth grayscale changes; the single-trigger pulse window lower limit ensures sufficient exposure time to avoid feature loss due to overly dark images. For example, the ROI physical boundary is set to a 300mm wide area in the middle of the conveyor belt, corresponding to an image x-coordinate range of 100~700 pixels; the pixel grayscale gradient threshold is determined by statistically analyzing the grayscale gradient values ​​of 1000 normal die-cut parts images, taking the 95th percentile as the threshold, and setting it to 20; the single-trigger pulse window lower limit is set to 1ms to ensure that the exposure time is not less than 1ms. The second set of constraints includes: suction cup geometric boundaries, throughput limits, multispectral alternation parameter boundaries, and solenoid valve response delay. The suction cup geometric boundaries define the effective suction range of the suction cup, preventing suction from exceeding its limits. The throughput limit ensures the system's production efficiency meets production line requirements. The multispectral alternation parameter boundaries ensure the stability of light source switching, preventing light source damage. The solenoid valve response delay compensates for the time difference between suction cup opening and closing, ensuring the accuracy of the suction action. For example, the suction cup geometric boundary is set to a circular area with a diameter of 20mm, corresponding to an image radius of 20 pixels; the system throughput limit is set to 60 pieces / minute; the minimum time interval for multispectral alternation is set to 50ms; and the solenoid valve response delay, determined through actual measurement, is set to 10ms. The first and second constraint rules are converted into semantic constraint vectors. By mapping each constraint rule to a fixed-length vector, these vectors are combined with the robot configuration state to perform mask reconstruction of the constraint boundary vector. The robot configuration state includes, but is not limited to, position, velocity, suction cup state, and air pressure feedback threshold. Using the parameter vector matrix as the query matrix and the constraint boundary vector as the key matrix, the dot product of the query matrix and the key matrix is ​​performed to obtain feature activation values. The normalized sorting action mask matrix represents the opening and closing state of each suction cup at different time steps. For example, the first constraint rule is converted into a 10-dimensional semantic constraint vector, the second constraint rule into a 15-dimensional semantic constraint vector, and the robot configuration state into a 5-dimensional vector, which are combined into a 30-dimensional constraint boundary vector. The parameter vector matrix is ​​flattened into a 1310720×8 query matrix, and the constraint boundary vector is expanded into a 1310720×30 key matrix. The dot product is performed to obtain 1310720×1 feature activation values. After normalization, four sorting action mask matrices are output, each with a size of 1280×1024×10. It should be noted that the higher the feature activation value, the greater the probability that the area is a defective area. The activation value threshold is determined by statistically analyzing the feature activation values ​​of a large number of defective samples. By comparing the feature activation value with the preset activation value threshold, the defect level is determined, and the connected components of the defective pixels are extracted in the sorting action mask matrix. The specific steps include: if the feature activation value is less than the preset minimum activation value threshold, the current feature activation value is marked as 'a' and assigned a first-level character, and 'a' is combined with the first-level character to generate a mild defect level. Under the condition of triggering a minor defect level, the maximum Euclidean distance of the connected components of the defective pixels is identified by rectangular meshing to determine the envelope half-width and envelope height of the sorting action, generating a rectangular boundary. Equal-interval interpolation sampling is then performed on the horizontal and vertical boundary axes of the rectangular boundary to obtain a rectangular mesh cell feature set with a total number of M0. It should be noted that minor defects are usually relatively regular in shape, and the rectangular envelope can simplify the sorting action planning while ensuring coverage of the defect area. If the feature activation value is greater than or equal to the preset minimum activation value threshold and less than or equal to the preset maximum activation value threshold, then the current feature activation value is marked as b and assigned a second-level character, and b is combined with the second-level character to generate a moderate defect level; if the feature activation value is greater than the preset maximum activation value threshold, then the current feature activation value is marked as c and assigned a third-level character, and c is combined with the third-level character to generate a severe defect level. Under the condition of triggering a moderate or high defect level, the geometric centroid of the connected domain of the defect pixel is identified through circular meshing. The envelope radius of the sorting action is determined based on the solenoid valve action response delay, generating a circular boundary. Sampling is performed according to a preset step size to obtain a circular unit feature set with a total number of M1; where M0=M1. It should be noted that moderate and severe defects are usually irregular in shape, and the circular envelope can better cover the defect area while ensuring the stability of the suction cup adsorption. The geometric centroid is the center position of the defect area, serving as the center of the circular envelope. The envelope radius takes into account the solenoid valve action response delay and the robot's movement speed, ensuring that the suction cup opens just when it reaches the defect position. M0=M1 ensures that the number of mesh units for mild and moderate to severe defects is the same, facilitating subsequent unified processing. Extract the suction cup opening and closing timing of two adjacent grid cells. Through connected component decomposition and reconstruction, close the beginning and end of the grid corresponding to the sorting action to form the target conflict region. It should be noted that if the same suction cup needs to adsorb two different regions at the same time, or if two suction cups need to adsorb the same region at the same time, this will lead to sorting conflicts. For example, if the suction cup opening and closing timings of two adjacent grid cells are t1~t2 and t1.5~t2.5 respectively, there is an overlap of 0.5 time steps. Through connected component decomposition and reconstruction, these two grid cells are merged into a single target conflict region, which is marked as the region that needs trajectory optimization. Additionally, if it is necessary to obtain the feature activation value of a grid cell, the built-in addressing resolver is called. For example, for a rectangular grid cell feature set: the dynamic row and column coordinate interval corresponding to the grid cell is read; the sorting action mask matrix is ​​extracted by matrix slicing index through the dynamic row and column coordinate interval to obtain the values ​​of all matrix elements in the interval, and the average or maximum value of all matrix elements in the interval is calculated. The average or maximum value is then determined as the feature activation value of the current grid cell. The sorting scheduling topology is constructed as follows: Based on the target conflict area, discrete feature extraction is performed on the mask matrix of each sorting action to obtain the suction cup code and the spatial position of the die-cut part to be sorted; the centroid extraction algorithm of the connected component is used to obtain the center pixel coordinates of the die-cut part in the target conflict area, and the center pixel coordinates are fused with the pulse count based on the preset hand-eye calibration matrix to generate the spatial position coordinates of the die-cut part in the robot base coordinate system and the timestamp data of the arrival at the corresponding suction cup grasping window; independent task nodes are constructed by pairwise combination, and a topological edge with time delay constraint is established between two independent task nodes to generate the sorting scheduling topology; the time delay constraint includes: determining multiple spatiotemporal interval partitions based on the suction cup opening and closing timing of the parameter vector matrix, and each spatiotemporal interval partition corresponds to a time window and spatial range; a control index is established based on each time interval partition, and the control index contains key information such as the time, location, suction cup and die-cut part involved in the conflict, which can quickly locate the conflict event; the control index of a sorting conflict event is recorded through the sorting scheduling topology to realize the association between the conflict event and the scheduling topology; Simultaneously, standard sorting trajectory sequences from non-target conflict areas are retrieved. Through discretization sampling, standard sorting trajectory sequences containing multi-dimensional spatial locations are obtained and mapped to the target conflict area. Spatiotemporal intersection operations are performed to determine whether the standard sorting trajectory will pass through the target conflict area. If the spatiotemporal intersection is empty, the sorting trajectory sequence corresponding to the target conflict area is determined as a predetermined sorting segment, and the corresponding independent task nodes and topological edges are locked and clustered and encapsulated as spatiotemporal background supernodes to reduce the search space for subsequent trajectory optimization. It should be noted that an empty spatiotemporal intersection indicates that the sorting trajectory sequence will not conflict with the target conflict area and can be directly reused. Locking the corresponding task nodes and topological edges prevents them from being modified in subsequent trajectory optimization. Clustering and encapsulating them as spatiotemporal background supernodes treats multiple related nodes and edges as a whole, greatly reducing the complexity of trajectory optimization and improving the optimization speed. At the same time, based on the channel physical mapping index of the sorting action mask matrix, the suction cup hardware code of the corresponding suction cup and the spatial position of the die-cut parts to be sorted are obtained by reverse lookup.

[0021] S3: Mark a sorting conflict event based on the target conflict area, optimize the sorting action trajectory, optimize the sorting conflict event synchronously, generate trajectory sub-solutions through an asynchronous dual time window judgment mechanism, and only backfill the fault-tolerant execution result data that has passed the verification to the main control log. The optimization of the sorting scheduling topology trajectory includes: constructing an objective function that minimizes the total time spent executing sorting tasks and the variance of suction cup load fluctuation: F = min(total time spent on sorting tasks) × first weight + min(variance of suction cup load fluctuation) × second weight; where F is the objective function, min(total time spent on sorting tasks) minimizes the total time spent on sorting tasks, ensuring system production efficiency, and min(variance of suction cup load fluctuation) minimizes the variance of suction cup load fluctuation, ensuring the balance of suction cup load and extending equipment lifespan; the first weight and the second weight range from 0 to 1, and the sum of the first weight and the second weight is 1, adaptively updated during the iterative training of the particle swarm optimization algorithm; it should be noted that in the process of constructing the objective function, the total time spent executing sorting tasks and the variance of suction cup load fluctuation need to be normalized to the range of 0 to 1, eliminating the influence of dimensions and making the calculation process physically meaningful; The particle swarm optimization (PSO) algorithm is used to perform sorting trajectory planning within a preset first time window and a preset second time window. The iterative solution process for the objective function is as follows: Initialize the time window and population: Establish a high-dimensional space and generate a reference vector set uniformly distributed in the high-dimensional target search space to provide direction for population evolution; According to the preset time span, the dynamic optimization space is decoupled and divided into a preset first time window and a preset second time window. The first time window is a coarse-grained time window, usually 2.0s to 5.0s. Within this window, the PSO algorithm is invoked to perform global iterative solution of the objective function with a larger spatial deviation step size to quickly obtain the globally optimal coarse trajectory, thus obtaining the first trajectory sub-solution; The second time window is a fine-grained time window, usually 50ms to 200ms. Within this window, the PSO algorithm is invoked, and the step size is switched from a larger step size to a smaller step size to perform fine optimization of the conflict area, thus obtaining the locally optimal fine trajectory, thus obtaining the second trajectory sub-solution. An initial population is randomly generated, with a maximum number of iterations L_max. The population consists of multiple particles, each representing a sequence of sorting motion trajectory points within the current sorting scheduling topology. Each particle has a unique position vector code and velocity vector code. The position vector is encoded as a sequence of three-dimensional spatial coordinates of the robotic arm's end effector on a continuous time axis, along with the opening and closing control status of the array suction cups at the corresponding time points. The velocity vector is encoded as the rate of change of the three-dimensional spatial coordinate sequence between the first and second time windows. The velocity vector encoding is used as the search direction for the solution, and the particle's position in space is used as the weight of the objective function. A binary competition selection strategy is adopted, in which the particle with the best fitness is selected by comparing fitness and added to the mating pool. This step is repeated until the number of particles in the mating pool reaches the maximum number of iterations L_max, where L_max is a positive integer. Each time, two particles are randomly selected from the mating pool, and simulated binary crossover and polynomial mutation operations are performed on them to generate two new particles, which are added to the subpopulation. The number of individuals in the subpopulation is filled to L_max, and then the subpopulation is merged with the parent population to form a mixed population. In the first time window, the particle swarm optimization algorithm is called, and the velocity vector encoding is controlled to perform global iterative solution of the objective function with a large spatial deviation step size to quickly obtain the globally optimal coarse trajectory. After the convergence condition is met, the first trajectory sub-solution is obtained. In the second time window, the particle swarm optimization algorithm is called, and the velocity vector encoding is controlled to switch from a large step size to a smaller step size. The search boundary is locked within the spatial bounding box of the target conflict region, and the objective function is locally refined iteratively solved to perform fine-grained avoidance optimization of the conflict region. After the convergence condition is met, the second trajectory sub-solution is obtained. In the iterative training of these two stages, the minimum value of each target is found from the mixed population as the ideal point, and the particles are encouraged to move towards the ideal point. The position vector code and velocity vector code of the corresponding particles are updated through the ideal point. The updated position vector and the updated velocity vector are concatenated and the concatenated vector is used as the target vector. Based on the target vector, the predetermined sorting segment in the first trajectory sub-solution is extracted, that is, the motion trajectory slice outside the target conflict area and undisturbed is extracted from the first trajectory sub-solution to preserve the globally optimal conflict-free path; the sorting segment to be optimized corresponding to the target conflict area in the second trajectory sub-solution is extracted, that is, the local avoidance trajectory slice within the fine-grained time window and specifically used to avoid the conflict area is extracted from the second trajectory sub-solution to preserve the locally optimal conflict resolution path; by applying particle swarm velocity constraints, the endpoint state vector of the predetermined sorting segment and the starting state vector of the sorting segment to be optimized are extracted and used as the velocity boundary of the particle swarm; a smooth transition segment is generated by B-spline, the extracted sorting segments are spliced ​​and merged into the sorting motion trajectory, and the objective function is updated with weights; After completing a single sorting action, the process includes: collecting air pressure feedback data through an air pressure sensor and comparing it with the robot's configuration status. Specifically, the air pressure feedback data is compared with a preset air pressure feedback threshold in the robot's configuration status. If the air pressure feedback data is less than the preset air pressure feedback threshold, it is determined that the air pressure feedback data indicates effective negative pressure, indicating sorting failure and triggering a secondary reverse air blowing command to update the backfill fault tolerance execution result. At the same time, the vibration acceleration of the current die-cutting station is retrieved, and structured encrypted log data is output to the server through encrypted private key signing. This process is repeated until the preset air pressure feedback threshold is reached. For example: Suppose that the air pressure sensor installed in the air path of each suction cup collects air pressure data at a frequency of 100Hz and compares it with the air pressure feedback threshold in the robot configuration. If the air pressure feedback data is less than the air pressure feedback threshold, the sorting is determined to have failed, triggering a second reverse air blowing command, and updating the fault-tolerant execution result to: sorting failed, second air blowing has been performed. At the same time, the vibration acceleration data of the current die-cutting station is retrieved. The log data is signed using the device's unique RSA-256 encryption private key, and structured encrypted log data containing task number, defect level, air pressure data, and execution result is output to the server. This process is repeated until the air pressure feedback data is greater than or equal to the preset air pressure feedback threshold.

[0022] Example 2: This embodiment of the invention provides an industrial robot quality inspection system based on industrial vision; the system includes: a parameter construction module, a mask analysis module, and an optimized sorting module, and the parameter construction module, the mask analysis module, and the optimized sorting module are communicatively connected; Parameter construction module: Alternately trigger the first polarization light source and the second polarization light source to acquire the first polarization image and the second polarization image of the die-cut part under test; determine the relative displacement vector of the first die-cutting station and the second die-cutting station by real-time pulse counting, and construct a parameter vector matrix by combining the first polarization image and the second polarization image; Mask Analysis Module: Based on the parameter vector matrix, a constraint rule engine is introduced, and the robot configuration state is jointly input into the preset mask analysis model. Through cross-attention matching, the defect level and target conflict area of ​​the die-cut part to be tested are identified. According to the target conflict area, the sorting action is executed. By analyzing the suction cup hardware code and the spatial position of the die-cut part to be sorted, the sorting scheduling topology is constructed, and the predetermined sorting path segment is retained as the background super node to generate the sorting action trajectory. Optimize the sorting module: Mark a sorting conflict event based on the target conflict area, optimize the sorting action trajectory, optimize the sorting conflict event synchronously, generate trajectory sub-solutions through an asynchronous dual time window judgment mechanism, and only backfill the fault-tolerant execution result data that has passed the verification to the main control log.

[0023] In the application, the various formulas mentioned are all calculated by removing dimensions and taking their numerical values. The formulas are derived from the most recent real-world situation by collecting a large amount of data and conducting software simulations. The formulas are set by those skilled in the art according to the actual situation.

[0024] The above embodiments can be implemented, in whole or in part, by software, hardware, firmware, or any other combination thereof. When implemented in software, the above embodiments can be implemented, in whole or in part, as a computer program product. Those skilled in the art will recognize that the units and algorithm steps of the various examples described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are implemented in hardware or software depends on the specific application and design constraints of the technical solution.

[0025] The units described as separate components may or may not be physically separate. The components shown as units may or may not be physical units; they may be located in one place or distributed across multiple network units. Some or all of the units can be selected to achieve the purpose of this embodiment, depending on actual needs.

[0026] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application.

Claims

1. A quality inspection method for industrial robots based on industrial vision, characterized in that, The method includes: Alternately trigger the first polarization light source and the second polarization light source to acquire the first polarization image and the second polarization image of the die-cut part under test; determine the relative displacement vector of the first die-cutting station and the second die-cutting station by counting the pulses triggered in real time, and construct a parameter vector matrix by combining the first polarization image and the second polarization image; Based on the parameter vector matrix, a constraint rule engine is introduced, and the robot configuration state is jointly input into the preset mask analysis model. The defect level and target conflict area of ​​the die-cut part to be tested are identified through cross-attention matching. According to the target conflict area, the sorting action is executed. By analyzing the suction cup hardware code and the spatial position of the die-cut part to be sorted, the sorting scheduling topology is constructed, and the predetermined sorting path segment is retained as the background super node to generate the sorting action trajectory. The target conflict area is defined as follows: under the condition of triggering a mild defect level, a rectangular grid cell feature set is obtained through rectangular meshing; under the condition of triggering a medium or high defect level, a circular cell feature set is obtained; the suction cup opening and closing sequence of two adjacent grid cells is extracted, and the grid corresponding to the sorting action is closed end to end through connected component partitioning and reconstruction to form the target conflict area. If the same suction cup needs to adsorb two different areas at the same time, or if two suction cups need to adsorb the same area at the same time, it will lead to sorting conflict. The background supernode is defined as follows: retrieve the standard sorting trajectory sequence of the non-target conflict area, obtain the standard sorting trajectory sequence containing multi-dimensional spatial location through discretization sampling, map it to the target conflict area, and perform spatiotemporal intersection operation: if the spatiotemporal intersection is empty, then the sorting trajectory sequence corresponding to the target conflict area is determined as the predetermined sorting segment, and the corresponding independent task node and topological edge are locked and clustered and encapsulated as a spatiotemporal background supernode; A sorting conflict event is marked based on the target conflict area, and the sorting action trajectory is optimized. The sorting conflict event is optimized synchronously, and a trajectory sub-solution is generated through an asynchronous dual time window judgment mechanism. Only the fault-tolerant execution result data that has passed the verification is backfilled to the main control log.

2. The industrial robot quality inspection method based on industrial vision according to claim 1, characterized in that, Acquire the first polarization image and the second polarization image of the die-cut part to be tested, including: In response to the first trigger pulse, the polarization CMOS photosensitive array is driven to be exposed for the first time, and the first polarization components of the die-cut part under test under the first polarization light source (horizontal, vertical, 45° oblique and 135° oblique) are captured and acquired simultaneously, and packaged to construct the first polarization image; wherein, the first polarization light source includes three bands of red, green and blue visible light. In response to the second trigger pulse, the polarization CMOS photosensitive array is driven to undergo a second exposure, simultaneously capturing and acquiring the second polarization components of the die-cut part under test under the second polarization light source, including horizontal, vertical, 45° oblique, and 135° oblique components, and packaging them to construct a second polarization image; wherein, the first polarization light source includes the near-infrared band.

3. The industrial robot quality inspection method based on industrial vision according to claim 1, characterized in that, Constructing the parameter vector matrix includes: Based on the first polarization image, the first feature component is separated and embedded into the first channel; Based on the second polarization image, the second feature vector is separated and embedded into the second channel; The relative displacement vector is embedded into the third channel, and the eigenvalues ​​of each channel are linearly mapped to the feature tensor space. Using the relative displacement vector, the first and second feature components corresponding to the overlapping areas of multiple die-cutting stations are affinely translated and recombined to construct the parameter vector matrix.

4. The industrial robot quality inspection method based on industrial vision according to claim 1, characterized in that, The constraint rule engine includes first constraint rules for visual analysis and second constraint rules for sorting execution: The first constraint rules include: ROI physical boundary, pixel grayscale gradient threshold, and lower limit of single trigger pulse window; The second set of constraints includes: suction cup geometric boundaries, throughput limits, parameter boundaries for multispectral alternation, and solenoid valve action response delay; The first and second constraint rules are converted into semantic constraint vectors, which are then combined with the robot configuration state to perform mask reconstruction of constraint boundary vectors. The parameter vector matrix is ​​used as the query matrix, and the constraint boundary vector is used as the key matrix. The dot product of the query matrix and the matrix is ​​performed to obtain feature activation values. After normalization, several sorting action mask matrices containing the suction cup opening and closing sequence are output. Among them, the robot configuration state includes at least the suction cup opening and closing sequence. The feature activation values ​​are compared with the preset activation value threshold to determine the defect level, including mild, moderate and severe defect levels, and the connected components of defective pixels are extracted from the sorting action mask matrix. Under the condition of triggering a mild defect level, the maximum Euclidean distance of the connected domain of the defective pixel is identified by rectangular meshing to determine the envelope half-width and envelope height of the sorting action, generate a rectangular boundary, and perform equal-interval interpolation sampling on the horizontal and vertical boundary axes of the rectangular boundary to obtain a rectangular mesh cell feature set with a total number of M0. Under the condition of triggering a moderate or high defect level, the geometric centroid of the connected domain of the defect pixel is identified by circular meshing. The envelope radius of the sorting action is determined based on the solenoid valve action response delay, a circular boundary is generated, and sampling is performed according to a preset step size to obtain a circular unit feature set with a total number of M1; where M0=M1.

5. The industrial robot quality inspection method based on industrial vision according to claim 4, characterized in that, Constructing a sorting scheduling topology includes: Based on the target conflict region, discrete feature extraction is performed on the mask matrix of each sorting action to obtain the suction cup code and the spatial position of the die-cut parts to be sorted. These are then combined in pairs to construct independent task nodes, and topological edges with time delay constraints are established to generate the sorting scheduling topology.

6. The industrial robot quality inspection method based on industrial vision according to claim 5, characterized in that, Time delay constraints include: Multiple spatiotemporal partitions are determined based on the suction cup opening and closing timing of the parameter vector matrix. A control index is established based on partitions of each time interval, and the control index of a sorting conflict event is recorded through the sorting scheduling topology.

7. The industrial robot quality inspection method based on industrial vision according to claim 1, characterized in that, Optimization of sorting scheduling topology trajectory includes: Construct an objective function that minimizes the total time spent on sorting tasks and the variance of suction cup load fluctuation. Use particle swarm optimization to perform sorting action trajectory planning in a preset first time window and a preset second time window respectively. Iterate the objective function to obtain the first trajectory sub-solution and the second trajectory sub-solution respectively. The predetermined sorting segment in the first trajectory sub-solution is extracted, and the sorting segment to be optimized corresponding to the target conflict area in the second trajectory subset is extracted. By applying particle swarm velocity constraints, a smooth transition segment is generated by B-spline. The extracted sorting segments are then spliced ​​and merged into the sorting action trajectory.

8. The industrial robot quality inspection method based on industrial vision according to claim 7, characterized in that, The first preset time window is a coarse-grained time window, and the second preset time window is a fine-grained time window.

9. The industrial robot quality inspection method based on industrial vision according to claim 1, characterized in that, After completing a single sorting operation, the following is included: Air pressure feedback data is collected by an air pressure sensor and compared with the robot's configuration status for analysis. If the air pressure feedback data indicates effective negative pressure, the sorting is determined to have failed and a secondary reverse air blowing command is triggered. The backfill fault tolerance execution result is updated, and the vibration acceleration of the current die-cutting station is retrieved. The structured encrypted log data is then output to the server through encrypted private key signature.

10. A system for applying the industrial vision-based quality inspection method for industrial robots as described in any one of claims 1-9, characterized in that, The system includes: The parameter construction module alternately triggers the first polarization light source and the second polarization light source to acquire the first polarization image and the second polarization image of the die-cut part under test; by counting the pulses triggered in real time, the relative displacement vector of the first die-cutting station and the second die-cutting station is determined, and a parameter vector matrix is ​​constructed by combining the first polarization image and the second polarization image; The mask analysis module, based on the parameter vector matrix, introduces a constraint rule engine and combines the robot configuration state with the input to the preset mask analysis model. It identifies the defect level and target conflict area of ​​the die-cut part to be tested through cross-attention matching. Based on the target conflict area, it performs sorting actions. By analyzing the suction cup hardware code and the spatial position of the die-cut part to be sorted, it constructs a sorting scheduling topology and retains the predetermined sorting path segment as background super nodes to generate sorting action trajectories. The target conflict area is defined as follows: under the condition of triggering a mild defect level, a rectangular grid cell feature set is obtained through rectangular meshing; under the condition of triggering a medium or high defect level, a circular cell feature set is obtained; the suction cup opening and closing sequence of two adjacent grid cells is extracted, and the grid corresponding to the sorting action is closed end to end through connected component partitioning and reconstruction to form the target conflict area. If the same suction cup needs to adsorb two different areas at the same time, or if two suction cups need to adsorb the same area at the same time, it will lead to sorting conflict. The background supernode is defined as follows: retrieve the standard sorting trajectory sequence of the non-target conflict area, obtain the standard sorting trajectory sequence containing multi-dimensional spatial location through discretization sampling, map it to the target conflict area, and perform spatiotemporal intersection operation: if the spatiotemporal intersection is empty, then the sorting trajectory sequence corresponding to the target conflict area is determined as the predetermined sorting segment, and the corresponding independent task node and topological edge are locked and clustered and encapsulated as a spatiotemporal background supernode; The sorting module is optimized by marking a sorting conflict event based on the target conflict area and optimizing the sorting action trajectory. The sorting conflict event is optimized synchronously, and a trajectory sub-solution is generated through an asynchronous dual-time window judgment mechanism. Only the fault-tolerant execution result data that has passed the verification is backfilled to the main control log.

Images