Group visual robot quality inspection method based on multi-agent reinforcement learning

Abstract

This invention relates to the technical field of industrial machine vision and collaborative control of swarm robots, and particularly to a quality inspection method for swarm vision robots based on multi-agent reinforcement learning. The method includes: acquiring images of the workpiece to be inspected at a preset sampling period, and writing robot pose identifiers and camera imaging parameter identifiers into each frame of the image to form a quality inspection observation set; performing N forward inferences on the quality inspection observation set under a defect detection network with a random deactivation layer, generating uncertainty prediction values ​​using the defect confidence variance, and calculating a risk score matrix to determine a risk region set; inputting the risk score matrix, the risk region set, and the robot pose identifiers into a multi-agent reinforcement learning model to output a candidate observation action set; acquiring supplementary images based on the candidate observation action set and updating the risk score matrix using weighted voting based on the uncertainty prediction values; writing risk regions that meet the threshold into a review task set and rolling them into the next sampling period.

Description

Technical Field

[0001] This invention relates to the technical field of industrial machine vision and collaborative control of swarm robots, and in particular to a quality inspection method for swarm vision robots based on multi-agent reinforcement learning. Background Technology

[0002] With the advancement of intelligent manufacturing and flexible production lines, machine vision quality inspection for appearance defects, assembly deviations, and surface flaws is gradually evolving from offline sampling inspection with a single camera and fixed workstation to process quality control with multiple cameras / multiple robots and online closed loops. Especially in scenarios involving multiple varieties and small batches, complex reflective materials, and obstructed structural components, the quality inspection system needs to effectively cover high-risk areas within a limited cycle time, while maintaining consistency and traceability in judgments under multiple perspectives, multiple exposures, and multiple motion postures. Current practices often use the confidence level of the depth detection network output as the basis for re-inspection, or repeat shooting through fixed inspection routes to compensate for the risk of missed detections. However, these strategies often fail to establish a calculable coupling relationship between imaging uncertainty and robot observation actions: on the one hand, confidence level does not equal reliability, and high-confidence misjudgments are not uncommon under complex textures, strong reflections, and domain offset conditions; on the other hand, full-scale repeated inspections significantly increase the acquisition and inference load, causing production line cycle time fluctuations and resource congestion, and concurrent observation by multiple robots is prone to problems of overlapping coverage and gaps in key areas, making it difficult to improve the missed detection rate and consistency simultaneously.

[0003] US10713769B2 discloses an active learning framework for training a defect classifier. It calculates the uncertainty of sample data points generated by the imaging subsystem and selects more informative data points for annotation and training, thereby reducing the cost of training data acquisition and improving the efficiency of classifier iteration. However, its core focus is on data selection and annotation loop in the training phase. It does not further map the uncertainty quantification results into executable observation action allocation, nor does it involve the organization of risk areas and online review scheduling under multi-robot collaborative coverage.

[0004] US20190294149A1 discloses a method and system for estimating the reliability / uncertainty of supervised learner output decisions, and establishes a correlation between uncertainty and exploratory behavior in autonomous platform control tasks to improve the credibility of control output; however, it is not a multi-perspective evidence fusion and risk area rolling review for industrial defect quality inspection, and it also lacks structured risk scoring matrices, risk area sets, candidate observation action sets, and constraint mechanisms for high-confidence conflicts among multiple robots.

[0005] Given the common problems of existing group visual quality inspection technologies, such as the difficulty in representing reliability with confidence levels, increased cycle time and computational overhead due to repeated acquisition, and the lack of action scheduling and consistency suppression for high-risk areas in multi-robot collaboration, this invention proposes a group visual robot quality inspection method based on multi-agent reinforcement learning. This method constructs a quality inspection observation set by writing robot pose identifiers and camera imaging parameter identifiers into workpiece images within a preset sampling period. Then, a defect detection network with a random deactivation layer performs N forward inferences and generates uncertainty prediction values ​​based on defect confidence variance. A risk scoring matrix is ​​constructed by combining defect confidence and uncertainty prediction values, and a risk region set is determined. The risk scoring matrix, risk region set, and robot pose identifiers are then input into a centrally trained, distributed execution multi-agent reinforcement learning model. Under risk constraint thresholds and high-confidence conflict penalty constraints, a candidate observation action set is output. The risk scoring matrix is ​​updated by weighted voting and fusion using uncertainty prediction values. Risk regions that meet the threshold conditions are written into the review task set and rolled into the candidate observation action set for the next sampling period. This allows for calculable resource scheduling and consistency improvement for missed detections and misjudgments. Summary of the Invention

[0006] The purpose of this section is to outline some aspects of the embodiments of the present invention and to briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section, as well as in the abstract and title of the present application, to avoid obscuring the purpose of this section, the abstract and title of the invention. Such simplifications or omissions shall not be used to limit the scope of the present invention.

[0007] In view of the aforementioned existing problems, the present invention is proposed.

[0008] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0009] As a preferred embodiment of the swarm visual robot quality inspection method based on multi-agent reinforcement learning described in this invention, wherein: under a preset sampling period, several swarm visual robots acquire images of the workpiece to be inspected through an airborne industrial camera, and write robot pose identifier and camera imaging parameter identifier for each frame of image to obtain a quality inspection observation set;

[0010] The quality inspection observation set is input into a defect detection network containing a random deactivation layer for N forward inferences. An uncertainty prediction value is generated by the variance of the N defect confidence outputs. A risk score matrix is ​​calculated based on the defect confidence and the uncertainty prediction value to determine the risk area set.

[0011] The risk scoring matrix, the risk region set, and the robot pose identifier are input into a multi-agent reinforcement learning model that is trained and distributed in a centralized manner. Under the risk constraint threshold and the high confidence conflict penalty constraint, the model outputs a set of candidate observation actions.

[0012] Based on the candidate observation action set, the group of visual robots is controlled to collect supplementary images, and the risk score matrix is ​​updated by using a weighted voting strategy with uncertainty prediction value as the weight. Risk areas with updated risk scores greater than or equal to the first risk threshold and uncertainty prediction values ​​greater than or equal to the first uncertainty threshold are written into the review task set and included in the candidate observation action set of the next sampling period.

[0013] The beneficial effects of this invention are as follows: This invention acquires images of the workpiece to be inspected and writes robot pose identifiers and camera imaging parameter identifiers into each frame of the image to form a quality inspection observation set, reducing missed detections and duplicate acquisitions caused by mismatched observations from multiple robots; it performs N forward inferences through a defect detection network containing a random deactivation layer and generates uncertainty prediction values ​​based on the defect confidence variance, and constructs a risk scoring matrix and a risk region set based on the defect confidence, transforming the risk of misjudgment from a single confidence level into a quantifiable regional risk measure; by inputting the risk scoring matrix, risk region set, and robot pose identifiers into a centralized training distribution, the invention enables the execution of... A multi-agent reinforcement learning model is used, which introduces risk constraint thresholds and high-confidence conflict penalties to form a mechanism that prioritizes coverage of high-risk areas and suppresses high-confidence disagreement judgments, thereby reducing the propagation of false detections caused by inconsistent group decisions. By supplementing image acquisition and updating the risk scoring matrix with weighted voting based on uncertainty prediction values, risk areas that meet the threshold conditions are rolled into the review task set and included in the next sampling cycle action generation. This improves the review efficiency and detection stability of difficult defects without increasing the full-scale repeated detection, thereby reducing the false negative rate and improving quality inspection consistency and production line cycle stability. Attached Figure Description

[0014] To more clearly illustrate the technical solutions of the embodiments of the present invention, the drawings used in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort. Wherein:

[0015] Figure 1 This is a flowchart illustrating the swarm visual robot quality inspection method based on multi-agent reinforcement learning as shown in this invention.

[0016] Figure 2 This is a schematic diagram illustrating the relationship between the set of verification tasks, the set of candidate supplementary observation areas, and the exclusion items when the sampling period is 310, as shown in this invention.

[0017] Figure 3 This is a schematic diagram illustrating the merging and deduplication of samples during sampling period 311 to form a target risk region set, as shown in this invention.

[0018] Figure 4 This is a schematic diagram of the high-confidence conflict penalty triggering and diversion results shown in this invention. Detailed Implementation

[0019] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0020] Based on the embodiments of this invention, all other embodiments obtained by those skilled in the art without inventive effort should fall within the scope of protection of this invention.

[0021] Many specific details are set forth in the following description in order to provide a full understanding of the invention. However, the invention may also be practiced in other ways different from those described herein, and those skilled in the art can make similar extensions without departing from the spirit of the invention. Therefore, the invention is not limited to the specific embodiments disclosed below.

[0022] In a preferred embodiment, this embodiment provides a swarm visual robot quality inspection method based on multi-agent reinforcement learning. This method uses a preset sampling period as the data organization unit, constructing a four-element record by combining workpiece image frames, robot pose identifiers, camera imaging parameter identifiers, and sampling period indexes. After consistency verification, these are merged to form a quality inspection observation set. Based on this, multiple forward inferences are performed on the quality inspection observation set using a defect detection network with a random deactivation layer to generate defect confidence and uncertainty prediction values, thereby constructing a risk scoring matrix and a risk region set. The risk scoring matrix, risk region set, and robot pose identifiers are then input into a centrally trained and distributed multi-agent reinforcement learning model to obtain a candidate observation action set. Subsequently, the swarm visual robot is controlled to collect supplementary images according to the candidate observation action set, and the risk scoring matrix is ​​fused and updated to form a review task set and a candidate supplementary observation region set. These two are then merged to obtain the target risk region set for the next sampling period, thus entering the next round of candidate observation action generation.

[0023] According to an embodiment of the present invention, in combination Figure 1 The flowchart shown illustrates a swarm visual robot quality inspection method based on multi-agent reinforcement learning, which specifically includes the following steps:

[0024] S1. Under a preset sampling period, several groups of vision robots acquire images of the workpiece to be inspected through an onboard industrial camera, and write robot pose identifiers and camera imaging parameter identifiers into each frame of the image to obtain a quality inspection observation set. It should be noted that in this step:

[0025] S1.1 At the beginning of each preset sampling period, each group of vision robots generates a sampling period index and writes the sampling period index into the record header information corresponding to the image of the workpiece to be inspected collected in this period.

[0026] Specifically, the preset sampling period is preferably 100 milliseconds, so that exposure triggering, pose sampling and imaging parameter reading are within the same time window and have stable alignment granularity; the sampling period index is preferably generated by a combination of the period start timestamp and robot identifier, and the frame number and camera identifier are written in the recording header information at the same time, so that the image sources of different robots and different cameras can still be distinguished under parallel acquisition conditions.

[0027] As an example, when the preset sampling period is 100 milliseconds, multiple workpiece image frames acquired within the same 100-millisecond time window are written to the same sampling period index; when entering the next time window, the sampling period index is updated once.

[0028] S1.2. Each group of visual robots triggers the airborne industrial camera to expose and acquire workpiece image frames within the time window corresponding to the sampling period index, and the robot motion controller reads the current pose parameters under the same sampling period index to generate robot pose identifier.

[0029] It should be noted that the exposure triggering conditions include at least one of the following: workpiece arrival signal, conveyor belt position count reaching a preset value, and robot end effector reaching a preset observation pose; in scenarios where there is relative motion between the workpiece and the robot, the triggering conditions further include a state item where the end effector speed is lower than a preset speed threshold, such as an end effector speed of less than 10 mm / s, in order to reduce the impact of motion blur on defect confidence fluctuations during exposure.

[0030] Specifically, the robot pose identifier includes the robot identifier, pose sampling timestamp, end-effector position parameters, end-effector posture parameters, and pose source identifier; among which the pose source identifier indicates that the pose is obtained by encoder calculation, inertial fusion, or visual fusion, so as to distinguish the time delay characteristics of different sources during consistency verification.

[0031] S1.3. Each group of visual robots reads the camera imaging parameters of the airborne industrial camera under the same sampling period index to generate camera imaging parameter identifiers, and binds the camera imaging parameter identifiers and robot pose identifiers to the workpiece image frames according to the frame number.

[0032] The camera imaging parameter identifier should include at least: camera identifier, imaging parameter version number, parameter effective start time stamp, parameter effective end time stamp, exposure time, gain and white balance mode identifier; in scenarios where lens zoom or aperture adjustment is present, the camera imaging parameter identifier should further include focal length and aperture settings.

[0033] In a preferred embodiment, the binding method includes: establishing an index record with the frame number as the key, writing the workpiece image frame, robot pose identifier, camera imaging parameter identifier, and sampling period index into the same quaternary record, and caching the quaternary record to a local queue and then merging and uploading it according to the sampling period index, thereby reducing the impact of asynchronous writing caused by network jitter on subsequent merging.

[0034] S1.4 Perform consistency verification on the four-element records of workpiece image frames, robot pose identifiers, camera imaging parameter identifiers, and sampling period indexes, and merge them according to the sampling period index to obtain the quality inspection observation set.

[0035] In a preferred embodiment, the consistency verification method includes the following three items, which are determined sequentially:

[0036] Sampling period attribution verification: Read the sampling period index in the quaternary record and verify whether the exposure timestamp of the workpiece image frame falls into the time window corresponding to the sampling period index. In this embodiment, the attribution rule of left-closed and right-open time window is preferred. Those that do not fall into the time window are marked as cross-period abnormal records.

[0037] Pose timing matching verification: For quaternary records not marked as cross-cycle abnormal records, calculate the absolute value of the time difference between the sampling timestamp of the robot pose identifier and the exposure timestamp of the workpiece image frame, and compare it with a preset time difference threshold; records that are greater than or equal to the preset time difference threshold are marked as pose mismatch records; the preset time difference threshold is related to the pose sampling frequency and the camera exposure delay, and in this embodiment, it is preferably 5 milliseconds to 20 milliseconds; for example, when the pose sampling frequency is 200 Hz and the camera hard trigger exposure delay is 2 milliseconds, the preset time difference threshold is selected as 10 milliseconds;

[0038] Image parameter effective range verification: For quaternary records not marked as pose mismatch records, verify whether the effective range of the camera imaging parameters covers the exposure timestamp of the workpiece image frame; those not covered are marked as parameter mismatch records.

[0039] After the consistency verification is completed, the four-element records that are not marked as cross-cycle abnormal records, pose mismatch records, or parameter mismatch records are grouped into a quality inspection observation set according to the sampling period index. At the same time, cross-cycle abnormal records, pose mismatch records, and parameter mismatch records are summarized into an abnormal record set. The abnormal record set preferably includes the sampling period index, robot identifier, camera identifier, frame number, and abnormal type, so as to provide a basis for the equipment status investigation of subsequent review tasks.

[0040] It should be noted that this embodiment establishes a unified attribution for multi-source asynchronous sampling data through sampling period indexing, and removes mismatched records by checking three aspects: sampling period attribution, pose timing matching, and effective range of imaging parameters. This ensures that the quality inspection observation set entering subsequent inference has consistency in the time and imaging condition dimensions, thereby giving the spatial index of the risk scoring matrix and the regional boundary of the risk area set a stable source. Compared with the method of relying solely on single timestamp splicing or simply caching and merging, this embodiment provides executable verification rules and abnormal record handling methods, reducing risk area drift and multi-robot action allocation deviation caused by mismatch.

[0041] S2. Input the quality inspection observation set into a defect detection network containing a random deactivation layer and perform N forward inference iterations. Generate an uncertainty prediction value using the variance of the N defect confidence outputs. Calculate a risk scoring matrix based on the defect confidence and the uncertainty prediction value to determine the risk region set. Note that the following should be noted in this step:

[0042] S2.1 Normalize and crop the workpiece image frames in the quality inspection observation set according to the preset input size to obtain the image input tensor set, and send the image input tensor set frame by frame into the defect detection network containing a random deactivation layer.

[0043] Specifically, the preset input size is preferably 640×640, the normalization preferably maps the pixel values ​​to a fixed range, and retains the original exposure mode identifier for subsequent analysis of the source of uncertainty fluctuations; the cropping method preferably adopts regional cropping based on the workpiece positioning reference; when the workpiece positioning reference is unavailable, center cropping is used as an alternative.

[0044] As an example, when the original image is 1920×1200, the shorter side can be scaled to 640 and then cropped to obtain a 640×640 image input tensor.

[0045] S2.2 For each image input tensor in the image input tensor set, while keeping the weight parameters of the defect detection network unchanged, enable a random deactivation layer and index it according to the number of inferences. Perform forward computation N times sequentially (N is preferably 20 times) to obtain the corresponding N sets of defect output tensors.

[0046] S2.3 For each candidate defect target in the defect output tensor set, take the Softmax output of its classification branch as the defect confidence score, and form a defect confidence score sequence by indexing the defect confidence scores of the same candidate defect target under N forward calculations according to the number of inferences.

[0047] It should be noted that, in this embodiment, a candidate defect target matching rule is preferably adopted: In the first inference result, a candidate defect target identifier is generated for each candidate defect target, and the candidate defect target identifier, its bounding box, and the defect category index are written into the candidate defect target record; in subsequent inference results, the consistency between the spatial overlap relationship of the bounding box with the candidate defect target record and the defect category index is used as the matching condition, and candidate defect targets that meet the matching condition are merged into the corresponding candidate defect target identifier; when multiple candidate defect targets meet the matching condition at the same time, the one with the largest spatial overlap is selected as the matching object; when there is no matching object, this inference is recorded as the missing inference number of the candidate defect target and written into the missing marker.

[0048] After matching and merging N inferences, a defect confidence sequence is formed for each candidate defect target. This defect confidence sequence is arranged sequentially according to the inference count index. To avoid incomparable uncertainty prediction values ​​due to inconsistent sequence lengths, this embodiment adopts a missing completion rule: for positions marked as missing inference counts, the defect confidence is filled into a preset lower bound value, and the position is kept marked as missing. The preset lower bound value is 0 or a fixed small value below the first confidence threshold, such as 0.05, to reflect the state that the candidate defect target has not been stably output in this inference. If the proportion of missing inference counts for a candidate defect target is greater than the preset missing proportion threshold, the candidate defect target is marked as an unstable target and written into the unstable mark field so that the target can be processed separately or its impact reduced during the subsequent risk scoring matrix construction. The preset missing proportion threshold is 0.40.

[0049] S2.4 Calculate the sample variance of the defect confidence sequence to obtain the uncertainty prediction value corresponding to the candidate defect target, and summarize the uncertainty prediction values ​​of each candidate defect target in the same workpiece image frame into the uncertainty prediction value set of that frame.

[0050] Preferably, each record in the frame uncertainty prediction value set carries simultaneously a location box index, a defect category index, a defect confidence level, and a candidate defect target uncertainty prediction value.

[0051] Specifically, the sample variance of the defect confidence sequence is calculated, including: first, calculating the mean of the defect confidence sequence to obtain the mean defect confidence; then, summing the squares of the deviations between the defect confidence at each inference number position in the defect confidence sequence and the mean defect confidence, and normalizing according to the denominator rule of the sample variance to obtain the predicted uncertainty value of the candidate defect target; to make the candidate defect targets of different defect categories or different confidence levels comparable, the predicted uncertainty value of the candidate defect target is further constrained by range: when the calculation result is less than 0, it is recorded as 0; when the calculation result is greater than a preset upper bound, it is recorded as a preset upper bound; the preset upper bound is determined based on N and the range of defect confidence values, for example, 0.25 is selected.

[0052] As an example, when N is 20, in the defect confidence sequence formed for a certain candidate defect target, if most inference outputs are concentrated around 0.80 and fluctuate little, the predicted uncertainty value of the candidate defect target corresponds to a small value; if the sequence shows obvious fluctuations such as 0.90, 0.60, 0.75 at different inference times or has multiple missing additions, the predicted uncertainty value of the candidate defect target corresponds to a large value, and a missing or unstable mark is retained in the uncertainty record of the candidate defect target, thereby providing a traceable basis for the subsequent generation of risk area set and the determination of review task set.

[0053] S2.5. Divide each workpiece image frame into a set of regional units according to a preset grid size, and associate the defect confidence and uncertainty prediction value of each candidate defect target in the frame with the index of the regional unit it covers.

[0054] As an example, the preset grid size is preferably 16×16 pixels.

[0055] Furthermore, the association rules include: determining the regional cell index covered by the candidate defect target location box as the coverage regional cell index set, writing the defect confidence and uncertainty prediction value of the candidate defect target into the record of the corresponding regional cell index, and writing the candidate defect target identifier to support subsequent multi-target convergence within the region.

[0056] S2.6 For each regional unit index, calculate the risk score of the regional unit according to the preset weight coefficient. The risk score consists of the weighted sum and product of the defect confidence and uncertainty prediction value of the corresponding candidate defect target. Arrange the risk scores of each regional unit according to the spatial index to obtain the risk score matrix.

[0057] For example, the mathematical expression of this risk scoring matrix is ​​as follows:

[0058]

[0059] Where R is the risk score of the regional unit, C is the defect confidence level of the candidate defect target falling into the regional unit, and U is the predicted uncertainty value of the candidate defect target corresponding to the defect confidence level. This is a weighting factor for the confidence level of the defect, such as 0.6; This is a weighting coefficient for the predicted uncertainty value of the candidate defect target, such as 0.2; , which is the weighting factor of the product term, such as 0.2; when C is high and U is also high, the product term makes the risk score sensitive to situations with high confidence but fluctuating output, thus making subsequent risk areas more likely to be included in the review.

[0060] S2.7 Select the regional unit indexes with risk scores greater than or equal to the first risk threshold on the risk scoring matrix, and merge adjacent regional unit indices according to the preset connectivity rules to obtain a set of risk regions.

[0061] The preset connectivity rules include: based on the two-dimensional grid coordinates of the region cell index, the adjacency relationship is determined by four-neighbor connectivity; when two region cell indices satisfy the adjacency relationship and both of their risk scores are greater than or equal to the first risk threshold, the two region cell indices are classified into the same connected component; for each connected component, the number of region cells is calculated and compared with the preset minimum number threshold, and the connected components with the number of region cells greater than or equal to the preset minimum number threshold are determined as risk regions in the risk region set.

[0062] It should be noted that the first risk threshold is determined based on the higher quantile of the risk score distribution of normal samples to reduce the probability of normal background noise triggering risk areas; for example, the first risk threshold is set to 0.60; the number of region units is obtained by counting the region unit indices within the connected components; the preset minimum number threshold is determined based on the minimum identifiable area of ​​the defect, for example, it can be set to 9 when the grid size is 16×16 pixels and the minimum defect size is about 40×40 pixels, in order to exclude isolated noise units.

[0063] Preferably, in this embodiment, the defect confidence sequence is obtained through multiple forward inferences under random deactivation conditions, and the sample variance is used to characterize the uncertainty prediction value of the candidate defect target. The output fluctuation is introduced into the risk scoring matrix as an independent dimension of risk assessment. Then, a risk region set is obtained through gridded mapping and connected component merging, so that the risk region set has spatial continuity and area constraints, thereby providing schedulable regional-level risk input for subsequent multi-agent reinforcement learning models and reducing the impact of isolated noise units on action allocation.

[0064] S3. Input the risk score matrix, risk region set, and robot pose identifier into a multi-agent reinforcement learning model with centralized training and distributed execution. Output a set of candidate observation actions under risk constraint thresholds and high-confidence conflict penalty constraints. Note that the following should be noted in this step:

[0065] S3.1. In each preset sampling period, the risk scoring matrix and the risk region set are encoded into a global risk state vector by spatial index, and the robot pose identifiers of each group of visual robots are concatenated to the global risk state vector to obtain the joint state input vector.

[0066] In this embodiment, to avoid changes in the dimension of the joint state input vector due to missing fields between sampling periods, a field template for the joint state input vector is predefined. The field template includes at least: a risk score matrix encoding segment, a risk region set encoding segment, and a robot pose encoding segment, and each field in the field template has a fixed position and a fixed length.

[0067] In a preferred embodiment, the encoding of the risk scoring matrix preferably adopts a rule of fixed spatial index, fixed traversal order, and fixed numerical range. Specifically, the two-dimensional grid coordinate range of the risk scoring matrix is ​​predetermined, and traversal is performed in a row-first manner, that is, traversing in ascending order of row coordinates, and within each row, traversing in ascending order of column coordinates. For each matrix element obtained by traversal, the risk score of its corresponding region cell index is read and written into the risk scoring matrix encoding segment according to a preset quantization rule. The quantization rule preferably constrains the risk score within a preset range and retains it with a fixed precision, such as retaining three decimal places. When the risk scoring matrix has invalid region cell indices due to field clipping, a preset placeholder value is filled into the position of the invalid region cell index and an invalid mark is written to ensure that the length of the risk scoring matrix encoding segment remains unchanged. Through this rule, the risk scoring matrix is ​​mapped to a one-dimensional sequence according to the spatial index and written into the risk scoring matrix encoding segment, so that the risk scoring matrix encoding segment has a fixed length and fixed index meaning in any sampling period.

[0068] In a preferred embodiment, the risk area set coding segment preferably adopts a coding rule with a fixed upper limit on the number of risk areas, fixed fields within the area, and a fixed sorting rule. Specifically, the upper limit on the number of risk areas is preset, and the risk area set coding segment is divided into several area slots, with each slot corresponding to a coding field group for one risk area. When the number of risk areas is less than the upper limit, a preset placeholder value is filled into the empty slot and an empty slot mark is written. When the number of risk areas exceeds the upper limit, the first few risk areas are selected according to a preset filtering rule and written into the slot. The preset filtering rule is preferably sorted from highest to lowest based on the maximum risk score within the risk area and then the first few items are selected. The coding field group for each risk area includes at least: risk area identifier, number of risk area area units, center grid coordinates of the risk area, risk... The system includes fields describing the region boundary range, the maximum risk score of the risk region, and the average risk score of the risk region. The center grid coordinates of the risk region are obtained by statistically analyzing the grid coordinates of the corresponding region unit index set. The boundary range description field is represented by the minimum row coordinate, maximum row coordinate, minimum column coordinate, and maximum column coordinate. The sorting rules for the risk region set are fixed, preferably using a three-level sorting rule: descending order of maximum risk score, descending order of region unit quantity, and ascending order of risk region identifier. This ensures the stability of the writing order of the risk region set coding segment under different sampling periods. Through this rule, the risk region set coding segment has fixed slots, fixed fields, and a fixed writing order, avoiding changes in the dimension of the joint state input vector caused by changes in the number of risk region sets.

[0069] In a preferred embodiment, before concatenating the robot pose identifiers of each group of visual robots to the global risk state vector, robot sorting rules and missing completion rules are first determined to ensure that the order of robot pose encoding segments is fixed. Specifically, the upper limit of the number of robots allowed by this sampling system is pre-registered, and a fixed robot sequence is obtained by sorting the robots in ascending order of robot identifiers using robot identifiers as sorting keys. For each robot in the fixed robot sequence, its robot pose identifier under the current sampling period index is read, and the robot pose identifier is written into the robot pose encoding segment according to the field template. The field template includes at least: robot identifier, pose sampling timestamp, end position parameter, end pose parameter, and pose source identifier. When a robot does not have a robot pose identifier that has passed the consistency check under the current sampling period index, a placeholder pose record is written using the missing completion rule, and a pose missing mark is written. The placeholder pose record is taken from the valid pose record of the robot in the previous sampling period. If it is also missing in the previous sampling period, the system initial pose record is taken. Through this rule, the robot pose encoding segment can be written according to the fixed robot sequence in each sampling period, and the field length is fixed.

[0070] Furthermore, after writing the risk scoring matrix encoding segment, risk region set encoding segment, and robot pose encoding segment, the three are concatenated in the order of the field template to obtain the joint state input vector. To ensure that this joint state input vector can be directly used as input for a multi-agent reinforcement learning model that is centrally trained and distributed for execution, this implementation method preferably performs a consistency check on the joint state input vector: checking whether its total length is equal to the length defined by the field template, checking whether the number of slot markers in the risk region set encoding segment is consistent with the current number of risk regions, and checking whether the missing markers in the robot pose encoding segment are consistent with the consistency check result. Sampling period indexes that fail the check are preferably written into the state anomaly record and do not enter the action generation step.

[0071] As an example, in a scenario with a preset grid of 40×40 risk scoring matrix, the risk scoring matrix encoding segment is fixed at 1600 elements; the upper limit of the number of risk regions is set to 8, so the risk region set encoding segment is fixed at 8 region slots, with each slot containing a fixed group of fields; the upper limit of the number of robots is set to 6, so the robot pose encoding segment is fixed at 6 groups of pose fields; if 3 risk regions are detected in the current sampling period, the first 3 region slots are written with the corresponding risk region field group, and the last 5 region slots are marked as empty slots; if the pose record of the 4th robot in this period fails the consistency check, its pose field group is written with the pose of the previous period and a pose missing mark is set; the length and field position of the joint state input vector obtained in this way remain unchanged in each sampling period, which facilitates the multi-agent reinforcement learning model to interpret the meaning of the input in different sampling periods.

[0072] S3.2 Input the joint state input vector into the multi-agent reinforcement learning model that is trained and distributed in a centralized manner to obtain the action score sequence corresponding to each group of visual robots, and generate an initial candidate observation action set containing the observation pose increment and camera imaging parameter adjustment from the action score sequence.

[0073] As an example, the mathematical expression for the candidate action is:

[0074]

[0075] in, For the first Action candidates selected by a group of vision robots; For the first A set of preset actions for a group of vision robots; For multi-agent reinforcement learning models, the first The robot outputs a motion scoring function; The input vector is the joint state. These are action candidates from a preset action set.

[0076] It should be noted that the preset action set in this embodiment is formed by a combination of observation pose increment levels and camera imaging parameter adjustment levels. For example, the observation pose increment includes multiple distance adjustments along the workpiece normal direction, and the camera imaging parameter adjustment includes multiple switching of exposure time, gain and white balance mode. Each action in the initial candidate observation action set is preferably written with an action candidate identifier, robot identifier and action score to support subsequent constraint screening and priority calculation.

[0077] S3.3 Calculate the coverage increment of the risk area set for each item in the initial candidate observation action set, and remove the candidate observation actions whose coverage increment is less than the risk constraint threshold from the initial candidate observation action set to obtain the constrained candidate observation action set.

[0078] Specifically, the calculation of the coverage increment is based on the overlap relationship between the field of view coverage area corresponding to the candidate observation action and the risk area set: the field of view coverage area under the candidate observation action is obtained according to the observation pose increment and the camera imaging parameter adjustment amount, and the number of overlapping area units between the field of view coverage area and each risk area is calculated; the coverage increment is preferably the increment ratio of the number of overlapping area units to the number of risk area units, and the corresponding risk area identifier is recorded.

[0079] It should be noted that the risk constraint threshold is determined based on the minimum effective supplementary observation ratio, such as 0.25, to avoid low-return actions consuming execution resources.

[0080] S3.4 For multiple robot candidate observation actions involving the same risk area in the set of constrained candidate observation actions, calculate the high confidence conflict penalty based on the defect confidence corresponding to the risk area. The high confidence conflict penalty is jointly triggered by the defect confidence being greater than or equal to a preset high confidence threshold and the difference between the defect confidence of different robots being greater than or equal to a preset conflict difference threshold.

[0081] It should be noted that, in this embodiment, the risk area attribution determination is preferably adopted: when the overlap ratio between the field of view coverage area corresponding to the candidate observation action and a certain risk area is greater than or equal to a preset overlap ratio threshold, the candidate observation action is determined to involve the risk area. The preset overlap ratio threshold is 0.30 in example; the preset high confidence threshold is determined based on the false alarm tolerance of the defect category, such as 0.80; the preset conflict difference threshold is determined based on the multi-robot observation consistency requirements, such as 0.15; after triggering, a conflict penalty flag is written for the candidate observation action involving the risk area, and a penalty intensity level is written so as to reflect the impact of conflict in the action priority score.

[0082] S3.5 Calculate the action priority score for each candidate observation action in the constrained candidate observation action set. The action priority score is obtained by weighting the coverage increment and the high confidence conflict penalty. Based on the action priority score, select the candidate observation action corresponding to the maximum value for each group of vision robots and summarize to obtain the candidate observation action set.

[0083] As an example, the mathematical formula for action priority score is:

[0084]

[0085] Where P is the action priority score; To cover the incremental change; G represents the intensity of the high-confidence conflict penalty; To cover the incremental weighting coefficients; The conflict penalty weighting coefficient.

[0086] Furthermore, this embodiment divides the action priority score into different score ranges, each corresponding to a different priority and a different execution operation, specifically including:

[0087] (1) When the action priority score falls into the first score interval, it is determined as the first priority action; the first score interval is preferably greater than 0.70; for the first priority action, the action is directly selected into the candidate observation action set and issued for execution within the same sampling period; at the same time, the field of view coverage area corresponding to the action is marked as the priority coverage area, so as to write the first priority coverage mark in the coverage increment record of the risk area within the same period.

[0088] (2) When the action priority score falls into the second score range, it is determined as the second priority action; the second score range is preferably 0.40 to 0.70; for the second priority action, the action is selected into the candidate observation action set and issued for execution within the same sampling period; when the trajectory point sequence is generated, transition trajectory points are added to reduce attitude change, and when the camera imaging parameters are updated, the parameter effective start time stamp and parameter version number are written in a segmented effective manner to reduce exposure fluctuations caused by parameter switching.

[0089] (3) When the action priority score falls into the third score range, it is determined to be the third priority action; the third score range is preferably 0.20 to 0.40; for the third priority action, it is not directly issued for execution in the current sampling period, but is written into the candidate action cache set, and in the next sampling period, the candidate action cache set is merged with the newly generated initial candidate observation action set before participating in the coverage increment screening, so as to avoid low-yield actions from diverting the observation resources of this period.

[0090] (4) When the action priority score is lower than the fourth score threshold, it is determined to be the fourth priority action; the fourth score threshold is preferably less than 0.20; for the fourth priority action, it is marked as a low-return action and removed from the candidate action cache set. At the same time, the risk area identifier and coverage increment involved in the action are written in the low-return coverage record so that the level configuration of the preset action set can be revised later.

[0091] Preferably, in this embodiment, low-coverage-benefit actions are first eliminated by coverage increment constraints, so that the action set entering the conflict penalty and priority calculation has a clear regional coverage increment contribution; the high-confidence conflict penalty gives a differentiated mark for the case where multiple robots point to the same risk area at the same time and the confidence level is significantly different; the action priority level rule further maps the score to the operation rules of execution / caching / elimination in the same period, so that the action priority score runs through the execution-side action management, avoiding only staying at the level of ranking indicators.

[0092] S4. Based on the candidate observation action set, control several groups of visual robots to collect supplementary images, and use a weighted voting strategy with uncertainty prediction values ​​as weights to fuse and update the risk score matrix. Risk regions with updated risk scores greater than or equal to the first risk threshold and uncertainty prediction values ​​greater than or equal to the first uncertainty threshold are written into the review task set and included in the candidate observation action set for the next sampling period. Note that the following should be noted in this step:

[0093] S4.1. The candidate observation action set is split according to the group visual robot identifier to obtain the target candidate observation action corresponding to each group visual robot, and the target candidate observation action is parsed into the observation pose increment and the camera imaging parameter adjustment amount.

[0094] In this embodiment, the candidate observation action set is formed by step S3.5, and each candidate observation action record is stored using a fixed field structure, including: robot identifier, action candidate identifier, risk area identifier, action priority score, coverage increment, observation pose increment field, camera imaging parameter adjustment amount field, and action generation sampling period index; wherein, the observation pose increment field consists of a position increment component field + an attitude increment component field; the camera imaging parameter adjustment amount field consists of an exposure time adjustment field, a gain adjustment field, and a white balance mode adjustment field; when the camera supports zoom or aperture adjustment, the camera imaging parameter adjustment amount field further includes a focal length adjustment field and an aperture adjustment field.

[0095] It should be noted that, in this embodiment, the splitting of the candidate observation action set preferably adopts the rule of robot identifier grouping, sorting within the same group, and outputting group by group. Specifically, it includes: reading the robot identifier of each candidate observation action record in the candidate observation action set, merging candidate observation action records with the same robot identifier into the same action group; establishing a group index for each action group, the group index at least recording the robot identifier, the number of action records in the group, and the list of action candidate identifiers in the group; when there is an unregistered robot identifier in the candidate observation action set, the action record is written into the abnormal action record set, and the action candidate identifier and sampling period index are written into the abnormal action record set to avoid ambiguity when issuing actions later.

[0096] In this embodiment, the selection of target candidate observation actions from each action group preferably adopts a fixed priority rule to avoid instability in control behavior caused by changes in the selection strategy under different sampling periods. The selection rules are executed in the following order: Main sorting rule: sort by action priority score from high to low, and take the first ranked candidate observation action record as the target candidate observation action; Parallel elimination rule: when there are multiple candidate observation action records with the same action priority score or a difference less than a preset parallel difference threshold, it is preferable to perform a secondary sorting by coverage increment from high to low; if there are still ties in the secondary sorting, the first item is taken according to the preset order of action candidate identifiers (e.g., according to the lexicographical order of action candidate identifiers); Executability constraint rule: if the observation pose increment corresponding to the first ranked candidate observation action record exceeds the robot kinematic amplitude limit configuration in the current sampling period, the record is skipped and the next record is taken; the amplitude limit configuration includes the maximum position step size and the maximum attitude step size, and an amplitude limit elimination mark is written in the action record so that the action set design can be revised later; through this rule, a unique target candidate observation action can be output for each robot action group, thereby obtaining the target candidate observation action set corresponding to each group of visual robots.

[0097] Furthermore, in this embodiment, the target candidate observation action is parsed into observation pose increment and camera imaging parameter adjustment amount according to the rules of field reading, normalization, and forming a distribution structure. Specifically, this includes: reading the observation pose increment field in the target candidate observation action record and parsing it into position increment and attitude increment; the position increment is preferably recorded as a three-axis increment in the robot's working coordinate system, and the attitude increment is preferably recorded as an attitude three-parameter or equivalent attitude representation; to ensure that the subsequent trajectory generation has a unified reference system, the observation pose increment is uniformly converted to the same reference coordinate system used by the robot controller during parsing, and the reference coordinate system identifier is written into the parsing result; reading the camera imaging parameter adjustment amount field in the target candidate observation action record and parsing it into exposure time adjustment, gain adjustment, and white balance mode. Adjustments are made as follows: When focal length or aperture adjustment fields exist, they are parsed synchronously and the parsing results are written. To avoid discontinuous jumps in camera parameter updates, exposure time and gain are limited and truncated according to the camera's allowable range during parsing. White balance mode adjustments are validated against a preset mode enumeration table. Illegal modes are written to the parameter anomaly record and replaced with the default mode. The parsed observation pose increment and camera imaging parameter adjustment are encapsulated into a target action distribution record. The target action distribution record includes at least: robot identifier, action candidate identifier, sampling period index, observation pose increment, camera imaging parameter adjustment, risk area identifier, action priority score, and coverage increment. This target action distribution record serves as the input for step S4.2 to support trajectory point sequence generation and camera imaging parameter updates.

[0098] As an example, in the candidate observation action set, there are 4 candidate observation action records with robot identifier R1, and the action priority scores are 0.82, 0.65, 0.41 and 0.18 respectively. Then, according to the main sorting rule, the record with a score of 0.82 is selected as the target candidate observation action. The observation pose increment field is analyzed to obtain the position increment as a single step adjustment along the workpiece normal direction and the attitude increment as a small angle correction. The camera imaging parameter adjustment amount field is analyzed to obtain the adjustment amount of increasing the exposure time by one stop and decreasing the gain by one stop. The analysis results are written into the target action issuance record and the action candidate identifier and the risk area identifier are retained so as to realize the comparison and traceability of actions and observation records when adding the quality inspection observation set in the future.

[0099] S4.2 Generate a sequence of motion trajectory points based on the observed pose increment and send it to the motion controller. At the same time, update the camera imaging parameters of the airborne industrial camera based on the camera imaging parameter adjustment amount and generate the updated camera imaging parameter identifier.

[0100] In an optional implementation, the trajectory point sequence is generated by taking the current pose identifier corresponding to the pose as the starting pose, and superimposing the observed pose increments to obtain the target termination pose. A trajectory point sequence is generated between the starting pose and the target termination pose according to a preset interpolation rule. The interpolation rule includes linear interpolation and velocity constraints, and a desired arrival time stamp is written for each trajectory point.

[0101] The camera imaging parameter update is preferably performed by writing the start time stamp of the parameter effect when the camera control module confirms the effect, and generating the updated camera imaging parameter identifier to ensure that subsequent consistency verification has a basis for the parameter effect range.

[0102] S4.3 After each group of visual robots reaches the final pose of the motion trajectory point sequence, the onboard industrial camera is triggered to acquire supplementary images, and robot pose identifiers and camera imaging parameter identifiers are written into the supplementary images to obtain a supplementary quality inspection observation set.

[0103] It should be noted that the triggering conditions include pose stability determination, which includes the end-effector velocity being lower than a preset velocity threshold and the pose change rate being lower than a preset change rate threshold. The supplementary quality inspection observation set is merged into the quality inspection observation set according to the sampling period index, and each frame of the merged image is input into a defect detection network containing a random deactivation layer to perform N forward inferences to generate the corresponding defect confidence and candidate defect target uncertainty prediction value. The defect confidence and candidate defect target uncertainty prediction value are then associated with the region unit index corresponding to the risk region set. The risk region uncertainty prediction value is preferably obtained by taking the candidate defect target uncertainty prediction value falling into the risk region according to a preset convergence rule. An example of the convergence rule is to take the maximum value or take the weighted average value, and write the risk region uncertainty prediction value into the risk region record for threshold comparison.

[0104] S4.4 Merge the supplementary quality inspection observation set into the quality inspection observation set according to the sampling period index, and perform N forward inferences on each frame of the merged image input to the defect detection network containing the random deactivation layer to generate the corresponding defect confidence and uncertainty prediction value, and associate the defect confidence and uncertainty prediction value with the region unit index corresponding to the risk region set.

[0105] S4.5 For each risk region in the risk region set, aggregate the confidence levels of multiple defects falling into that risk region, and use the reciprocal of the sum of the uncertainty prediction value corresponding to each defect confidence level and the preset constant as the voting weight to obtain a weighted voting weight set.

[0106] It should be noted that the preset constant should preferably be a small positive number to avoid excessive voting weight.

[0107] S4.6. Based on the weighted voting weight set, the defect confidence is weighted and summed to obtain the fusion confidence. The fusion confidence is then combined with the uncertainty prediction value of the risk area according to the preset weight coefficient to calculate the updated risk score. This updated risk score replaces the risk score of the corresponding area cell index in the risk score matrix, resulting in the updated risk score matrix.

[0108] As an example, the updated risk score is:

[0109]

[0110]

[0111] in, The confidence level of integration for risk areas; The number of defects that fall into this risk area; For the first The voting weights corresponding to the confidence levels of each defect; For the first Confidence level of defects; Update the risk score for the risk area; This represents the predicted value for uncertainty in the risk area; The confidence weighting coefficient is used for fusion. The weighting coefficient for the predicted uncertainty value of the risk area; These are the weighting coefficients for the product terms.

[0112] S4.7 For each risk region in the updated risk scoring matrix, compare its updated risk score with the first risk threshold and its predicted uncertainty value with the first uncertainty threshold, and output a judgment indicator based on the comparison results:

[0113] When the updated risk score is greater than or equal to the first risk threshold and the predicted uncertainty value of the risk area is greater than or equal to the first uncertainty threshold, the first judgment identifier is output and the risk area is written into the review task set. The first judgment identifier is defined as review task identifier = RCHK-1, which means that the risk level reaches the threshold and the output fluctuation reaches the threshold. Correspondingly, the risk area identifier, the updated risk score, the predicted uncertainty value of the risk area, the set of robot identifiers involved, the set of corresponding image frame numbers, and the set of imaging parameter version numbers are written into the review task set as evidence items for subsequent review workstations or manual review. At the same time, the risk area is added to the target risk area set in the next sampling period to trigger the generation of further supplementary observation actions.

[0114] When the updated risk score is greater than or equal to the first risk threshold and the predicted uncertainty value of the risk area is less than the first uncertainty threshold, a second judgment identifier is output and the risk area is excluded from the review task set. The second judgment identifier is defined as stable defect identifier = STBL-2, which means that the risk level reaches the threshold and the output fluctuation is lower than the threshold. The corresponding risk area is written into the stable defect handling set. The stable defect handling set includes the risk area identifier, fusion confidence, updated risk score and corresponding workpiece batch identifier, and triggers the workpiece batch to enter the isolation or rework process in the subsequent process, so that the judgment and the quality inspection topic form a closed loop.

[0115] When the updated risk score is less than the first risk threshold and the predicted uncertainty value of the risk area is greater than or equal to the first uncertainty threshold, a third judgment identifier is output and the risk area is written into the candidate supplementary observation area set. The third judgment identifier is defined as the re-observation identifier = REOBS-3, which means that the risk level has not reached the threshold but the output fluctuation has reached the threshold. The risk area is written into the candidate supplementary observation area set and marked as needing to increase the observation evidence density. In the next sampling period, it is incorporated into the target risk area set to prompt the multi-agent reinforcement learning model to assign a better observation pose or more suitable imaging parameters to the area to reduce uncertainty.

[0116] When the updated risk score is less than the first risk threshold and the predicted uncertainty value of the risk area is less than the first uncertainty threshold, a fourth judgment flag is output and the risk area is simultaneously excluded from the review task set and the candidate supplementary observation area set. The fourth judgment flag is defined as pass flag = PASS-4, which means that the risk level and output fluctuation are both lower than the threshold. The risk area is written into the pass record set and the rolling supplementary observation allocation for the area is terminated to avoid duplicate occupation of observation resources.

[0117] S4.8 Merge the set of review tasks and the set of candidate supplementary observation regions to obtain the set of target risk regions for the next sampling period. Input the set of target risk regions and the robot pose identifier into a multi-agent reinforcement learning model that is trained and distributed to output the set of candidate observation actions for the next sampling period under the risk constraint threshold and the high confidence conflict penalty constraint.

[0118] It should be noted that the above-mentioned merging and refeeding makes the joint state input vector of the next sampling period spatially focused on the areas that need to be reviewed and re-observed, thereby making the candidate observation action set more concentrated around the key areas and reducing repeated observations of low-risk stable areas. Compared with the method of ending the single-period supplementary observation, this embodiment makes the review task set items have higher evidence density and clearer data traceability by updating the risk score matrix + risk area uncertainty prediction value + judgment mark refeeding in a rolling manner.

[0119] In a preferred embodiment, step S4.8 can be applied to an online appearance quality inspection scenario for stamped parts. In this scenario, the stamped parts to be inspected pass through the inspection station sequentially with a conveyor belt at a fixed pace. Several swarm vision robots are arranged around the inspection station. Each swarm vision robot is equipped with an onboard industrial camera and has the ability to adjust the observation pose and camera imaging parameters. The camera imaging parameters include at least exposure time and gain. In this embodiment, a preset sampling period is used as the time reference for data organization and action rolling updates. An example of a preset sampling period is 100 milliseconds. Each frame of the workpiece image is divided into a set of region units using a preset grid size. The grid size is set to 16×16 pixels. A first risk threshold and a first uncertainty threshold are used for screening and determining risk areas. The first risk threshold is set to 0.60, and the first uncertainty threshold is set to 0.12. A risk constraint threshold is used for screening candidate sets of observed actions. The risk constraint threshold is set to 0.25. A preset high confidence threshold and a preset conflict difference threshold are used to suppress conflicts in the same area involving multiple robots. The preset high confidence threshold is set to 0.80, and the preset conflict difference threshold is set to 0.15. An overlap ratio threshold is used to determine actions involving the same risk area. The overlap ratio threshold is set to 0.30.

[0120] Reference Figure 2 In the online appearance quality inspection scenario of this stamped part, the number of group vision robots is set to 4, denoted as the first robot, the second robot, the third robot, and the fourth robot. Within a sampling period with a sampling period index of 310, each group vision robot completes supplementary image acquisition, generation of defect confidence and uncertainty prediction values, fusion and update of risk score matrix, and output of judgment label according to steps S4.1 to S4.7. For example, in the judgment output of step S4.7, a risk area entry is written into the review task set, denoted as risk area A, with its risk area label A-07, updated risk score value of 0.82, uncertainty prediction value of 0.18, and the number of area units corresponding to this risk area is 14; candidate supplementary observation area Two risk area entries are written into the domain set, denoted as Risk Area B and Risk Area C, respectively. Risk Area B has a risk area identifier of B-12, an updated risk score of 0.55, an uncertainty prediction value of 0.21, and a number of regional units of 10. Risk Area C has a risk area identifier of C-03, an updated risk score of 0.58, an uncertainty prediction value of 0.14, and a number of regional units of 8. At the same time, risk area entries with a pass identifier are excluded from the review task set and the candidate supplementary observation area set, and risk area entries with a stable high-risk identifier are included in the stable treatment set but not in the merging objects of this step, so that the observation resources of the next sampling period are focused on the areas corresponding to the review identifier and the re-observation identifier.

[0121] according to Figure 2 As shown, the solid-lined thick-outlined circles represent risk area entries written into the review task set (corresponding to risk area A-07 in this example). These entries are prioritized for merging into the target risk area set in the next sampling period and have priority in subsequent action generation. The long dashed-lined circles represent one of the risk area entries written into the candidate supplementary observation area set (corresponding to risk area B-12 in this example). This means that the risk score does not meet the conditions for writing into the review task set, but the uncertainty is high, and supplementary observations are still needed in the next sampling period to reduce uncertainty. The dotted-lined dashed circles represent another risk area entry written into the candidate supplementary observation area set. (Corresponding to risk area C-03 in this example), it belongs to the same re-observation category as the long dashed line, but different entries are distinguished by different line types, reflecting the coexistence of multiple risk areas within the same set; the light-colored, low-transparency solid circle indicates risk area entries that have passed the identification. These entries have been excluded from the review task set and the candidate supplementary observation area set and will not be included in the merging objects of the next sampling period; the light-colored, low-transparency dashed circle indicates risk area entries that have been identified as stable high-risk and have entered the stable treatment set. These entries will not be included in the merging objects of S4.8 in this example, so that the observation resources of the next sampling period will focus on the areas corresponding to the review and re-observation identifications.

[0122] Figure 2 The rectangular container on the left represents the set of review tasks, the rectangular container in the middle represents the set of candidate supplementary observation areas, and the rectangular container on the right represents the set of entries that have been excluded or transferred to the stable treatment set. The circles falling into different containers only indicate that they have been written into the classification results of the corresponding set in the judgment output with sampling period index 310, thereby defining the boundary of the range of merged input objects with sampling period index 311.

[0123] Reference Figure 3At the start of the sampling period with sampling period index 311, step S4.8 is executed to merge the review task set and the candidate supplementary observation area set to obtain the target risk area set for the next sampling period. During the merging process, it is preferable to perform deduplication and sorting on the risk area entries: when two risk area entries overlap spatially, and the proportion of the number of overlapping area units to the smaller of the two area units is greater than or equal to a preset proportion threshold, they are merged into the same target risk area entry; when merging is not triggered, the entries are written independently; and it is preferable to prioritize the review task set entries and then the candidate supplementary observation area entries. The observation area set entries are written into the target risk area set in the following order, and sorted in descending order of uncertainty prediction value within the same category. Based on the above rules, in this example, the three entries of risk area A-07, risk area B-12 and risk area C-03 are all written into the target risk area set, and the order of the entries in the target risk area set is A-07, B-12 and C-03. Each entry retains the updated risk score value, uncertainty prediction value, number of regional units, boundary range field and center grid coordinate field, which serve as the basis for risk area coding for the state input of the next sampling period.

[0124] After the target risk region set is determined, within a sampling period with a sampling period index of 311, each group of visual robots generates a robot pose identifier for this period based on the consistency verification passed in step S1, and writes the robot pose identifier into the joint state input vector in a fixed order. As a specific example, the robot pose identifiers of the first, second, third, and fourth robots correspond to the position parameters and attitude parameters of their end effectors in the workstation coordinate system within this sampling period, respectively. The end effector position parameters of the first robot are 1.20 meters, 0.35 meters, and 0.80 meters; the end effector position parameters of the second robot are 0.85 meters, 0.40 meters, and 0.78 meters; the end effector position parameters of the third robot are 1.15 meters, 0.10 meters, and 0.82 meters; and the end effector position parameters of the fourth robot are 0.90 meters, 0.15 meters, and 0.80 meters. Subsequently, the target risk region set and the robot pose identifiers are input into a multi-agent reinforcement learning model that is centrally trained and distributed for execution. Under the risk constraint threshold and high confidence conflict penalty constraint, the model outputs a set of candidate observation actions with a sampling period index of 311.

[0125] In this set of candidate observation actions, each swarm of vision robots corresponds to at least one action entry containing an observation pose increment and a camera imaging parameter adjustment. As a specific example, the first robot outputs an action entry involving risk area A-07, with an example of an observation pose increment of 20 mm forward along the workpiece normal direction and an 8-degree deflection around the vertical axis, and an example of a camera imaging parameter adjustment of increasing the exposure time by 1 stop and decreasing the gain by 1 stop. This action entry provides a coverage increment of 0.32 for risk area A-07. The second robot outputs an action entry involving risk area B-12, with an example of an observation pose increment of 15 mm lateral translation and a downward tilt angle. The camera imaging parameter adjustment is 6 degrees, with an example of increasing the gain by 1 level. The coverage increment of this action item for the risk area B-12 is 0.28. The fourth robot outputs an action item involving the risk area C-03. Its observation pose increment is an example of lateral translation of 12 mm and deflection around the vertical axis of 10 degrees. The camera imaging parameter adjustment is an example of increasing the exposure time by 1 level. The coverage increment of this action item for the risk area C-03 is 0.26. When further performing coverage increment screening on the above action items, since the coverage increment of each action item is greater than or equal to the risk constraint threshold of 0.25, the above action items enter the constraint candidate observation action set.

[0126] Furthermore, a high-confidence conflict penalty determination is performed on the set of constrained candidate observation actions; as a specific example, refer to... Figure 4 Both the first and third robots output action items involving risk area A-07, and the fusion confidence score example for risk area A-07 is 0.86, meeting the preset high confidence threshold of 0.80. Meanwhile, the defect confidence scores for the same risk area differ among the different robots. The defect confidence score example for the first robot is 0.88, while that for the third robot is 0.70, with a difference of 0.18, reaching the preset conflict difference threshold of 0.15. Therefore, a high confidence conflict penalty is triggered for the third robot's action item involving risk area A-07. After the penalty is triggered, the third robot no longer selects action items pointing to risk area A-07 in the action priority score comparison, but instead selects action items covering the boundary area of ​​risk area B-12. This makes the allocation of observation resources for the target risk area set more dispersed within this sampling period, thereby reducing the probability of multiple robots repeatedly observing a single high-confidence risk area.

[0127] Therefore, in the above-mentioned online appearance quality inspection scenario of stamped parts, step S4.8 merges the set of review tasks and the set of candidate supplementary observation areas to form a set of target risk areas, and feeds the set of target risk areas and robot pose labels back into the multi-agent reinforcement learning model that is centrally trained and distributed for execution. This allows the set of candidate observation actions for the next sampling period to be generated spatially around the corresponding areas of the review labels and re-observation labels. Under the risk constraint threshold and high confidence conflict penalty constraint, the action items are screened and diverted, so that the key risk areas obtain higher observation density and avoid repeated observation of the same area.

[0128] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit it. Although the present invention has been described in detail with reference to preferred embodiments, those skilled in the art should understand that modifications or equivalent substitutions can be made to the technical solutions of the present invention without departing from the spirit and scope of the technical solutions of the present invention, and all such modifications or substitutions should be covered within the scope of the claims of the present invention.

Claims

1. A method for group vision robot quality inspection based on multi-agent reinforcement learning, characterized in that, include: Under a preset sampling period, several groups of visual robots acquire images of the workpiece to be inspected through an onboard industrial camera, and write robot pose identifiers and camera imaging parameter identifiers for each frame of image to obtain a quality inspection observation set. The quality inspection observation set is input into a defect detection network containing a random deactivation layer for N forward inferences. An uncertainty prediction value is generated by the variance of the N defect confidence outputs. A risk score matrix is ​​calculated based on the defect confidence and the uncertainty prediction value to determine the risk area set. The risk scoring matrix, the risk region set, and the robot pose identifier are input into a multi-agent reinforcement learning model that is trained and distributed in a centralized manner. Under the risk constraint threshold and the high confidence conflict penalty constraint, the model outputs a set of candidate observation actions. Based on the candidate observation action set, the group of visual robots is controlled to collect supplementary images, and the risk score matrix is ​​updated by using a weighted voting strategy with uncertainty prediction value as the weight. Risk areas with updated risk scores greater than or equal to the first risk threshold and uncertainty prediction values ​​greater than or equal to the first uncertainty threshold are written into the review task set and included in the candidate observation action set of the next sampling period.

2. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 1, wherein, The quality inspection observation set is obtained, including: At the beginning of each preset sampling period, each group of visual robots generates a sampling period index and writes the sampling period index into the record header information corresponding to the workpiece image to be inspected collected in this period. Each group of visual robots triggers the airborne industrial camera to expose and acquire workpiece image frames within the time window corresponding to the sampling period index, and the robot motion controller reads the current pose parameters under the same sampling period index to generate robot pose identifiers. Each group of visual robots reads the camera imaging parameters of the airborne industrial camera under the same sampling period index to generate a camera imaging parameter identifier, and binds the camera imaging parameter identifier and the robot pose identifier to the workpiece image frame according to the frame number; The consistency of the four-element records of the workpiece image frame, the robot pose identifier, the camera imaging parameter identifier, and the sampling period index is checked and merged according to the sampling period index to obtain the quality inspection observation set.

3. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 2, wherein, The consistency verification method includes: For each quaternion record, the sampling period index is read, and the exposure timestamp of the workpiece image frame is checked to see if it falls into the time window corresponding to the sampling period index. Those that do not fall into the time window are marked as cross-period abnormal records. For quaternary records not marked as cross-cycle abnormal records, check whether the time difference between the sampling timestamp of the robot pose identifier and the exposure timestamp of the workpiece image frame is less than a preset time difference threshold, and mark those that are greater than or equal to the preset time difference threshold as pose mismatch records. For quaternion records not marked as pose mismatch records, check whether the effective range of the camera imaging parameter identifier covers the exposure timestamp of the workpiece image frame, and mark those not covered as parameter mismatch records; and output the consistency verification result composed of the cross-cycle abnormal records, the pose mismatch records and the parameter mismatch records.

4. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 1 or 2, characterized in that, Perform N forward inference iterations on the quality inspection observation set, and generate an uncertainty prediction value based on the variance of the N defect confidence outputs, including: The workpiece image frames in the quality inspection observation set are normalized and cropped according to a preset input size to obtain an image input tensor set, and the image input tensor set is fed frame by frame into a defect detection network containing a random deactivation layer. For each image input tensor in the aforementioned image input tensor set, the random deactivation layer is enabled while keeping the defect detection network weight parameters unchanged, and indexed by the number of inferences. Perform the forward computation N times in sequence to obtain the corresponding N sets of defect output tensors; For each candidate defect target in each set of defect output tensors, the Softmax output of its classification branch is taken as the defect confidence score, and the defect confidence scores of the same candidate defect target under N forward calculations are indexed by the number of inferences to form a defect confidence score sequence. The sample variance is calculated for the defect confidence sequence to obtain the uncertainty prediction value corresponding to the candidate defect target, and the uncertainty prediction values ​​of each candidate defect target in the same workpiece image frame are summarized into the uncertainty prediction value set of that frame.

5. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 4, wherein, A risk scoring matrix is ​​calculated based on the defect confidence level and the uncertainty prediction value to determine a set of risk regions, including: Each frame of the workpiece image is divided into a set of region units according to a preset grid size, and the defect confidence and uncertainty prediction value of each candidate defect target in the frame are associated with the index of the region unit it covers. For each of the aforementioned regional unit indices, a risk score for that regional unit is calculated according to a preset weighting coefficient. The risk score is composed of a weighted sum and product of the defect confidence and uncertainty prediction values ​​of the corresponding candidate defect target. The risk scores of each regional unit are arranged according to spatial index to obtain the risk score matrix. Select the regional unit index with a risk score greater than or equal to the first risk threshold on the risk scoring matrix, and merge adjacent regional unit indices according to a preset connectivity rule to obtain a risk region set.

6. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 5, wherein, The preset connectivity rules include: determining adjacency based on the two-dimensional grid coordinates of the region unit index using four-neighbor connectivity; when two region unit indices satisfy the adjacency relationship and both of their risk scores are greater than or equal to the first risk threshold, the two region unit indices are assigned to the same connected component; for each connected component, the number of region units is calculated and compared with a preset minimum number threshold, and connected components with a number of region units greater than or equal to the preset minimum number threshold are determined as risk regions in the risk region set.

7. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 5, wherein, Output the set of candidate observation actions, including: In each preset sampling period, the risk scoring matrix and the risk region set are encoded into a global risk state vector by spatial index, and the robot pose identifiers of each group of visual robots are concatenated to the global risk state vector to obtain a joint state input vector. The joint state input vector is input into a multi-agent reinforcement learning model that is trained and distributed in a centralized manner to obtain the action score sequence corresponding to each group of visual robots. The action score sequence is then used to generate an initial set of candidate observation actions that includes the observation pose increment and the camera imaging parameter adjustment. The coverage increment of the risk area set is calculated for each item in the initial candidate observation action set, and candidate observation actions whose coverage increment is less than the risk constraint threshold are removed from the initial candidate observation action set to obtain the constraint candidate observation action set. For multiple robot candidate observation actions involving the same risk area in the set of constrained candidate observation actions, a high-confidence conflict penalty is calculated based on the defect confidence level corresponding to the risk area. The high-confidence conflict penalty is jointly triggered by the defect confidence level being greater than or equal to a preset high-confidence threshold and the difference in defect confidence levels between different robots being greater than or equal to a preset conflict difference threshold. For each candidate observation action in the constrained candidate observation action set, an action priority score is calculated, wherein the action priority score is obtained by weighting the coverage increment and the high confidence conflict penalty, and the candidate observation action corresponding to the maximum value is selected for each group of vision robots based on the action priority score, and the candidate observation action set is obtained by summarizing.

8. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 7, wherein, Controlling the group of visual robots to acquire supplementary images based on the candidate observation action set includes: The candidate observation action set is split according to the group visual robot identifier to obtain the target candidate observation action corresponding to each group visual robot, and the target candidate observation action is parsed into observation pose increment and camera imaging parameter adjustment amount; The motion trajectory point sequence is generated based on the observed pose increment and sent to the motion controller. At the same time, the camera imaging parameters of the airborne industrial camera are updated based on the camera imaging parameter adjustment amount, and the updated camera imaging parameter identifier is generated. After each group of visual robots reaches the final pose of the motion trajectory point sequence, the onboard industrial camera is triggered to acquire supplementary images, and the robot pose identifier and the camera imaging parameter identifier are written into the supplementary images to obtain a supplementary quality inspection observation set.

9. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 8, wherein, The risk scoring matrix is ​​updated by fusing the weighted voting strategy, including: The supplementary quality inspection observation set is merged into the quality inspection observation set according to the sampling period index, and each frame of the merged image is input into a defect detection network containing a random deactivation layer to perform N forward inferences to generate the corresponding defect confidence and uncertainty prediction value. The defect confidence and uncertainty prediction value are then associated with the region unit index corresponding to the risk region set. For each risk region in the risk region set, multiple defect confidence scores falling into that risk region are aggregated, and the reciprocal of the sum of the uncertainty prediction value corresponding to each defect confidence score and a preset constant is used as the voting weight to obtain a weighted voting weight set. The defect confidence is obtained by weighted summation based on the weighted voting weight set. The fused confidence is then combined with the uncertainty prediction value of the risk area according to a preset weight coefficient to calculate the updated risk score. This updated risk score replaces the risk score of the corresponding area cell index in the risk score matrix, resulting in the updated risk score matrix.

10. The multi-agent reinforcement learning based collective visual robot inspection method according to claim 9, wherein, Also includes: For each risk region in the updated risk scoring matrix, its updated risk score is compared with the first risk threshold, and its uncertainty prediction value is compared with the first uncertainty threshold. Based on the comparison results, a judgment identifier is output: when the updated risk score is greater than or equal to the first risk threshold and the uncertainty prediction value is greater than or equal to the first uncertainty threshold, the first judgment identifier is output and the risk region is written into the review task set. When the updated risk score is greater than or equal to the first risk threshold and the uncertainty prediction value is less than the first uncertainty threshold, a second judgment flag is output and the risk area is excluded from the review task set. When the updated risk score is less than the first risk threshold and the uncertainty prediction value is greater than or equal to the first uncertainty threshold, the third judgment identifier is output and the risk area is written into the candidate supplementary observation area set. When the updated risk score is less than the first risk threshold and the uncertainty prediction value is less than the first uncertainty threshold, the fourth judgment flag is output and the risk area is simultaneously excluded from the review task set and the candidate supplementary observation area set. The set of verification tasks and the set of candidate supplementary observation regions are merged to obtain the set of target risk regions for the next sampling period. The set of target risk regions and the robot pose identifier are then input into a multi-agent reinforcement learning model that is trained and distributed to output a set of candidate observation actions for the next sampling period under risk constraint threshold and high confidence conflict penalty constraint.

Images