Robot inspection line defect recognition system based on visual positioning

Abstract

This invention discloses a vision-based robot inspection system for production line defect identification, belonging to the field of manufacturing automation technology. The system comprises: a visual acquisition, collaborative reasoning, execution storage, self-evolutionary update, and heatmap optimization module working collaboratively to achieve automatic identification, location, and marking of defects in production line workpieces. It utilizes lightweight neural networks for synchronous reasoning, combined with a confidence fusion algorithm to improve accuracy, and automatically updates the model based on inspection data. Simultaneously, it optimizes the robot inspection path and process through heatmap analysis. This invention achieves synchronous parallel processing and comprehensive judgment of visual positioning and defect identification, improving inspection accuracy and real-time performance. Its self-evolutionary capability adapts to defect changes, and it intelligently optimizes the inspection path and provides early warnings through heatmap analysis. Ultimately, it constructs a dynamic and intelligent closed-loop quality control system, comprehensively enhancing the reliability and adaptability of production line inspection.

Description

Technical Field

[0001] This invention relates to the field of manufacturing automation technology, specifically to a robot-based defect identification system for production lines using visual positioning. Background Technology

[0002] In the process of intelligent transformation of the manufacturing industry, workpiece quality inspection on the production line is a core link to ensure production continuity and product reliability. With the expansion of production scale and the increase in product process complexity, traditional manual inspection is no longer able to meet the needs of efficient production. Automated and intelligent inspection technologies have become an inevitable trend in the industry. Visual inspection technology, with its non-contact and high-response characteristics, has been widely used in production line quality inspection. The flexible movement capability of industrial robots provides hardware support for inspection execution. The integration of the two has become an important path to achieve comprehensive and accurate quality inspection. Modern production lines not only require accurate identification of workpiece surface defects, but also need to clarify key information such as the spatial location and degree of impact of defects to provide data support for subsequent defect handling and process optimization. Against this background, the integrated and parallel processing of visual positioning and defect identification, as well as the adaptability of the inspection system to dynamic working conditions, have become key breakthroughs in improving the level of production line quality inspection, driving related technologies to develop in a more efficient, intelligent and adaptable direction.

[0003] Traditional production line defect detection technologies have many limitations, making it difficult to meet the high-quality and high-efficiency quality inspection requirements of modern manufacturing. Some technologies separate visual positioning and defect recognition into independent processes, resulting in redundant and time-consuming workflows and excessively long detection cycles, impacting overall production line efficiency. Furthermore, the inference models used are mostly fixed architectures, lacking dynamic adjustment mechanisms and unable to cope with dynamic scenarios such as changes in production line conditions and the emergence of new defects. Over long-term use, detection accuracy is prone to decline. In terms of data management, traditional methods lack a systematic classification and storage strategy, resulting in the inefficient handling of image data, feature information, and inference results generated during the detection process. The scattered storage of results makes it difficult to achieve effective reuse and in-depth analysis, and cannot provide sufficient data support for subsequent optimization. At the same time, traditional detection paths are mostly preset fixed trajectories, which cannot be dynamically adjusted according to the characteristics of defect distribution, and are prone to detection blind spots or repeated detection, resulting in waste of resources. Moreover, there is a lack of effective analysis and early warning mechanism for defect distribution trends, which makes it impossible to identify potential quality risks in advance, resulting in delayed process adjustments and difficulty in reducing the defect incidence rate from the source. In addition, traditional systems lack self-optimization capabilities. When detection requirements or production line parameters change, the model and process need to be manually readjusted, which is cumbersome and has poor adaptability. Summary of the Invention

[0004] The purpose of this invention is to overcome the shortcomings of existing technologies and provide a vision-based robot inspection system for production line defect identification. Through the collaborative work of five modules—visual acquisition, collaborative reasoning, execution and storage, self-evolutionary updating, and heatmap optimization—it achieves automatic identification, location, and marking of defects in production line workpieces. A lightweight neural network is used to simultaneously execute visual positioning and defect identification, combined with a three-dimensional confidence fusion algorithm to improve judgment accuracy. It can automatically update model parameters based on inspection data and utilize heatmap analysis to optimize robot inspection paths and production line processes, achieving intelligent and adaptive defect detection and early warning.

[0005] To solve the above-mentioned technical problems, the present invention provides the following technical solution: a visual positioning-based robot inspection system for production line defect identification, the system comprising: Visual acquisition module: Deploys industrial cameras and supporting light sources to acquire images of the areas to be inspected on the production line, preprocesses the images, extracts visual feature vectors, and provides a high-quality data foundation for subsequent inference calculations; Collaborative Reasoning Module: Deploys a lightweight neural network, combines the visual feature vectors output by the visual acquisition module, and performs parallel reasoning for visual localization and defect recognition simultaneously. It uses a three-dimensional confidence fusion algorithm to comprehensively judge the reasoning results and outputs the localization coordinates, defect type, severity, and related detection results. Execution storage module: Based on the detection results output by the collaborative reasoning module, it parses and generates robot motion instructions, controls the robot to complete the defect marking operation, and classifies and stores the detection data to provide a data foundation for model self-evolution update and heat map optimization; Self-evolution update module: Real-time monitoring of defect sample increment and model detection status. When the training trigger condition is met, relying on the detection data stored in the execution storage module, the defect identification model parameters are incrementally updated through the spatiotemporal driven evolution algorithm. The optimized model parameters are synchronized to the collaborative inference module to ensure continuous optimization of inference accuracy. Heatmap Optimization Module: Based on the defect coordinate data stored in the execution storage module, a heatmap of the spatial distribution of defects on the production line is constructed. The heatmap features are analyzed to generate suitable detection paths for the robot, guide the robot to adjust its detection trajectory, and output a multimodal detection report to complete defect distribution trend analysis and anomaly warning.

[0006] Furthermore, in the vision acquisition module, the images of the production line to be inspected are images of the surface and key parts of various workpieces to be inspected during the production process, including the key inspection points of the workpiece's machined surface, joints, and marked areas. During image acquisition, the industrial camera moves synchronously with the robot's end effector, and captures images of the key areas of each workpiece to be inspected according to the preset inspection point sequence of the production line. The supporting light source is turned on synchronously to assist the camera in clearly capturing the details of the workpiece surface, ensuring that the acquired image can completely present the surface state of the workpiece. The core visual feature vector of the image is a set of key visual information that can characterize the defects on the workpiece surface, including three core contents: edge contour, texture distribution, and color features of the workpiece surface. During feature vector extraction, feature analysis is performed on the preprocessed image to capture the edge contour features of the workpiece surface, sort out the texture distribution rules of the image, extract the color feature information of the image, and integrate the three types of features to form a core visual feature vector that can accurately reflect the surface state of the workpiece.

[0007] Furthermore, in the collaborative reasoning module, a lightweight neural network is deployed on the robot's local edge controller. This is a simplified neural network adapted to the computing power of the edge, used to interface with the visual feature vectors output by the vision acquisition module. The lightweight neural network integrates a visual positioning reasoning branch and a defect recognition reasoning branch. The two reasoning branches are set up independently and run synchronously. The visual positioning reasoning branch retrieves production line scene features based on the input visual feature vector for comparison, establishes the correspondence between the input features and the reference features, completes the production line scene feature matching, and then converts the two-dimensional pixel coordinates in the image into three-dimensional physical coordinates that the robot can recognize based on the matched feature correspondence, completing the spatial coordinate conversion. The defect recognition reasoning branch distinguishes between normal and abnormal areas on the workpiece surface based on the same visual feature vector, and then identifies the defect-related information corresponding to the abnormal area. The two reasoning branches synchronously output positioning-related data and defect-related data, providing basic data for the three-dimensional confidence fusion algorithm to carry out comprehensive defect judgment.

[0008] Furthermore, in the collaborative reasoning module, the mathematical expression for the three-dimensional confidence fusion algorithm is:

[0009] in, This represents the confidence level of the defect, with a value ranging from 0 to 1. The higher the value, the greater the probability that the defect exists. This indicates the potential defect target to be detected; This represents the visual feature matching degree, calculated from image edge, texture, and color gradient features, with a value range of 0-1. Represents visual feature vectors; This represents the positioning accuracy weight, which is derived from the coordinate error output by visual positioning. The smaller the error, the higher the weight. The value range is 0-1. Indicates location coordinate information; This represents the thermal value of the defect spatial distribution, corresponding to the production line coordinates. The historical defect incidence rate, with a value ranging from 0 to 1. and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. , , Let be the weighting coefficient, satisfying + + =1, which can be dynamically adjusted according to the characteristics of the production line.

[0010] Furthermore, in the collaborative reasoning module, the output positioning coordinates are the spatial coordinates of the defect on the production line workpiece. These coordinates are obtained by the visual positioning reasoning branch combining the visual feature vector output by the visual acquisition module to map the defect location in the image to the actual spatial coordinate system of the production line. The defect type is the defect category corresponding to the abnormal area on the workpiece surface. This is determined by the defect recognition reasoning branch based on the visual feature vector to distinguish between normal and abnormal areas of the workpiece, and then by matching the contour, texture, and color features of the abnormal area with the defect category features. The severity is the level of impact of the defect on the workpiece quality. Different levels are determined based on the defect confidence level output by the three-dimensional confidence fusion algorithm, combined with the production line quality standards. Specifically, when the defect area is less than or equal to a preset threshold A1, or the depth is less than or equal to a preset threshold D1, or the length is less than or equal to a preset threshold D1, the defect type is determined by the defect type. When the threshold L1 is set, the defect is classified as minor. When the area of ​​the defect is greater than A1 and less than or equal to the preset threshold A2, or the depth is greater than D1 and less than or equal to the preset threshold D2, or the length is greater than L1 and less than or equal to the preset threshold L2, the defect is classified as moderate. When the area of ​​the defect is greater than A2, or the depth is greater than D2, or the length is greater than L2, the defect is classified as severe. A1, A2, D1, D2, L1, and L2 are numerical thresholds preset according to the quality requirements of the production line workpiece, used to quantify and distinguish defects of different severity. The relevant core detection results include defect confidence, detection timestamp, and detection batch information. The defect confidence is directly output by the three-dimensional confidence fusion algorithm, the detection timestamp is the inference completion time, and the detection batch information is associated with the batch of the currently inspected workpiece for quality traceability and data analysis.

[0011] Furthermore, in the execution storage module, robot motion commands control the robot to complete the defect marking operation. These commands include robot motion trajectory commands, joint posture adjustment commands, and marking action commands. The motion trajectory commands are generated based on the defect location coordinates to determine the robot's movement path and speed parameters from its initial position to the defect location. The joint posture adjustment commands are generated based on the orientation of the workpiece surface where the defect is located to ensure that the marking mechanism maintains an adapted posture to the defect surface. The marking action commands are generated in combination with the defect type and severity to determine the marking force, marking style, and marking range, ensuring that the marking is clear and identifiable without damaging the workpiece, thus achieving accurate marking of various defects.

[0012] Furthermore, in the execution storage module, the classified storage of detection data is specifically classified and stored according to four dimensions: defect attributes, workpiece information, detection process, and data type. Specifically, the data quantification parameters stored according to defect attributes include defect type, severity, location coordinates, defect area, depth, and length; the data stored according to workpiece information includes workpiece unique identifier ID, batch number, production time, material, model, and production line station; the data stored according to the detection process includes detection timestamp, detection equipment number, operator ID, and detection environment parameters; and the data stored according to data type includes raw image data output by the vision acquisition module, visual feature vectors, inference result data packets output by the collaborative inference module, and robot motion command data packets generated by the execution storage module.

[0013] Furthermore, in the self-evolutionary update module, the mathematical expression of the spatiotemporal driven evolutionary algorithm is:

[0014] in, This is the parameter update amount for the defect identification model, used to adjust the model weights and achieve model self-evolution; Represents the current set of parameters of the model; The learning rate has a range of (0, 1) and a default value of 0.001. It is used to control the magnitude of parameter updates and balance update efficiency with training stability. The total number of defect samples participating in this incremental training is calculated from the newly added detection data that meet the training trigger conditions in the execution storage module; This is the defect sample index, with a value range of 1-n, used to traverse all incremental samples participating in training; For the i-th defect sample The confidence level of the true defects is provided by manual annotation or historical verification data, serving as a supervisory signal for training; For the i-th defect sample The model predicts the confidence level of defects, which is determined by the current defect identification model for the samples. The output is obtained through inference calculation; For model parameters The gradient vector, through the prediction error ( The backpropagation algorithm of −) is used to calculate and indicate the optimal direction for parameter adjustment; This is the heatmap influence coefficient, with a value range of (0,1) and a default value of 0.1. It is used to control the weight of spatial defect distribution on model updates, balancing the influence of time dimension error and spatial dimension distribution. The time step is defined as the time interval between two model self-evolution update operations, in seconds, reflecting the driving force of the time dimension on model evolution. This is a heat map showing the spatial distribution of defects, where and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. The values ​​in the heat map indicate the historical probability or density of defects occurring at that spatial location. The rate of change of the defect heatmap over time is used to capture the dynamic evolution trend of defect distribution.

[0015] Furthermore, in the heatmap optimization module, a heatmap of the spatial distribution of production line defects is constructed based on the defect coordinate data stored in the execution storage module. The data covers the production line spatial coordinates, defect type, severity, and detection timestamp of each defect. Discrete defect points are converted into a continuous density field through kernel density estimation and spatial interpolation algorithms, covering the X-axis in the length direction, the Y-axis in the width direction, and the Z-axis in the depth or height direction of the production line workpiece, forming a three-dimensional spatial distribution. Two-dimensional heatmaps can also be generated by workstation batch or time slice. The density calculation is based on the weighted sum of the number and severity of defects in a unit space to obtain the defect density value of each spatial unit. The higher the density, the darker the corresponding color, intuitively presenting the risk concentration area. At the same time, features such as the distribution trend and time evolution of hot spots are extracted to generate suitable detection paths for robots, guide the adjustment of detection trajectories, output multimodal detection reports, and complete defect distribution trend analysis and anomaly warning.

[0016] Furthermore, the heatmap optimization module includes a multimodal inspection report comprising three parts: visualization, data analysis, and application guidance. The visualization part outputs a heatmap of the spatial distribution of defects on the production line, a pie chart of defect type distribution, and a time-dimensional trend curve, intuitively displaying the spatial aggregation state and dynamic changes of defects. The data analysis part extracts high-incidence defect areas, high-frequency defect types, and distribution trend inflection points to identify potential anomaly risks. The application guidance part outputs robot inspection path optimization suggestions and production line process adjustment directions based on the analysis results, providing decision support for quality control. The report has a built-in anomaly warning mechanism that automatically triggers warning signals when heat values ​​suddenly increase or distribution patterns deviate from historical benchmarks, identifying and marking abnormal areas and potential causes. In addition, the report also includes a data summary, extracting key indicators such as defect incidence rate, risk level ratio, and trend prediction conclusions, providing intuitive decision-making basis for production line quality control and process optimization.

[0017] Compared with existing technologies, this vision-based robotic inspection system for production line defect identification has the following advantages: I. This invention integrates visual acquisition and collaborative reasoning modules to achieve synchronous parallel processing of visual positioning and defect recognition, breaking the limitations of traditional separate operations. By leveraging multi-dimensional feature fusion, it achieves comprehensive judgment of reasoning results, allowing positioning accuracy and defect recognition accuracy to support each other. The visual acquisition module comprehensively captures the features of key areas of the workpiece, providing rich evidence for subsequent reasoning. The collaborative reasoning module, relying on the efficient processing capabilities of the adapted edge, quickly completes feature comparison and coordinate conversion. At the same time, through a multi-factor fusion judgment method, it fully considers feature matching, positioning accuracy, and spatial distribution patterns, significantly reducing the probability of misjudgment and missed judgment, improving the real-time performance and reliability of production line inspection, and laying a solid foundation for robots to accurately execute marking operations. It effectively solves the efficiency and accuracy bottlenecks caused by asynchronous positioning and recognition and single judgment dimensions in traditional inspection.

[0018] II. This invention constructs a dynamically optimized detection system through the synergistic effect of a self-evolutionary update and a heatmap optimization module. This system continuously improves defect identification capabilities and intelligently adjusts detection paths. The self-evolutionary update module monitors detection data and model status in real time. Based on the evolutionary laws of incremental samples and spatiotemporal dimensions, it automatically adjusts model parameters to continuously adapt to various defect changes occurring on the production line. It maintains high-efficiency detection performance without manual intervention. The heatmap optimization module constructs a spatial distribution heatmap based on historical detection data, intuitively presenting areas of concentrated defect risks. Based on this, it generates suitable detection paths to guide the robot to dynamically adjust its trajectory. At the same time, it achieves defect trend analysis and anomaly warning through multimodal reporting, providing data support for production line process optimization. This comprehensively improves the intelligence and adaptability of production line detection, helps achieve closed-loop optimization of production line quality control, and meets the continuous iteration quality control needs of the production line.

[0019] Other advantages, objectives and features of the invention will be set forth in part in the description which follows, and in part will be apparent to those skilled in the art from the following examination or study, or may be learned from the practice of the invention. Attached Figure Description

[0020] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the accompanying drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without any creative effort.

[0021] Figure 1 This is a flowchart of a vision-based robot-based defect identification system for production lines. Figure 2 This is a schematic diagram of data transmission in a vision-based robot inspection production line defect identification system. Figure 3 This is a schematic diagram of data transmission in the collaborative reasoning module of the present invention. Detailed Implementation

[0022] To further illustrate the technical means and effects of the present invention in achieving its intended purpose, the following detailed description of the specific implementation methods, structures, features, and effects of the present invention, in conjunction with the accompanying drawings and preferred embodiments, is provided below.

[0023] Example

[0024] Gear workpiece defect detection on machining production line This embodiment is applied to the mass production line of gear manufacturing in a large machinery manufacturing enterprise. Addressing the defect detection needs of various gear types, including cylindrical gears and bevel gears, such as surface cracks, tooth wear, joint deformation, and missing markings, a vision-based robotic defect identification system is deployed. This system comprehensively covers key quality inspection stages from the completion of raw material processing to the final product's shipment, ensuring that gear products meet assembly accuracy and service life requirements. Figure 1 As shown.

[0025] The vision acquisition module, serving as the core of the system's data input, integrates an industrial camera and a matching high-brightness ring light source into the end effector of a six-axis industrial robot. The camera lens employs high-resolution optical components, and the light source brightness can adaptively adjust according to the reflective characteristics of the gear material. During the image acquisition phase, the industrial camera moves synchronously with the robot's end effector according to the preset inspection point sequence on the production line. It captures key areas of each gear under inspection from multiple angles, including the gear end face, tooth tip, and tooth root. The matching light source is simultaneously activated, using oblique illumination to reduce the interference of metal reflections on the gear surface on image quality. This clearly captures the fine textures of the gear's machined surface, the fit of the tooth groove joints, and the complete shape of the marked areas, avoiding the loss of details due to insufficient light or reflections. In the image preprocessing phase, the system performs noise reduction, enhancement, and normalization on the acquired raw images, removing interference information caused by environmental dust and shooting vibrations, and retaining effective content that reflects the true state of the gear. In the feature vector extraction stage, the system performs multi-dimensional feature analysis on the preprocessed image, accurately captures the edge contour features of the gear surface, clearly sorts out the distribution pattern of the tooth groove texture, accurately extracts the color feature information of different regions, and performs weighted integration of the three types of features to form a core visual feature vector that can comprehensively and accurately reflect the state of the gear surface. This provides solid and reliable information support for subsequent collaborative reasoning and ensures that the reasoning process can be carried out based on complete and effective data.

[0026] The collaborative reasoning module undertakes the core data processing tasks. Its lightweight neural network is deployed on the robot's local edge controller. This neural network has been optimized for computing power adaptation, significantly reducing computing resource consumption while ensuring inference accuracy. It can quickly respond to the visual feature vectors output by the vision acquisition module, avoiding the impact of data transmission delays or insufficient computing power on detection efficiency. The lightweight neural network integrates a visual positioning inference branch and a defect recognition inference branch. The two inference branches are set independently and run synchronously, breaking the efficiency bottleneck caused by the traditional serial processing mode and significantly shortening the detection cycle. Based on the input core visual feature vector, the visual positioning inference branch automatically retrieves the preset gear standard scene benchmark feature library of the production line. Through the feature point matching algorithm, it establishes a one-to-one correspondence between the input features and the benchmark features, completing the accurate matching of production line scene features. Then, based on the matched feature correspondence and combined with the 3D modeling data of the gear, it accurately converts the 2D pixel coordinates in the image into 3D physical coordinates that the robot can recognize, realizing the spatial positioning of the defect and ensuring that subsequent marking operations can accurately target the defect area. The defect identification inference branch, based on the same core visual feature vector, uses a deep learning algorithm to distinguish normal machined areas on the gear surface from abnormal areas with problems such as cracks, wear, and deformation. It then compares the contour shape, texture changes, and color differences of the abnormal areas with a pre-defined defect category feature library to determine the specific defect type corresponding to the abnormal area, providing a clear basis for subsequent labeling and process adjustments. In the inference result determination stage, the system calls a three-dimensional confidence fusion algorithm. The mathematical expression of the three-dimensional confidence fusion algorithm is:

[0027] in, Indicates the confidence level of the defect; This indicates the potential defect target to be detected; Indicates the degree of visual feature matching; Represents visual feature vectors; Indicates the positioning accuracy weight; Indicates location coordinate information; This represents the thermal value indicating the spatial distribution of defects. and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. , , The weighting coefficients comprehensively consider visual feature matching degree, positioning accuracy weight, and defect spatial distribution heat value. The outputs of the two inference branches are fully integrated and comprehensively judged, effectively avoiding misjudgment and omission problems caused by single-dimensional judgment, significantly improving the accuracy and reliability of defect judgment. The final output includes the spatial coordinates of the defect on the gear, the specific defect type, and the severity level of its impact on product quality. Figure 3 As shown.

[0028] The execution storage module, serving as the core of system action execution and data management, closely integrates with the output of the collaborative reasoning module to achieve seamless integration of detection and handling. During the robot motion command generation phase, the system accurately parses and generates robot motion trajectory commands, joint posture adjustment commands, and marking action commands based on the defect location coordinates, type, and severity output by the collaborative reasoning module. The motion trajectory command clearly defines the optimal movement path and speed parameters for the robot from its initial standby position to the surface of the gear containing the defect. Path planning fully considers the layout space of the gear production line and the location of surrounding equipment to avoid collisions during movement. The speed parameters are dynamically adjusted according to the distance to the defect and the precision of the gear to ensure smooth and efficient movement. The joint posture adjustment command is generated based on the orientation angle of the gear surface containing the defect. By adjusting the rotation angle of each joint of the robot, it ensures that the marking mechanism maintains a perpendicular or appropriately tilted posture to the gear defect surface, avoiding marking position offset due to posture deviation. The marking action instructions are generated based on the defect type and severity. Fine lines are used for crack-like defects, while block markings are used for wear and deformation defects. The marking force is precisely controlled according to the gear surface hardness, ensuring clear and legible markings without causing secondary damage to the gear surface. The marking range is strictly limited to 1-2mm outside the defect edge to ensure no impact on the gear's subsequent use. In the data storage phase, the system classifies and stores inspection data according to four dimensions: defect attributes, workpiece information, inspection process, and data type. The defect attribute dimension covers quantitative information such as defect type, severity, location coordinates, defect area, depth, and length. The workpiece information dimension includes traceability information such as the gear's unique identifier ID, production batch number, production time, material type, and production line station. The inspection process dimension records process information such as inspection timestamp, inspection equipment number, operator ID, and environmental temperature and humidity. The data type dimension stores raw and processed data, including original image data, visual feature vectors, inference result data packages, and robot motion instruction data packages. This systematic classification and storage method makes data retrieval and retrieval more convenient, providing a standardized and complete data foundation for subsequent model updates, quality traceability, and process analysis.

[0029] The self-evolutionary update module endows the system with continuous optimization capabilities, ensuring that detection performance can adapt to dynamic changes in production line conditions. The module monitors the increase in gear defect samples and the model detection status in real time, specifically including indicators such as the number of new defect samples, the frequency of new defects, and the deviation rate between model inference results and manual verification results. When the number of new defect samples reaches a preset threshold, or the model detection deviation rate exceeds the allowable range, i.e., when the training trigger condition is met, the module automatically uses the multi-dimensional detection data stored in the execution storage module to launch the spatiotemporal driven evolutionary algorithm to incrementally update the defect recognition model parameters. The mathematical expression of the spatiotemporal driven evolutionary algorithm is:

[0030] in, This represents the parameter update amount for the defect identification model; Represents the current set of parameters of the model; The learning rate; This represents the total number of defective samples participating in this incremental training. For defect sample indexing; For the i-th defect sample The confidence level of the true defect; For the i-th defect sample The model predicts the confidence level of defects; For model parameters The gradient vector; This represents the influence coefficient of the heat map. For time step; This is a heat map showing the spatial distribution of defects. and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. This represents the rate of change of the defect heatmap over time. During the update process, the algorithm fully utilizes the difference between the confidence levels of actual defects in historical inspection data and the confidence levels of defects predicted by the model. Combined with the temporal trend of the defect spatial distribution heatmap, it precisely adjusts the model's weight parameters, enabling the model to quickly learn the characteristic patterns of new defects and adapt to changes in defect characteristics caused by factors such as tool wear and machine tool parameter drift during gear machining. The optimized model parameters are synchronized to the collaborative inference module in real time, replacing the original parameters in the inspection work. This allows the system to achieve autonomous iteration of inspection capabilities without manual intervention, maintaining stable and efficient inspection performance over the long term and avoiding a decrease in inspection accuracy due to changes in production line conditions.

[0031] The heatmap optimization module enables dynamic optimization and risk warning of the inspection process, providing proactive support for production line quality control. Based on the defect coordinate data stored in the execution storage module, the module integrates relevant information such as defect type, severity, and inspection timestamps. Through kernel density estimation and spatial interpolation algorithms, it converts discrete defect point data into a continuous density field, constructing a three-dimensional heatmap of the production line defect spatial distribution covering the gear's length (X-axis), width (Y-axis), and height (Z-axis). It also supports generating two-dimensional heatmaps by production batch and inspection station, meeting the analysis needs of different scenarios. During density calculation, the system assigns different weights based on defect severity, with severe defects having a higher weight than minor defects. The defect density value for each spatial unit is obtained by weighted summation of the number of defects per unit space and their weights. Higher density corresponds to a darker heatmap color, intuitively and clearly presenting areas of concentrated gear defect risk, helping staff quickly locate weak points in quality control. The module further extracts the distribution trends and temporal evolution characteristics of hotspot areas in the heatmap, analyzes the changing patterns and spatial migration trajectories of defect occurrence frequency, and generates suitable inspection paths for the robot based on these characteristics. This guides the robot to adjust its inspection trajectory, increasing the inspection frequency and coverage density in high-defect areas, reducing ineffective inspections in defect-free or low-defect areas, improving the targeting and overall efficiency of the inspection work, and avoiding the omission of key risk points. Simultaneously, the module outputs a multimodal inspection report. The visualization section includes a heatmap of defect spatial distribution, a pie chart of defect type distribution, and a time-dimensional trend curve, allowing staff to quickly grasp the overall defect situation through intuitive charts. The data analysis section deeply extracts high-defect areas, high-frequency defect types, and inflection points in distribution trends, accurately identifying potential anomalies. The application guidance section outputs robot inspection path optimization suggestions and specific guidance on gear machining tool change cycles and cutting parameter adjustment directions based on the analysis results. The report has a built-in anomaly warning mechanism. When the heat value suddenly surges or the defect distribution pattern deviates from the historical benchmark range, the system automatically triggers an audible and visual warning signal, accurately marks the abnormal area, and combines historical data to infer potential causes, such as excessive tool wear or machine tool accuracy deviation, reminding staff to investigate and deal with the problem in a timely manner, thereby reducing the defect incidence rate from the source.

[0032] This embodiment utilizes a visual acquisition module to accurately capture key features of the gear's critical areas, providing robust data support for subsequent reasoning. The collaborative reasoning module, relying on parallel reasoning and a 3D confidence fusion algorithm, achieves efficient and accurate defect localization and identification. The execution and storage module ensures accurate and lossless marking operations while standardizing the storage of detection data. The self-evolutionary update module allows the model to autonomously adapt to changes in operating conditions, maintaining stable detection performance. The heatmap optimization module dynamically optimizes the detection path and provides risk warnings. The entire process automates, refines, and intelligently detects gear defects, significantly improving detection efficiency and accuracy, effectively mitigating quality risks, and providing comprehensive support for quality control in machining production lines.

[0033] Example

[0034] Detection of soldering defects on circuit boards in electronic component production lines This embodiment is applied to a circuit board assembly line in the consumer electronics industry. Addressing the defect detection needs of printed circuit boards (PCBs) used in mobile phones, computers, and other electronic products, it deploys a vision-based robotic defect identification system. This system covers key quality inspection stages after PCB mounting, soldering, and cleaning, ensuring the electrical performance and assembly reliability of the PCBs and preventing electronic product malfunctions due to soldering defects. Figure 2 As shown.

[0035] The vision acquisition module, designed for the precision and fragility of circuit boards, utilizes a miniaturized, high-resolution industrial camera and a low-power bar light source, integrated into the end effector of a lightweight industrial robot. The camera, equipped with a macro lens, clearly captures the detailed features of even tiny solder joints. The light source employs a cold light source design to avoid damage to circuit board components due to heat. During the image acquisition phase, the industrial camera moves synchronously with the robot's end effector according to the pre-set inspection points on the production line. It focuses on precisely capturing key points on the circuit board, such as chip solder joints, resistor and capacitor solder joints, pin connections, and marked areas. The robot maintains a low and stable movement speed during image capture to prevent camera shake caused by inertia from affecting image quality. The accompanying bar light source is simultaneously activated, uniformly covering the imaging area with parallel light, highlighting the color differences and contour boundaries between solder joints, the substrate, and pins. It clearly captures irregular shapes of solder joints caused by poor soldering, empty areas formed by missing solder joints, darkened solder joints due to oxidation, and gaps at pin connections, effectively avoiding misjudgments of defects caused by uneven lighting. In the image preprocessing stage, the system performs noise reduction, sharpening, and contrast adjustment on the acquired raw images, removing interference from residual flux and environmental dust, and enhancing the feature details of key areas such as solder joints and pins. In the feature vector extraction stage, the system performs refined feature analysis on the preprocessed images, accurately capturing the edge contour features of solder joints, clearly identifying the texture distribution patterns on the solder joint surface, and precisely extracting the color feature information of solder joints, pins, and the substrate. These three types of features are integrated to form a core visual feature vector that comprehensively reflects the soldering status and surface condition of the circuit board. This ensures that subsequent collaborative reasoning processes can be based on accurate and complete feature information, providing a solid foundation for defect identification and localization.

[0036] The lightweight neural network of the collaborative reasoning module is specially optimized to adapt to the limited computing power of the robot's local edge controller, ensuring the accuracy of inference results while achieving rapid inference. The lightweight neural network integrates a visual positioning inference branch and a defect recognition inference branch. These two branches operate independently and synchronously, allowing the other branch to start operation without waiting for the first to complete, significantly shortening the overall inference time and meeting the quality inspection requirements of high-speed PCB production lines. The visual positioning inference branch, based on the input core visual feature vector, retrieves a baseline feature library of the PCB production line scene. This library contains standard layouts, component positions, and solder point coordinates for different PCB models. A feature matching algorithm establishes a correspondence between the input features and the baseline features, achieving accurate feature matching of the production line scene. Based on the matched feature correspondence, combined with the PCB's 2D layout diagram and 3D modeling data, the 2D pixel coordinates in the image are accurately converted into 3D physical coordinates recognizable by the robot, accurately locating the spatial position of defects on the PCB and providing precise location guidance for subsequent marking operations. The defect identification inference branch, based on the same core visual feature vector, uses a deep learning algorithm to distinguish between normal soldering areas and abnormal areas on the circuit board surface. It then compares the contour shape, texture features, and color changes of the abnormal areas with a pre-defined defect category feature library to identify the corresponding defect type, such as cold solder joints, missing solder joints, or solder joint oxidation. Simultaneously, it uses the defect's shape and size to preliminarily determine its severity. In the inference result judgment stage, the system calls a three-dimensional confidence fusion algorithm, comprehensively considering visual feature matching degree, positioning accuracy weight, and defect spatial distribution thermal values. It fully integrates and comprehensively judges the output results of the two inference branches, effectively filtering out interference caused by minor component offsets and shooting angle deviations, reducing misjudgments and omissions. Finally, it outputs the precise location coordinates of the defect on the circuit board, the specific defect type, and the severity level of its impact on electrical performance.

[0037] The execution storage module fully considers the precision characteristics of the circuit board, emphasizing accuracy and gentleness when generating robot motion commands to avoid damage to the circuit board. During the robot motion command generation phase, the system accurately parses and generates robot motion trajectory commands, joint posture adjustment commands, and marking action commands based on the defect location coordinates, type, and severity output by the collaborative inference module. The motion trajectory command plans the optimal movement path based on the defect location coordinates, avoiding sensitive components and precision circuits on the circuit board. The movement speed is set to a low and constant speed to ensure the robot moves smoothly from its initial position to the defect location, avoiding inertial impact due to excessive speed. The joint posture adjustment command is generated based on the orientation of the circuit board surface where the defect is located. By precisely adjusting the rotation angle of each joint of the robot, it ensures that the marking mechanism maintains a parallel or slightly tilted adaptive posture with the defect surface, avoiding collisions or scratches between the marking mechanism and circuit board components. The marking action instructions are generated based on the defect type and severity. Red dot marks are used for critical defects such as cold solder joints and missing solder joints, while yellow thin lines are used for minor defects such as solder joint oxidation and blurred markings. The marking intensity is controlled at an extremely low level, leaving only slight, erasable marking marks on the circuit board surface without damaging solder joints, leads, or the substrate. The marking range is strictly limited to the outer edge of the defect to ensure that the electrical performance of the circuit board is not affected. During the data storage phase, the system categorizes and stores inspection data according to four dimensions: defect attributes, workpiece information, inspection process, and data type. The defect attribute dimension covers quantitative information such as defect type, severity, location coordinates, defect area, and depth. The workpiece information dimension includes traceability information such as the circuit board's unique identifier ID, production batch number, production time, material model, and production line station. The inspection process dimension records process information such as inspection timestamp, inspection equipment number, operator ID, and inspection environment temperature, humidity, and air pressure. The data type dimension stores information such as raw image data, visual feature vectors, inference result data packages, and robot motion command data packages. This systematic classification and storage method facilitates subsequent traceability of product quality, analysis of defect causes, and data support for model updates, ensuring that the inspection data of each circuit board is searchable and traceable.

[0038] The self-evolutionary update module continuously monitors the incremental status of circuit board soldering defect samples and the model detection status, focusing on indicators such as the number of newly added defect samples, the frequency of new defect types, and changes in model detection accuracy. When the number of newly added defect samples such as cold solder joints, missing solder joints, and solder joint oxidation reaches the preset training trigger condition, or when the model detection accuracy falls below a set threshold, the module automatically uses the multi-dimensional detection data stored in the execution storage module to launch the spatiotemporal driven evolutionary algorithm to incrementally update the defect identification model parameters. During the update process, the algorithm fully utilizes the difference between the confidence level of actual defects in historical detection data and the confidence level of defects predicted by the model, combined with the changing trend of the defect spatial distribution heatmap over time, to accurately adjust the model's weight parameters. This allows the model to quickly learn the characteristic patterns of new solder joint defects and adapt to the fine-tuning of process parameters such as soldering temperature and soldering time in the circuit board production line, as well as the changes in defect characteristics brought about by the soldering of new components. The optimized model parameters are synchronized to the collaborative inference module in real time, replacing the original parameters in the detection work. This allows the system to autonomously adapt to the dynamic changes in production line conditions, maintain stable and accurate detection performance in the long term, eliminate the need for frequent manual model debugging, and reduce production line operation and maintenance costs.

[0039] The heatmap optimization module, based on defect coordinate data, type, severity, and detection timestamps stored in the execution storage module, constructs a heatmap of the spatial distribution of defects on the production line using kernel density estimation and spatial interpolation algorithms. The heatmap can cover the X-axis (length), Y-axis (width), and Z-axis (height) of the circuit board, forming a three-dimensional spatial distribution. It can also generate two-dimensional heatmaps by production batch and inspection station to meet the needs of different analysis scenarios. During density calculation, the system assigns different weights based on defect severity. Defects that severely affect electrical performance, such as cold solder joints and missing solder joints, have a higher weight than secondary defects like solder joint oxidation and blurred markings. The system obtains the defect density value for each spatial unit by weighted summing of the number of defects per unit space and their weights. Higher density corresponds to a darker heatmap color, visually presenting high-risk areas for circuit board soldering defects, such as the circuit board area corresponding to a specific soldering station or the solder joint of a specific component, helping staff quickly locate high-risk areas for quality problems. The module deeply analyzes heatmap features, extracting the distribution trends and temporal evolution patterns of hotspot areas, such as whether defects spread from a certain area to the surrounding areas and whether the frequency of defect occurrence increases over time. Based on these features, it generates suitable inspection paths for the robot, guiding the robot to adjust its inspection trajectory, increasing the inspection frequency and coverage density of high-defect areas, and reducing the inspection time for defect-free or low-defect areas. This improves the targeting and efficiency of inspection work and avoids missing key risk points due to fixed trajectories. Simultaneously, the module outputs multimodal inspection reports. The visualization section includes a heatmap of defect spatial distribution, a pie chart of defect type distribution, and a time-dimensional trend curve, allowing staff to quickly grasp the overall distribution and changes of defects through intuitive charts. The data analysis section extracts high-incidence defect areas, high-frequency defect types, and inflection points in distribution trends, accurately identifying potential anomalies, such as parameter drift of a welding equipment or improper operation by operators. The application guidance section outputs robot inspection path optimization suggestions based on the analysis results, as well as adjustment directions for process parameters such as welding temperature, welding time, and flux dosage, providing a clear basis for production line process improvement. The report has a built-in anomaly warning mechanism. When the heat value suddenly surges or the defect distribution pattern deviates from the historical benchmark range, the system automatically triggers an early warning signal, accurately marks the abnormal area, and combines historical data to infer potential causes, reminding staff to stop the machine in time for investigation and avoid the production of batches of unqualified products.

[0040] This embodiment addresses the precision characteristics of circuit boards by employing a visual acquisition module to capture minute defect features, a collaborative reasoning module to rapidly locate and identify defects, an execution and storage module to achieve gentle yet accurate labeling and standardized data storage, a self-evolutionary update module to ensure the model adapts to dynamic changes in the production line, and a heatmap optimization module to optimize the detection path and provide timely anomaly warnings. The system is fully adapted to the high-speed, precision production needs of circuit board production lines, comprehensively covering the detection of welding-related defects, significantly improving quality inspection efficiency and accuracy, reducing the generation of batches of non-conforming products, and providing strong support for high-quality production in electronic component production lines.

[0041] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention in any way. Although the present invention has been disclosed above with reference to preferred embodiments, it is not intended to limit the present invention. Any person skilled in the art can make some modifications or alterations to the above-disclosed technical content to create equivalent embodiments without departing from the scope of the present invention. Any simple modifications, equivalent changes and alterations made to the above embodiments based on the technical essence of the present invention without departing from the scope of the present invention shall still fall within the scope of the present invention.

Claims

1. A robot-based defect identification system for production lines using vision positioning, characterized in that, The system includes: Visual acquisition module: Deploys industrial cameras and supporting light sources to acquire images of the areas to be inspected on the production line, preprocesses the images, and extracts the visual feature vectors of the images; Collaborative reasoning module: Deploys a lightweight neural network, combines the visual feature vectors output by the visual acquisition module, and performs parallel reasoning for visual localization and defect recognition simultaneously. It then uses a three-dimensional confidence fusion algorithm to comprehensively judge the reasoning results and outputs the detection results. Execution storage module: Based on the detection results output by the collaborative reasoning module, it parses and generates robot motion instructions, controls the robot to complete the defect marking operation, and classifies and stores the detection data; Self-evolution update module: Real-time monitoring of defect sample increment and model detection status. When the training trigger condition is met, relying on the detection data stored in the execution storage module, the defect identification model parameters are incrementally updated through the spatiotemporal driven evolution algorithm. The optimized model parameters are then synchronized to the collaborative inference module. Heatmap Optimization Module: Based on the defect coordinate data stored in the execution storage module, a heatmap of the spatial distribution of defects on the production line is constructed. The heatmap features are analyzed to generate suitable detection paths for the robot, guide the robot to adjust its detection trajectory, and output a multimodal detection report to complete defect distribution trend analysis and anomaly warning.

2. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the vision acquisition module, the images of the production line to be inspected are images of the surface and key parts of various workpieces to be inspected during the production process, including the key inspection points of the workpiece's machined surface, joints, and marked areas. During image acquisition, the industrial camera moves synchronously with the robot's end effector, and captures images of the key areas of each workpiece to be inspected according to the preset inspection point sequence of the production line. The supporting light source is turned on synchronously to assist the camera in clearly capturing the details of the workpiece surface. The core visual feature vector of the image is a set of key visual information that can characterize the defects of the workpiece surface, including three core contents: edge contour, texture distribution, and color features of the workpiece surface. During feature vector extraction, feature analysis is performed on the preprocessed image to capture the edge contour features of the workpiece surface, sort out the texture distribution rules of the image, extract the color feature information of the image, and integrate the three types of features to form a core visual feature vector that can accurately reflect the state of the workpiece surface.

3. The visual positioning-based robot inspection production line defect identification system according to claim 1, characterized in that, In the collaborative reasoning module, a lightweight neural network is deployed on the robot's local edge controller. It is a simplified neural network adapted to the computing power of the edge and is used to interface with the visual feature vectors output by the vision acquisition module. The lightweight neural network integrates a visual positioning reasoning branch and a defect recognition reasoning branch. The two reasoning branches are set up independently and run synchronously. The visual positioning reasoning branch retrieves production line scene features based on the input visual feature vector, compares them, establishes the correspondence between the input features and the reference features, completes the production line scene feature matching, and then converts the two-dimensional pixel coordinates in the image into three-dimensional physical coordinates that the robot can recognize based on the matched feature correspondence, thus completing the spatial coordinate conversion. The defect recognition reasoning branch distinguishes between normal and abnormal areas on the workpiece surface based on the same visual feature vector, and then identifies the defect-related information corresponding to the abnormal area.

4. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the collaborative reasoning module, the mathematical expression of the three-dimensional confidence fusion algorithm is: in, Indicates the confidence level of the defect; This indicates the potential defect target to be detected; Indicates the degree of visual feature matching; Represents visual feature vectors; Indicates the positioning accuracy weight; Indicates location coordinate information; This represents the thermal value indicating the spatial distribution of defects. and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. , , These are the weighting coefficients.

5. The visual positioning-based robot inspection production line defect identification system according to claim 1, characterized in that, In the collaborative reasoning module, the output positioning coordinates are the spatial coordinates of the defect on the production line workpiece. These coordinates are obtained by the visual positioning reasoning branch combining the visual feature vectors output by the visual acquisition module to map the defect position in the image to the actual spatial coordinate system of the production line. The defect type is the defect category corresponding to the abnormal area on the workpiece surface. This is determined by the defect recognition reasoning branch based on the visual feature vectors to distinguish between normal and abnormal areas of the workpiece, and then by matching the contour, texture, and color features of the abnormal area with the defect category features. The severity is the level of the defect's impact on the workpiece quality. Different levels are divided based on the defect confidence output by the three-dimensional confidence fusion algorithm and the production line quality standards.

6. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the execution storage module, robot motion commands control the robot to complete the defect marking operation. These commands include robot motion trajectory commands, joint posture adjustment commands, and marking action commands. The motion trajectory commands are generated based on the defect location coordinates to determine the robot's movement path and speed parameters from its initial position to the defect location. The joint posture adjustment commands are generated based on the orientation of the workpiece surface where the defect is located to ensure that the marking mechanism maintains an adapted posture with the defect surface. The marking action commands are generated in combination with the defect type and severity to determine the marking force, marking style, and marking range.

7. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the execution storage module, the detection data is categorized and stored according to four dimensions: defect attributes, workpiece information, detection process, and data type. Specifically, the data quantified by defect attributes includes defect type, severity, location coordinates, defect area, depth, and length; the data categorized by workpiece information includes workpiece unique identifier ID, batch number, production time, material, model, and production line station; the data categorized by detection process includes detection timestamp, detection equipment number, operator ID, and detection environment parameters; and the data categorized by data type includes raw image data output by the vision acquisition module, visual feature vectors, inference result data packets output by the collaborative inference module, and robot motion command data packets generated by the execution storage module.

8. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the self-evolutionary update module, the mathematical expression of the spatiotemporal driven evolutionary algorithm is: in, This represents the parameter update amount for the defect identification model; Represents the current set of parameters of the model; The learning rate; This represents the total number of defective samples participating in this incremental training. For defect sample indexing; For the i-th defect sample The confidence level of the true defect; For the i-th defect sample The model predicts the confidence level of defects; For model parameters The gradient vector; This represents the influence coefficient of the heat map. For time step; This is a heat map showing the spatial distribution of defects. and These represent the spatial coordinates of the workpiece in the horizontal and vertical directions of the production line, respectively. This represents the rate of change of the defect heatmap over time.

9. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, In the heatmap optimization module, a heatmap of the spatial distribution of production line defects is constructed based on the defect coordinate data stored in the execution storage module. The data covers the production line spatial coordinates, defect type, severity, and detection timestamp of each defect. Discrete defect points are converted into a continuous density field through kernel density estimation and spatial interpolation algorithms, covering the X-axis of the length direction, the Y-axis of the width direction, and the Z-axis of the height direction of the production line workpiece to form a three-dimensional spatial distribution. Two-dimensional heatmaps can also be generated by workstation batches. The density calculation is based on the weighted sum of the number and severity of defects in a unit space to obtain the defect density value of each spatial unit. The higher the density, the darker the corresponding color, which intuitively presents the risk concentration area. At the same time, the distribution trend and time evolution characteristics of hot spot areas are extracted.

10. The robot-based defect identification system for production lines based on vision positioning according to claim 1, characterized in that, The heatmap optimization module includes a multimodal inspection report comprising three parts: visualization, data analysis, and application guidance. The visualization part outputs a heatmap of the spatial distribution of defects on the production line, a pie chart of defect type distribution, and a time-dimensional trend curve. The data analysis part extracts high-incidence defect areas, high-frequency defect types, and distribution trend inflection points to identify potential anomaly risks. The application guidance part outputs robot inspection path optimization suggestions and production line process adjustment directions based on the analysis results. The report has a built-in anomaly warning mechanism that automatically triggers a warning signal when the heat value suddenly increases or the distribution pattern deviates from the historical benchmark, identifying and marking abnormal areas and potential causes. In addition, the report also includes a data summary, extracting key indicators.

Images