A smart camera vision guidance method, system, device and medium capable of rapid commissioning deployment

Abstract

This application relates to a method, system, device, and medium for intelligent camera visual guidance that can be quickly debugged and deployed, belonging to the field of visual guidance technology. The intelligent camera visual guidance method includes: parsing the visual detection task, extracting the theoretical operating area and target detection requirements, and constructing a theoretical visual task model; connecting and parsing the user layout interface to obtain the actual operating area, and calculating parameter correction coefficients and layout fusion coefficients in combination with the theoretical operating area; selecting and aggregating corresponding visual detection algorithms according to the target detection requirements, and generating a visual detection algorithm package in combination with the algorithm's basic parameters; correcting the visual detection algorithm package according to the target detection process and parameter correction coefficients to obtain a detection algorithm correction package; extracting deployment parameters to be verified based on the layout fusion coefficients and component layout position decomposition detection algorithm correction package; verifying the deployment parameters to be verified, and converting them into visual guidance signals to adjust the component frame layout in combination with the theoretical visual task model.

Description

Technical Field

[0001] This application relates to the field of visual guidance technology, and in particular to a smart camera visual guidance method, system, device and medium that can be quickly debugged and deployed. Background Technology

[0002] Machine vision systems convert the captured target into image signals using machine vision products, which are then transmitted to a dedicated image processing system to obtain the target's shape information. Based on pixel distribution, brightness, color, and other information, the image system converts this information into digital signals. The image system then performs various calculations on these signals to extract the target's features, and finally controls the on-site equipment based on the judgment results.

[0003] In the industrial field, machine vision can be used to detect defects in items, locate product positions, and confirm product quantities. It can also be linked with robotic arms to convert signals collected by vision cameras and guide the robotic arm to move to a designated position for processing, thereby improving installation accuracy and processing efficiency.

[0004] Existing patents disclose an initial visual guidance position correction method and system for workpiece grasping. First, the camera's internal parameters are calibrated and a spatial geometric model with three-dimensional coordinates is established. Multiple workpiece images are acquired, and the workpieces in the images are labeled using rectangular target detection boxes. Based on the labeled data, a target detection model based on a deep learning method is trained to obtain the target detection boxes and coordinate positions of the workpiece center in the scene. The target position is converted into a three-dimensional coordinate position according to the intrinsic and extrinsic parameters of the robotic arm and camera, and finally, the three-dimensional coordinates are converted into the final guidance coordinates. The robotic arm moves according to the guidance coordinates to complete the workpiece grasping. This invention, through a deep learning-based target detection method, can achieve efficient and high-precision workpiece positioning with strong robustness. Through the precise positioning of this device, other pose estimation devices can be connected after positioning to improve the accuracy of pose recognition.

[0005] The existing technical solutions mentioned above have the following drawbacks: 1. When the vision system is deployed to a new factory area, it needs to be debugged in combination with the on-site construction environment. However, the debugging time of conventional vision systems is too long, which affects delivery and production. Summary of the Invention

[0006] To address the shortcomings of existing technologies, the purpose of this application is to provide a smart camera vision guidance method, system, device, and medium that can be quickly debugged and deployed. By using low-code thinking, the vision inspection algorithms of each module are encapsulated to achieve visualization of the on-site debugging process of vision inspection, simplify the deployment process, and improve the efficiency of use and delivery. All modules involved in this system are deployed inside the smart camera, and the smart camera can be applied to various production lines to meet the product inspection needs of different customers.

[0007] This was achieved using the following technical solutions: In a first aspect, this application provides a smart camera vision guidance method that can be quickly debugged and deployed, comprising: Acquire and analyze the visual detection task, extract the theoretical operating area and target detection requirements, and construct a theoretical visual task model; Connect to and parse the user layout interface to obtain the actual operation area, and combine it with the theoretical operation area to calculate the parameter correction coefficient and layout fusion coefficient; Based on the target detection requirements, select and aggregate the appropriate visual detection algorithms, and combine the basic parameters of the algorithms to generate a visual detection algorithm package; Based on the target detection process and parameter correction coefficients, the visual detection algorithm package is corrected to obtain the detection algorithm correction package; Based on the layout fusion coefficient and the component layout position decomposition and detection algorithm, the correction package is used to extract the deployment parameters to be verified. Verify the deployment parameters to be verified, and combine them with the theoretical visual task model to convert them into visual guidance signals to adjust the component box layout.

[0008] By adopting the above technical solution, based on task parsing and component adaptation algorithms, correction coefficients are generated by comparing theoretical operating areas with actual parameters, the visual inspection algorithm is dynamically adjusted and associated with the device layout, and finally transformed into guidance signals to achieve precise component movement. An automated closed-loop deployment system from task modeling, algorithm correction to physical execution is constructed, which significantly improves the adaptability, deployment efficiency and execution accuracy of visual inspection.

[0009] This application is further configured to: acquire and parse a visual detection task, extract the theoretical operating region and object detection requirements, and construct a theoretical visual task model, including: The relevant documents for visual inspection tasks are analyzed to obtain the visual task types and task implementation environments. Analyze historical operational behaviors based on visual task types to extract theoretical operational areas; Based on the task implementation environment and the theoretical operation area, the visual inspection requirements are summarized to obtain the target task requirements; Based on the requirements of the target task and the type of visual task, the visual theoretical operation framework is associated and integrated to construct a theoretical visual task model.

[0010] By adopting the above technical solutions, the task documents are parsed based on natural language processing and knowledge graph construction algorithms to extract component requirements and detection requirements. The model-driven systems engineering approach is used to connect and fuse the visual computing framework to construct a theoretical visual task model. This achieves an integrated and standardized transformation from requirements to model, significantly improving the accuracy and efficiency of visual inspection system design.

[0011] This application further includes: connecting and parsing the user layout interface to obtain the actual operation area, and combining it with the theoretical operation area to calculate the parameter correction coefficient and layout blending coefficient, including: The user layout interface is connected and self-checked according to the communication protocol to form a project interface layout cluster. The project interface layout cluster is parsed using the parameter configuration tool to obtain the actual operation area; Based on the component type, the actual operation area and the theoretical operation area are mapped and compared to construct an attribute comparison mapping table; Based on the attribute comparison mapping table, the attribute parameters are corrected and the parameter correction coefficients are calculated. The guide interface is measured, and the guide component layout is obtained by combining the component box layout. The layout of the guiding components is compared and verified based on the theoretical layout of the components in the theoretical operation area, and the layout fusion coefficient is calculated.

[0012] By adopting the above technical solution, automatic connection and information parsing of component clusters are realized based on protocol parsing and device self-testing algorithms. Parameter correction coefficients are calculated through attribute mapping and deviation analysis algorithms, and fusion coefficients are obtained by comparing actual and theoretical layouts using spatial geometric analysis. This significantly improves the accuracy and environmental adaptability of the visual inspection system deployment.

[0013] This application further specifies: selecting and aggregating relevant visual detection algorithms according to object detection requirements, and generating a visual detection algorithm package based on the algorithm's basic parameters, including: The target detection requirements are analyzed and refined, and task subcategories, performance indicators, and display formats are extracted. The algorithm framework is matched according to the task subclass, and the target algorithm model is determined by combining the actual operation area. If the actual operating area is an embedded device, then select the lightweight model; Visual processing algorithms are integrated and merged based on performance metrics to generate an algorithm aggregation list; The algorithm aggregation sheets are arranged in chronological order according to the detection process flow, and the algorithm implementation time sequence sheets are obtained by combining the basic parameters of each algorithm. The output algorithm of the target recognition result is bound to the display format to determine the inference and labeling algorithm; Based on the process identifier, the algorithm implementation timeline, inference labeling algorithm, and target algorithm model are encapsulated to obtain the visual detection algorithm package.

[0014] By adopting the above technical solution, based on the requirements decomposition and component awareness algorithm, lightweight or full models are matched by task subclasses, multiple algorithms are connected and integrated according to performance indicators and time-series orchestration is performed, and finally a visual inspection algorithm package adapted to different computing power and processes is encapsulated and generated, which significantly improves the accuracy and adaptability of the inspection system and the deployment flexibility.

[0015] This application further specifies: based on the target detection process and parameter correction coefficients, the visual detection algorithm package is corrected to obtain a detection algorithm correction package, including: The target detection process is broken down into several process implementation segments based on the process implementation timestamp; Based on the process implementation stage, the visual inspection algorithm package is mapped and aligned to determine the process parameters to be corrected. The parameter correction coefficients are matched with the process parameters to be corrected based on the parameter attributes to obtain parameter correction pairs; The parameter correction pair is checked and verified according to the attribute boundary constraint interval; if the parameter correction pair is not located in the attribute boundary constraint interval, the process parameter to be corrected in the current parameter correction pair is replaced with the upper or lower limit value of the attribute boundary constraint interval. No, then keep the current parameter correction unchanged; Based on parameter correction, the visual detection algorithm package is integrated and packaged to generate a detection algorithm correction package.

[0016] By adopting the above technical solution, based on the process time sequence decomposition and parameter mapping matching algorithm, the parameters to be corrected are identified by process segment alignment, and the out-of-limit values ​​are dynamically adjusted by combining boundary constraint verification. Finally, a detection algorithm correction package that is accurately adapted to the target detection process is generated, which significantly improves the adaptive adjustment capability and deployment accuracy of the vision algorithm to the actual working conditions.

[0017] This application further specifies: based on the layout blending coefficient and the component layout position decomposition detection algorithm, the deployment parameters to be verified are extracted, including: The detection algorithm correction package is deconstructed according to the data processing flow to determine the corresponding data operation module; Based on the component layout, the data processing modules are associated and decomposed to determine the module processing chain; The module processing chain is logically verified based on the layout fusion coefficient, and deployment-related parameters are extracted. Sort deployment-related parameters according to performance metrics and mark parameter priorities; Based on parameter priority and theoretical benchmark thresholds, deployment-related parameters are filtered to obtain deployment parameters to be verified.

[0018] By adopting the above technical solution, the algorithm package is decomposed into a module processing chain based on the deconstruction and mapping algorithm. The deployment parameters are extracted by logical verification in combination with the layout fusion coefficient. The core parameters to be verified are selected according to priority and theoretical threshold. This achieves fine-grained adaptation between visual algorithm and component layout, and significantly improves the accuracy of deployment parameters and verification efficiency.

[0019] This application is further configured to: verify the deployment parameters to be verified, and, in conjunction with the theoretical visual task model, convert them into visual guidance signals to adjust the component box layout, including: Based on the parameter category and component type, perform on-site verification of the deployment parameters to be verified, and calculate the parameter deviation value; If the parameter deviation value is within the preset deviation parameter range, the current deployment parameter to be verified is determined to have passed the verification and remains unchanged; No, then replace the current deployment parameters to be verified with the actual measured values; Based on the deployed user layout interface, target objects in the work scene are detected in real time, and target detection information is output. The target detection information is analyzed, the evaluation index is calculated, and the results are compared with the performance index in the theoretical vision task model. If any evaluation indicator fails to meet the current performance indicator, the non-compliant indicator is traced back to determine the corresponding attribute parameter and calculate the deviation between the current attribute parameter and the theoretical attribute parameter. Based on the deviation and the component layout control model, visual guidance signals are generated and transmitted and the component frame layout is adjusted in conjunction with the communication protocol.

[0020] By adopting the above technical solution, based on real-time detection and deviation analysis algorithms, the source of deviation is intelligently traced through on-site parameter verification and performance index comparison. Signal conversion and motion control algorithms are used to generate guidance commands to drive the precise movement of components. An adaptive closed-loop optimization system from parameter verification to execution is constructed, which significantly improves the deployment accuracy and response efficiency of the vision inspection system.

[0021] Secondly, this application also provides a smart camera vision guidance system that can be quickly debugged and deployed, employing the following technical solution: A rapidly deployable and debuggable intelligent camera vision guidance system, used to implement an intelligent camera vision guidance method, comprising: The task construction module is used to acquire and parse visual detection tasks, extract theoretical operating areas and target detection requirements, and build theoretical visual task models. The deviation calculation module is used to parse the user layout interface, obtain the actual operation area, and calculate the parameter correction coefficient and layout fusion coefficient in combination with the theoretical operation area. The algorithm aggregation module is used to select and aggregate the corresponding visual detection algorithms according to the target detection requirements, and generate a visual detection algorithm package by combining the basic parameters of the algorithms. The parameter correction module is used to correct the visual detection algorithm package based on the target detection process and parameter correction coefficients to obtain the detection algorithm correction package. The layout extraction module is used to extract the deployment parameters to be verified based on the layout fusion coefficient and the component layout position decomposition and detection algorithm correction package. The guidance control module is used to verify the deployment parameters to be verified and, in conjunction with the theoretical visual task model, convert them into visual guidance signals to adjust the layout of the component boxes.

[0022] By adopting the above technical solutions, the visual inspection algorithm is dynamically corrected based on task parsing and component adaptation algorithms. Guidance signals are generated through parameter verification and deviation analysis to achieve precise movement. An automated closed-loop deployment system from task modeling to physical execution is constructed, which significantly improves the adaptability, deployment efficiency and execution accuracy of visual inspection.

[0023] Thirdly, this application also provides an electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0024] Fourthly, this application also provides a storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the intelligent camera vision guidance method that can be quickly debugged and deployed as described above.

[0025] In summary, the beneficial technical effects of this application are as follows: Based on task parsing and component adaptation algorithms, the visual inspection algorithm is dynamically corrected. Guidance signals are generated through parameter verification and deviation analysis to achieve precise movement. An automated closed-loop deployment system from task modeling to physical execution is constructed, which improves the adaptability, deployment efficiency and execution accuracy of visual inspection. By encapsulating the visual inspection algorithms of each module using low-code thinking, the visual inspection on-site debugging process is visualized, simplifying the deployment process, improving usage efficiency, and increasing delivery efficiency. Attached Figure Description

[0026] Figure 1 This is a schematic diagram of the overall process of the intelligent camera visual guidance method in this application; Figure 2This is a flowchart illustrating step S4 in the intelligent camera visual guidance method of this application; Figure 3 This is a schematic diagram of the intelligent camera vision guidance system in this application. Detailed Implementation

[0027] The present application will be further described in detail below with reference to the accompanying drawings.

[0028] Reference Figure 1 This application discloses a method for visual guidance of a smart camera that can be quickly debugged and deployed, comprising: S1: Acquire and parse the visual detection task, extract the theoretical operating area and target detection requirements, and construct a theoretical visual task model; S2: Connect and parse the user layout interface, obtain the actual operation area, and calculate the parameter correction coefficient and layout fusion coefficient in combination with the theoretical operation area; S3: Select and aggregate the appropriate visual detection algorithms according to the target detection requirements, and generate a visual detection algorithm package by combining the basic parameters of the algorithms; S4: Based on the target detection process and parameter correction coefficients, correct the visual detection algorithm package to obtain the detection algorithm correction package; S5: Correct the package based on the layout fusion coefficient and the component layout position decomposition and detection algorithm, and extract the deployment parameters to be verified; S6: Verify the deployment parameters to be verified, and combine them with the theoretical visual task model to convert them into visual guidance signals to adjust the component box layout.

[0029] In this embodiment, when deploying an automated optical inspection system in a semiconductor wafer manufacturing plant, the wafer dicing defect detection task issued by the production management system is first acquired and analyzed. The theoretical operating area (such as the center coordinates of the 12 chip units to be inspected on each wafer and the field of view under a 200x microscope objective) and target detection requirements (such as dicing width deviation ≤ 0.5μm and chipping length ≤ 3μm) are extracted from the process documents to construct a theoretical visual task model containing the ideal detection path and parameter thresholds.

[0030] Subsequently, when connecting to and parsing the deployment engineer's UI layout interface, the system obtains the actual operating area manually calibrated on the actual machine through drag-and-drop interaction (e.g., the overall field of view shifts by 15μm and rotates by 0.2° due to the equipment base's levelness deviation), and performs spatial registration with the theoretical area. It automatically calculates parameter correction coefficients including translation ΔX = +15μm and rotation angle θ = +0.2°, as well as a layout fusion coefficient (scaling factor 0.9998) reflecting the mapping relationship between the theoretical and actual layouts. Based on this, the system selects the dicing track localization algorithm, edge extraction algorithm, and defect classification algorithm from the algorithm library according to the target detection requirements, and loads the default algorithm basic parameters (e.g., Ca). The edge detection threshold range is 50-120, and the minimum defect area is 5 pixels. These are aggregated into an initial visual inspection algorithm package. Based on the target detection process (e.g., the current batch of wafers uses a copper process, with a reflectivity 15% higher than standard silicon wafers) and parameter correction coefficients, the exposure time and gain parameters in the algorithm package are adaptively corrected (the exposure time is adjusted from the default 10ms to 8ms, and the gain is reduced from 18dB to 15dB), generating a detection algorithm correction package. Based on the layout fusion coefficient, the correction algorithm package is decomposed according to the actual component layout position, and the coordinate offset of 12 detection units, the actual scanning start point of each field of view, and the corresponding light source brightness compensation value are extracted to verify the deployment parameters.

[0031] Finally, during the verification process, the system substitutes the deployment parameters to be verified into the theoretical vision task model for virtual simulation, confirming that all detection units are within the effective field of view and the edge extraction error is less than the preset ±0.1μm. Based on this, a visual guidance signal is generated, and the layout of the component boxes in the UI interface is dynamically adjusted. The green highlighted box on the display prompts the engineer to confirm the consistency between the actual coverage area of ​​each detection unit and the theoretical position, thereby completing the high-precision AOI system deployment guidance.

[0032] Preferably, step S1 includes: The relevant documents for visual inspection tasks are analyzed to obtain the visual task types and task implementation environments. Analyze historical operational behaviors based on visual task types to extract theoretical operational areas; Based on the task implementation environment and the theoretical operation area, the visual inspection requirements are summarized to obtain the target task requirements; Based on the requirements of the target task and the type of visual task, the visual theoretical operation framework is associated and integrated to construct a theoretical visual task model.

[0033] In this embodiment, a detailed description of the visual inspection task is obtained through user input, task documents, or system instructions. The task description may include the detection target, application scenario, performance indicators, environmental conditions, etc.

[0034] Determine the type of visual inspection task, such as defect detection, target localization, classification and counting, size measurement, behavior recognition, etc.

[0035] Define the physical environment in which the task takes place (indoor / outdoor, lighting conditions, background complexity), the characteristics of the object being measured (size, shape, material, motion state), and the real-time requirements (such as frame rate and latency).

[0036] Based on the detection accuracy and field of view requirements, the theoretical parameters of the required camera are derived: Resolution: Calculated from the minimum detectable target size and field of view, ensuring that each target occupies enough pixels; Frame rate: Determined based on the target's motion speed and detection frequency requirements; Sensor type: such as CMOS or CCD, considering sensitivity, dynamic range, and noise characteristics; Lens parameters: focal length, aperture, depth of field, to ensure clear imaging and coverage of the required field of view.

[0037] Lighting system requirements: Analyze whether the ambient lighting is sufficient, whether supplementary light sources (such as ring lights, backlights, structured lights) are needed, and the requirements for the color, intensity, and uniformity of the light sources.

[0038] Computing component requirements: Estimate the theoretical configuration of processors (CPU / GPU), memory, storage, etc., required based on algorithm complexity and real-time requirements.

[0039] Other components: such as trigger sensors (photoelectric switches, encoders), mechanical supports, protective housings, etc., are extracted according to the site conditions.

[0040] Clearly define the specific target categories to be inspected from the task description (such as product model, defect type, object location), and establish a list of target attributes (shape, color, texture, size range).

[0041] Accuracy requirements: accuracy, recall, false positive rate, false negative rate, etc., may need to reach 99% or higher.

[0042] Speed ​​requirements: single frame processing time, system throughput (e.g., how many objects are detected per second).

[0043] Stability requirements: Robustness under different lighting conditions, angles, and occlusions.

[0044] Output requirements: the format of the detection results (e.g., coordinate boxes, classification labels, defect levels), the output data format (JSON, image annotation), and whether real-time visualization is required.

[0045] Based on the task type and the trade-off between accuracy and speed, a suitable theoretical model architecture is selected: For general object detection, YOLO, Faster R-CNN, SSD, etc. can be considered.

[0046] For defect detection, segmentation-based networks (U-Net) or anomaly detection models can be considered.

[0047] For classification purposes, ResNet, EfficientNet, etc. can be considered.

[0048] Input: Image size (e.g., 640×480), color space (RGB / grayscale), preprocessing method (normalization, data augmentation).

[0049] Output: target category, bounding box coordinates, confidence score, or pixel-level segmentation map.

[0050] Algorithm flow design: Describes the theoretical flow from image acquisition to result output, including preprocessing, feature extraction, detection head design, and postprocessing (NMS, threshold filtering).

[0051] Performance prediction: Based on the complexity of the theoretical model, estimate the running speed (FPS) and memory usage on the specified components to ensure that real-time requirements are met.

[0052] Evaluation metrics setting: Define the evaluation metrics for model training and testing, such as mAP, IoU, and F1-score, and align them with the accuracy requirements of the task.

[0053] Preferably, step S2 includes: The user layout interface is connected and self-checked according to the communication protocol to form a project interface layout cluster. The project interface layout cluster is parsed using the parameter configuration tool to obtain the actual operation area; Based on the component type, the actual operation area and the theoretical operation area are mapped and compared to construct an attribute comparison mapping table; Based on the attribute comparison mapping table, the attribute parameters are corrected and the parameter correction coefficients are calculated. The guide interface is measured, and the guide component layout is obtained by combining the component box layout. The layout of the guiding components is compared and verified based on the theoretical layout of the components in the theoretical operation area, and the layout fusion coefficient is calculated.

[0054] In this embodiment, according to the system design scheme, components such as camera, lens, light source, trigger sensor, and computing unit (industrial control computer / embedded device) are correctly connected according to electrical and communication interfaces (such as USB, GigE, Camera Link, GPIO).

[0055] Install drivers and software development kits (SDKs) to ensure the system can recognize and communicate with the device.

[0056] Perform a self-test command on each component to confirm that it is functioning normally. Perform sensor calibration on the camera (such as dead pixel correction and field flatness correction), and perform brightness stability testing on the light source.

[0057] Read the device's inherent properties through the SDK or configuration tools, for example: Camera: Sensor model, resolution, pixel size, dynamic range, signal-to-noise ratio, maximum frame rate, data bit depth.

[0058] Lens: focal length, aperture range, minimum focusing distance, distortion parameters, interface type.

[0059] Light source: type (LED / halogen / laser), wavelength / color temperature, rated power, and illumination uniformity data.

[0060] Computing unit: CPU / GPU model, memory capacity, storage type, network bandwidth.

[0061] Dynamic parameter acquisition: Reads the currently configured operating parameters, such as exposure time, gain, white balance, trigger mode, light source brightness level, etc., and records their set values.

[0062] The above information is compiled into an actual component parameter table, which includes device ID, model, serial number, firmware version, and key parameter values.

[0063] Use laser rangefinders, protractors, or 3D scanning equipment to measure key spatial data: Camera installation height, pitch angle, yaw angle, roll angle; working distance from the lens optical center to the surface of the object being measured; relative positions of multiple cameras (baseline length, included angle); installation position of the light source, illumination angle, and distance from the object being measured.

[0064] Photograph the calibration board or a reference object of known size, calculate the actual field of view (FOV), and compare it with the theoretical design value.

[0065] For multi-camera systems, the actual overlap ratio of each camera's field of view is calculated by capturing calibration patterns of the overlapping areas, and the coordinates of the overlapping boundaries are recorded.

[0066] From the previously constructed theoretical vision task model, the following theoretical values ​​are obtained: required camera resolution, frame rate, and sensor sensitivity requirements; ideal working distance, field of view, and depth of field; light source illuminance, color temperature, and uniformity requirements; and processor computing power requirements (such as TFLOPS).

[0067] The theoretical and actual parameters are arranged by category to form a comparison matrix, which clarifies the parameter items that need to be corrected.

[0068] For each quantifiable parameter, calculate the deviation between the actual and theoretical values: Resolution deviation: The ratio of actual resolution (pixels) to theoretical resolution.

[0069] Field of view deviation: The difference between the actual field of view angle and the theoretical field of view angle.

[0070] Illuminance deviation: The ratio of actual illuminance to theoretically required illuminance.

[0071] Distortion variables: The difference between the lens distortion coefficients (radial and tangential) and the ideal model is calculated through calibration.

[0072] Depth of field deviation: The intersection / difference between the actual depth of field range and the theoretical depth of field range.

[0073] Comprehensive deviation assessment: Analyze whether the deviation is within the allowable tolerance range. If it exceeds the tolerance, the component or task model needs to be readjusted.

[0074] Parameter correction coefficients include: geometric correction coefficients (scale correction: if the actual resolution does not match the theoretical resolution, calculate the scaling factor kscale = theoretical pixel size / actual pixel size, used for image scaling or target size conversion. Distortion correction parameters: generate a distortion correction mapping table based on the calibration results, used for subsequent image distortion correction. Perspective correction: if the camera mounting angle deviates from the theoretical angle, calculate the perspective transformation matrix to correct the image to an ideal front view). Photometric correction factor (brightness correction: the ratio of actual light source illuminance to theoretical value, used to calculate gain coefficient for image brightness normalization. White balance correction: if the light source color temperature does not match the theory, calculate the color correction matrix (CCM) to adjust the image color); Performance correction factors (Frame rate compensation: If the actual frame rate is lower than the theoretical frame rate, calculate the time delay factor and adjust the algorithm processing frequency or introduce frame interpolation. Signal-to-noise ratio correction: Adjust the algorithm detection threshold or noise reduction intensity according to the actual signal-to-noise ratio.) The layout fusion coefficients include: coordinate unification transformation matrix (for multi-camera systems, the rotation matrix R and translation vector t from each camera coordinate system to the global coordinate system are calculated through joint calibration to achieve coordinate unification); field of view overlap weight (based on the size and position of the overlapping area, the pixel weights during fusion are calculated, for example, linear weighting or maximum value fusion is used in the overlapping area); stitching parameters (transformation parameters for image stitching are generated (such as homography matrix) to ensure seamless stitching of multiple images); depth of field fusion coefficient (if multi-focal plane shooting is involved, the focus metric of each image (such as Laplacian variance) is calculated to generate a depth of field fusion weight map).

[0075] Preferably, step S3 includes: The target detection requirements are analyzed and refined, and task subcategories, performance indicators, and display formats are extracted. The algorithm framework is matched according to the task subclass, and the target algorithm model is determined by combining the actual operation area. If the actual operating area is an embedded device, then select the lightweight model; Visual processing algorithms are integrated and merged based on performance metrics to generate an algorithm aggregation list; The algorithm aggregation sheets are arranged in chronological order according to the detection process flow, and the algorithm implementation time sequence sheets are obtained by combining the basic parameters of each algorithm. The output algorithm of the target recognition result is bound to the display format to determine the inference and labeling algorithm; Based on the process identifier, the algorithm implementation timeline, inference labeling algorithm, and target algorithm model are encapsulated to obtain the visual detection algorithm package.

[0076] In this embodiment, based on the previously extracted target detection requirements, the specific task subcategories are further clarified: general target detection (requiring localization and classification), instance segmentation (requiring pixel-level contours), key point detection, or defect detection? Target size range: large targets, small targets, multi-scale targets? Target motion state: static, low-speed motion, high-speed motion? Background complexity: Simple and uniform background, complex and cluttered background, dynamic background? Quantification of performance metrics: Extract specific quantitative metrics from the requirements, including: accuracy: mAP@0.5, mAP@0.5:0.95, recall rate, false positive rate, etc.

[0077] Speed: Single frame processing time (milliseconds), FPS, latency requirements.

[0078] Robustness: tolerance to changes in illumination, tolerance to occlusion, and tolerance to changes in viewing angle.

[0079] Output format requirements: Clearly define the expression format of the detection results (bounding box coordinates, segmentation mask, confidence score, category label) and its coordinate system (image pixel coordinates, world coordinates).

[0080] Based on task subclasses, suitable basic algorithm models are selected from existing algorithm libraries (such as MMDetection, Detectron2, YOLO series, TensorFlow Object Detection API): Single-stage detectors: YOLOv5 / v8 / v9, SSD, RetinaNet (fast, suitable for real-time).

[0081] Two-stage detectors: Faster R-CNN, Mask R-CNN (high accuracy, suitable for small targets).

[0082] Transformer-based detectors: DETR, Deformable DETR (suitable for complex scenarios).

[0083] Segmentation models: U-Net, DeepLab, Mask2Former (for instance segmentation).

[0084] Lightweight models: MobileNet-SSD, EfficientDet (suitable for edge devices).

[0085] Based on the actual operating area (computing power of computing units, memory bandwidth), algorithms that cannot meet real-time requirements are eliminated. For example, if using an embedded GPU (such as Jetson Nano), lightweight models are preferred; if using a high-performance server, large models can be selected.

[0086] When a single algorithm cannot meet all requirements, an aggregation scheme is designed, including: Cascaded detection: First, a fast coarse detector is used to filter candidate regions, and then a high-precision fine detector is used for accurate identification (such as YOLO detection followed by ResNet classification).

[0087] Multi-scale fusion: Combining models with different input scales to improve multi-scale target detection capabilities.

[0088] Ensemble learning: Training multiple models with different initializations for the same task, and voting or averaging the results during inference to improve robustness.

[0089] Divide and conquer: Decompose complex scenes into multiple sub-tasks (such as foreground segmentation + object detection), process them separately with dedicated models, and finally fuse the results.

[0090] The parameter correction coefficients (such as distortion correction and color correction) and layout fusion coefficients (such as multi-camera stitching weights) calculated in the early stages can be used as preprocessing or post-processing modules and seamlessly integrated with the detection algorithm. For example, distortion correction can be applied before the input image, or coordinate transformation can be applied after multi-camera detection to unify the results.

[0091] Input size determination: Set the model input size (e.g., 640×640, 1280×720) based on task requirements, component computing power, and actual image resolution. Considering the deviation between the actual and theoretical field of view, it may be necessary to scale the image to match the model training scale.

[0092] Anchor box design: If the algorithm relies on anchor points (such as YOLO, Faster R-CNN), calculate the size and proportion of a custom anchor box based on the target size distribution and the actual pixel size, and replace the default anchor points to improve positioning accuracy.

[0093] Post-processing parameters: Set the IoU threshold, confidence threshold, and category filtering rules for non-maximum suppression (NMS) to balance recall and false positive rate.

[0094] Data augmentation strategies: Based on the characteristics of actual ambient lighting and noise, design online data augmentation (such as random brightness adjustment, Gaussian noise, occlusion simulation) to enhance the robustness of the model.

[0095] Loss function weights: If it is necessary to adjust the balance between classification and regression losses, set the corresponding weight coefficients.

[0096] Dataset preparation: Collect or label image data that matches the actual scene, and divide it into training, validation, and test sets proportionally. If a pre-trained model already exists, use transfer learning and fine-tune it with only a small amount of target domain data.

[0097] Training parameter configuration: Set batch size, learning rate, optimizer (such as SGD, Adam), number of iterations, learning rate decay strategy, and use the validation set to monitor overfitting.

[0098] Model training and tuning: Perform training, periodically evaluate validation set metrics, and adjust hyperparameters based on the results. If performance is unsatisfactory, return to step three to adjust the aggregation strategy or replace the base model.

[0099] Model export: Export the trained model to a standard format (such as ONNX, TensorRT, OpenVINO) for deployment.

[0100] Preprocessing module embedding: Image preprocessing steps are integrated into the front end of the algorithm, including: applying distortion correction (based on the geometric correction coefficient in step six), applying color correction (based on the photometric correction coefficient), image scaling, normalization, and channel conversion.

[0101] Embedded post-processing module: Converts detection results (pixel coordinates) into world coordinates or a unified coordinate system through a coordinate transformation matrix, fuses multi-camera results (such as stitching and weighted averaging), and applies NMS and threshold filtering.

[0102] Inference engine encapsulation: The entire processing flow is encapsulated into callable functions or service interfaces, supporting single image, video stream, or batch processing. Consider using asynchronous and pipelined optimizations to improve throughput.

[0103] Preferably, step S4 includes: The target detection process is broken down into several process implementation segments based on the process implementation timestamp; Based on the process implementation stage, the visual inspection algorithm package is mapped and aligned to determine the process parameters to be corrected. The parameter correction coefficients are matched with the process parameters to be corrected based on the parameter attributes to obtain parameter correction pairs; The parameter correction pair is checked and verified according to the attribute boundary constraint interval; if the parameter correction pair is not located in the attribute boundary constraint interval, the process parameter to be corrected in the current parameter correction pair is replaced with the upper or lower limit value of the attribute boundary constraint interval. No, then keep the current parameter correction unchanged; Based on parameter correction, the visual detection algorithm package is integrated and packaged to generate a detection algorithm correction package.

[0104] In this embodiment, the process requirements are extracted by clarifying the specific process requirements from the target detection task description, including: the characteristics of the object to be detected (shape, size, material, surface features), the detection environment conditions (light type, background interference, motion state), the detection accuracy requirements (allowable error range, minimum detectable defect size), the detection cycle time and real-time requirements (single detection time, system throughput), and the output result format (coordinates, size, classification label, image annotation).

[0105] Process and Algorithm Mapping: This involves mapping process requirements to each stage of the algorithm processing flow, identifying which stages may be affected by component deviations and require correction. For example: image acquisition is affected by distortion and uneven lighting; target localization is affected by scale deviation and perspective distortion; target classification is affected by color distortion and sharpness; and multi-sensor fusion is affected by coordinate system deviation.

[0106] Read the following correction coefficients from the generated configuration file (such as JSON / YAML): Geometric correction factors: distortion correction parameters (radial and tangential distortion coefficients), perspective transformation matrix, and scale factor.

[0107] Luminous correction factors: luminance gain factor, color correction matrix (CCM), white balance factor.

[0108] Layout fusion coefficients: multi-camera coordinate transformation matrix (rotation matrix R, translation vector t), field of view overlap weight map, and stitching homography matrix.

[0109] Performance correction factors: frame rate compensation factor, signal-to-noise ratio adjustment threshold.

[0110] Check whether the correction coefficients are within a reasonable range, such as whether the distortion coefficients are too large or whether the transformation matrix is ​​invertible, to avoid algorithm failure due to calibration errors.

[0111] The preprocessing module corrections include: image distortion correction integration: a distortion correction step is inserted into the algorithm preprocessing flow, using geometric correction coefficients to remap the input image, eliminating lens distortion and perspective distortion. This can be implemented using OpenCV's initUndistortRectifyMap and remap functions.

[0112] Illumination and Color Correction: A brightness gain coefficient is applied to adjust the image brightness globally or locally, making the image brightness approximate the theoretical training data distribution. A color correction matrix is ​​applied to perform color conversion on the RGB image to compensate for differences in light source color temperature and ensure the consistency of color features.

[0113] Size normalization adjustment: Adjust the image scaling ratio according to the scale scaling factor to match the actual pixel size with the input size expected by the algorithm. If the actual resolution deviates from the theoretical value, the aspect ratio or padding must be maintained during scaling.

[0114] Model input parameter correction includes: Input size reconfiguration: Adjusting the size parameters of the model input layer based on the deviation between the actual image resolution and the theoretical model input size. If the model supports dynamic input, these parameters are set directly; otherwise, scaling or cropping needs to be added in preprocessing, and the mapping relationship between anchor boxes and output coordinates needs to be adjusted accordingly. Anchor box and ROI correction: Based on the actual target size distribution (converted from the ratio of actual pixel size to theoretical value), the size and ratio of the anchor boxes are recalculated, replacing the model's default anchors to improve positioning accuracy.

[0115] If the algorithm uses candidate regions (such as RPN in Faster R-CNN), adjust the candidate box generation parameters (such as anchor scales and ratios) according to the actual field of view.

[0116] Mean and standard deviation adjustment: If the photometric correction changes the brightness distribution of the image, the normalization parameters (mean and standard deviation) of the model input need to be adjusted accordingly to keep them consistent with those during training.

[0117] Internal model parameter adjustments include: Transfer learning fine-tuning: If component deviations are significant (e.g., a new lens introducing different distortion modes, or a light source altering color distribution), and preprocessing alone cannot fully compensate for them, consider using a small amount of on-site collected data to fine-tune the model. Collect labeled on-site images (a small number is sufficient). Freeze most network layers, fine-tuning only the last few layers or specific branches to adapt the model to the new data distribution. During fine-tuning, keep the preprocessing correction coefficients unchanged and use the corrected images as input.

[0118] Depending on the performance correction coefficients of the actual computing units (such as frame rate compensation), the model may need to be quantized (INT8) or pruned to meet real-time requirements with limited computing power.

[0119] The post-processing module corrections include: coordinate transformation and mapping: the coordinates of the detection box output by the model (image pixel coordinate system) are transformed to the actual physical coordinate system (such as world coordinates, robot base coordinates) through the inverse transformation of geometric correction coefficients or coordinate transformation matrix to meet the process requirements.

[0120] For multi-camera systems, the detection results of each camera are unified to the global coordinate system using layout fusion coefficients (transformation matrices) and then fused (e.g., weighted averaging, non-maximum suppression across cameras).

[0121] Multi-view fusion: In overlapping areas, detection results are weighted and fused according to the field-of-view overlap weight map to avoid duplicate counting or boundary jumps. For the stitched image, a stitching homography matrix is ​​applied to ensure the accurate positioning of the detection results on the stitched image.

[0122] Confidence and Threshold Adjustment: Adjust the detection confidence threshold based on the actual signal-to-noise ratio correction coefficient. If the ambient noise is high, appropriately increase the threshold; if the lighting is weak and the features are not obvious, appropriately decrease the threshold. Dynamically adjust the IoU threshold and classification threshold of NMS according to the allowable false detection rate of the process.

[0123] The above correction steps are integrated into a unified algorithm processing pipeline, forming a detection algorithm correction package, which includes: a corrected preprocessing module (including distortion correction, color correction, etc.); a model file with corrected input parameters (or a model with fine-tuned weights); a corrected post-processing module (including coordinate transformation and fusion logic); and a correction coefficient configuration file and loader.

[0124] Verify the correctness of each correction module using typical test images: Check if the image is flat after distortion correction. Check if the color chart is close to the standard after color correction. Check if the detection box matches the physical position after coordinate transformation.

[0125] Run the complete correction package on the actual component layout interface, collect a batch of field images, calculate the detection accuracy (mAP, recall) and processing speed, and compare them with the process requirements to ensure compliance.

[0126] Preferably, step S5 includes: The detection algorithm correction package is deconstructed according to the data processing flow to determine the corresponding data operation module; Based on the component layout, the data processing modules are associated and decomposed to determine the module processing chain; The module processing chain is logically verified based on the layout fusion coefficient, and deployment-related parameters are extracted. Sort deployment-related parameters according to performance metrics and mark parameter priorities; Based on parameter priority and theoretical benchmark thresholds, deployment-related parameters are filtered to obtain deployment parameters to be verified.

[0127] In this embodiment, the generated detection algorithm correction package is deconstructed to clarify its component modules: Preprocessing module: Image distortion correction, color correction, size normalization, etc., relying on geometric correction coefficients and photometric correction coefficients.

[0128] Core detection module: Loads the model file with corrected parameters (such as anchor box adjustment, input size configuration), and may fine-tune the weights.

[0129] Post-processing module: coordinate transformation, multi-view fusion, confidence threshold adjustment, depending on layout fusion coefficients (transformation matrix, overlap weight, splicing parameters).

[0130] Draw a flowchart of the algorithm processing, labeling the input data source (single camera / multiple cameras), output result format, and parameters passed between modules for each module. Identify which parameters are strongly related to component layout and position (e.g., coordinate transformation matrix) and which are only related to inherent component properties (e.g., distortion coefficient).

[0131] Obtain the actual deployment location information of each vision sensor (camera) from on-site installation records or 3D measurement data: Absolute position: The three-dimensional coordinates (x, y, z) of the camera in the world coordinate system.

[0132] Attitude angles: pitch, yaw, roll, or the corresponding rotation matrix.

[0133] Relative position: The baseline length and relative angle between cameras in a multi-camera system.

[0134] Field of view: The actual measured field of view boundary, working distance, and depth of field range.

[0135] Compare the deployment location data with the previously calculated layout fusion coefficients to ensure that the coefficients are calculated based on the current actual location. If any deviation is found (such as camera position movement), the fusion coefficients need to be recalibrated or updated.

[0136] For multi-camera systems, the algorithm correction package is decomposed into multiple independent processing chains based on the physical camera. Each processing chain contains: The camera's proprietary preprocessing parameters (such as the camera's distortion coefficients and color correction matrix).

[0137] The core detection model (the same model may be shared by all cameras, but the input size or anchor points may vary depending on the field of view).

[0138] The transformation matrix [R|t] from the camera to the global coordinate system (extracted from the layout blending coefficients).

[0139] The recognition algorithm package includes components shared across cameras, such as global fusion logic (stitching, overlapping region processing) and common detection model weights. These modules will be retained in the global layer after decomposition.

[0140] From each decomposed processing chain, all parameters directly related to the deployment location and requiring on-site verification are extracted to form a list of parameters to be verified, as shown in Table 1: Table 1 List of parameters to be verified Geometric calibration parameters Single-camera distortion coefficients (k1,k2,p1,p2) Preprocessing Verify the accuracy of distortion correction Camera intrinsic parameter matrix (fx, fy, cx, cy) Preprocessing Verify that the focal length and principal point are consistent with the calibration values. Extrinsic parameter matrix (R,t) Post-processing Verify the correctness of the camera-to-world coordinate transformation. Photometric calibration parameters Brightness gain coefficient Preprocessing Verify if the illumination compensation is appropriate Color Correction Matrix (CCM) Preprocessing Verify color reproduction accuracy Field of view and scale parameters Actual field of view (FOV) Core Detection The detection range is consistent with the theory. Pixel equivalent (mm / pixel) Core Detection Verify dimensional measurement accuracy Anchor point frame size Core Detection Verify target scale adaptability Fusion parameters Overlapping region weight map Post-processing Verify the smoothness of multi-camera fusion splicing homography matrix Post-processing Verify image stitching accuracy Position of the detection box after coordinate transformation Post-processing Verify the accuracy of physical coordinate positioning Performance parameters Single frame processing time Full process Verification of real-time compliance Resource usage (CPU / GPU / Memory) Full process Verify component capacity Based on their impact on detection accuracy and system stability, each parameter is assigned a priority (high / medium / low). High-priority parameters (such as extrinsic matrix and pixel equivalent) must be rigorously verified, while low-priority parameters (such as minor deviations in the color correction matrix) can be appropriately relaxed.

[0141] Preferably, refer to Figure 2 Step S6 includes: Based on the parameter category and component type, perform on-site verification of the deployment parameters to be verified, and calculate the parameter deviation value; If the parameter deviation value is within the preset deviation parameter range, the current deployment parameter to be verified is determined to have passed the verification and remains unchanged; No, then replace the current deployment parameters to be verified with the actual measured values; Based on the deployed user layout interface, target objects in the work scene are detected in real time, and target detection information is output. The target detection information is analyzed, the evaluation index is calculated, and the results are compared with the performance index in the theoretical vision task model. If any evaluation indicator fails to meet the current performance indicator, the non-compliant indicator is traced back to determine the corresponding attribute parameter and calculate the deviation between the current attribute parameter and the theoretical attribute parameter. Based on the deviation and the component layout control model, visual guidance signals are generated and transmitted and the component frame layout is adjusted in conjunction with the communication protocol.

[0142] In this embodiment, a list of deployment parameters to be verified is loaded from the detection algorithm correction package, including geometric calibration parameters (distortion coefficients, intrinsic parameters, extrinsic parameters), photometric parameters, field-of-view scale parameters, fusion parameters, and performance parameters. Simultaneously, expected values ​​or allowable ranges from the theoretical vision task model are loaded.

[0143] Geometric parameter verification: Using a high-precision calibration board (such as a checkerboard or dot matrix), multiple images are captured at the actual deployment location of the equipment. An online calibration program is run to calculate the current intrinsic parameters, extrinsic parameters, and distortion coefficients. The calculation results are compared with the parameters in the correction package to ensure that the deviation is within the allowable range (e.g., rotation angle error <0.5°, translation error <2mm). If the deviation exceeds the tolerance, a recalibration process is triggered.

[0144] Photometric parameter verification: Photograph a standard color chart, extract the average RGB values ​​of the color patches in the image, and compare them with the standard values ​​to verify the effectiveness of the color correction matrix (CCM) and luminance gain coefficient. If the color difference ΔE exceeds the threshold (e.g., 5), adjust the light source or recalculate the correction coefficients.

[0145] Field of view and scale verification: Place a reference object of known size (such as a standard gauge block) within the field of view, run the detection algorithm, measure the pixel size of the reference object, and calculate the actual pixel equivalent (mm / pixel). Compare with the theoretical value; the error should be less than the process requirement (e.g., ±1%). Simultaneously verify whether the target position output by the detection algorithm matches the actual physical position.

[0146] Fusion parameter verification: For multi-camera systems, feature points are placed in the overlapping area, and their coordinates are detected from each camera. The coordinates are then transformed to the global coordinate system using a coordinate transformation matrix, and the consistency error (e.g., <3mm) is checked. Simultaneously, the smoothness of the stitched image in the overlapping area is verified.

[0147] Performance parameter verification: Use performance monitoring tools to record the actual single-frame processing time, CPU / GPU utilization, and memory usage to ensure that real-time requirements are met (e.g., processing time ≤ 50ms).

[0148] Record the verification results to the deployment file. If the parameters pass the verification, the current correction package is confirmed to be valid; if there is a slight deviation but it is still within the allowable range, update the actual measured value to the correction package configuration; if there is a serious deviation, the components need to be recalibrated or adjusted and then verified again.

[0149] The vision sensor (camera) is activated to acquire images of the current work scene in real time. A standardized input image is obtained by applying validated preprocessing modules (distortion correction, color correction, and size normalization).

[0150] Target detection and localization: Run the core detection algorithm (with corrected anchor points, input dimensions, etc.) to identify target workpieces or feature points in the image, and output their position (bounding box, key point coordinates) and category confidence in the image pixel coordinate system.

[0151] Using the verified camera intrinsic parameters and distortion coefficients, the image pixel coordinates are converted into normalized coordinates in the camera coordinate system.

[0152] Using the verified camera extrinsic parameters (rotation matrix R and translation vector t), transform the camera coordinates to the global world coordinate system (or the robot's base coordinate system). For multi-camera systems, unify the target positions detected by each camera to the same global coordinate system, then fuse them (e.g., using a weighted average) to obtain the final target physical coordinates. The detected target information is compared with the requirements in the theoretical vision task model. The theoretical model may include: the target's standard pose (e.g., gripping point, assembly position); the allowable deviation range (tolerance); and motion path constraints (obstacle avoidance area, speed limit). For example, if the task requires gripping a workpiece to a specified position, the theoretical model provides the target pose where the workpiece should be placed.

[0153] Reference Figure 3 A smart camera vision guidance system that can be quickly debugged and deployed, applied to a smart camera vision guidance method, including: The task construction module is used to acquire and parse visual detection tasks, extract theoretical operating areas and target detection requirements, and build theoretical visual task models. The deviation calculation module is used to parse the user layout interface, obtain the actual operation area, and calculate the parameter correction coefficient and layout fusion coefficient in combination with the theoretical operation area. The algorithm aggregation module is used to select and aggregate the corresponding visual detection algorithms according to the target detection requirements, and generate a visual detection algorithm package by combining the basic parameters of the algorithms. The parameter correction module is used to correct the visual detection algorithm package based on the target detection process and parameter correction coefficients to obtain the detection algorithm correction package. The layout extraction module is used to extract the deployment parameters to be verified based on the layout fusion coefficient and the component layout position decomposition and detection algorithm correction package. The guidance control module is used to verify the deployment parameters to be verified and, in conjunction with the theoretical visual task model, convert them into visual guidance signals to adjust the layout of the component boxes.

[0154] Each module of this system is deployed inside the smart camera, and the smart camera can be applied to various production lines to meet the product testing needs of different customers.

[0155] The implementation principle of this embodiment is as follows: In the visual inspection scenario of adhesive coating quality on a certain automotive body welding production line, the vision system first acquires and parses the door adhesive coating inspection task issued by the workshop MES system, extracts component information such as the theoretical width of the adhesive strip (3mm) and height (2mm) from the process standard document, as well as defect detection requirements such as adhesive breakage, bubbles, and trajectory deviation, and constructs a theoretical visual task model that includes the detection coordinate system and tolerance limits. Subsequently, the system connects to and analyzes the 3D line laser contour sensor and industrial camera deployed at the robot's end effector via the Profinet interface. It acquires the actual sensor's laser wavelength, scanning frequency, and lens distortion parameters, and compares these with theoretical information to calculate a gain correction coefficient of 0.92 to compensate for illumination changes and a coordinate fusion coefficient reflecting the robot's installation angle. Based on the requirements for adhesive strip defect detection, the system selects and aggregates line laser algorithms suitable for micron-level contour analysis and deep learning-based image segmentation algorithms from the algorithm library, combining them with the basic scanning resolution to generate a visual inspection algorithm package. According to the width and viscosity characteristics of the adhesive strip in different door areas, the algorithm is optimized region-by-region based on the gain correction coefficient, resulting in a detection algorithm correction package. Based on the coordinate fusion coefficient and the robot's actual deployment position on the production line, the correction package is decomposed into deployment parameters to be verified for each scanning trajectory. Finally, the detection accuracy of the deployment parameters is verified by using standard adhesive strip test pieces. Once confirmed, the results are converted into visual guidance signals to control the robot to move along the contour of the car door and trigger the camera to collect and analyze data in real time, thus achieving online full inspection of the adhesive coating quality.

[0156] An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When one or more programs are executed by one or more processors, the one or more processors implement any of the methods in the above scheme.

[0157] A storage medium storing at least one instruction, at least one program, code set, or instruction set, wherein the at least one instruction, at least one program, code set, or instruction set is loaded and executed by a processor to implement the intelligent camera vision guidance method as described above.

[0158] The embodiments described in this specific implementation are preferred embodiments of this application and are not intended to limit the scope of protection of this application. Therefore, all equivalent changes made in accordance with the structure, shape and principle of this application should be covered within the scope of protection of this application.

Claims

1. A method for quickly debugging and deploying intelligent camera vision guidance, characterized in that, include: Acquire and analyze the visual detection task, extract the theoretical operating area and target detection requirements, and construct a theoretical visual task model; Connect and parse the user layout interface to obtain the actual operation area, and calculate the parameter correction coefficient and layout fusion coefficient in combination with the theoretical operation area; Based on the target detection requirements, select and aggregate the corresponding visual detection algorithms, and combine the basic parameters of the algorithms to generate a visual detection algorithm package; Based on the target detection process and the parameter correction coefficients, the visual detection algorithm package is corrected to obtain the detection algorithm correction package; Based on the layout fusion coefficient and the component layout position, the detection algorithm correction package is decomposed to extract the deployment parameters to be verified. The deployment parameters to be verified are validated and, in conjunction with the theoretical visual task model, are converted into visual guidance signals to adjust the component frame layout.

2. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The process of acquiring and parsing the visual detection task, extracting the theoretical operating region and target detection requirements, and constructing a theoretical visual task model includes: The relevant documents for visual inspection tasks are analyzed to obtain the visual task types and task implementation environments. Based on the visual task type, historical operation behaviors are analyzed to extract the theoretical operation area; Based on the task implementation environment and the theoretical operating area, the visual detection requirements are summarized to obtain the target task requirements; Based on the target task requirements and the visual task type, the visual theoretical computation framework is associated and fused to construct a theoretical visual task model.

3. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The process of connecting and parsing the user layout interface to obtain the actual operation area, and combining it with the theoretical operation area, calculates parameter correction coefficients and layout blending coefficients, including: The user layout interface is connected and self-checked according to the communication protocol to form a project interface layout cluster. The actual operation area is obtained by parsing the project interface layout cluster using the parameter configuration tool; Based on the component type, the actual operating area and the theoretical operating area are mapped and compared to construct an attribute comparison mapping table; Based on the attribute comparison mapping table, the attribute parameters are corrected and the parameter correction coefficients are calculated. The guide interface is measured, and the guide component layout is obtained by combining the component box layout. The layout of the guiding components is compared and verified based on the theoretical layout of the components in the theoretical operating area, and the layout fusion coefficient is calculated.

4. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The step of selecting and aggregating appropriate visual detection algorithms according to the target detection requirements, and generating a visual detection algorithm package by combining the basic parameters of the algorithms, includes: The target detection requirements are analyzed and refined, and task subcategories, performance indicators, and display formats are extracted. The algorithm framework is matched according to the task subclass, and the target algorithm model is determined by combining the actual operation area. Based on the performance indicators, the visual processing algorithms are integrated and merged to generate an algorithm aggregation list; The algorithm aggregation list is arranged in time sequence according to the detection process flow, and combined with the basic algorithm parameters corresponding to each algorithm, an algorithm implementation time sequence list is obtained. The output algorithm of the target recognition result is bound according to the display format to determine the inference and labeling algorithm; The algorithm implementation timeline, the inference labeling algorithm, and the target algorithm model are encapsulated according to the process identifier to obtain a visual detection algorithm package.

5. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The step of modifying the visual detection algorithm package based on the target detection process and the parameter correction coefficients to obtain a modified detection algorithm package includes: The target detection process is broken down into several process implementation segments based on the process implementation timestamp; The visual inspection algorithm package is mapped and aligned according to the process implementation segment to determine the process parameters to be corrected. The parameter correction coefficient is matched with the process parameter to be corrected based on the parameter attributes to obtain the parameter correction pair; The parameter correction pair is detected and verified according to the attribute boundary constraint interval; if the parameter correction pair is not located in the attribute boundary constraint interval, the process parameter to be corrected in the current parameter correction pair is replaced with the upper or lower limit value of the attribute boundary constraint interval. No, then keep the current parameter correction unchanged; Based on the parameter corrections, the visual detection algorithm package is integrated and encapsulated to generate a detection algorithm correction package.

6. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The step of decomposing the detection algorithm correction package based on the layout fusion coefficient and component layout position, and extracting the deployment parameters to be verified, includes: The detection algorithm correction package is deconstructed according to the data processing flow to determine the corresponding data operation module; The data processing modules are associated and decomposed according to the component layout position to determine the module processing chain; The module processing chain is logically verified based on the layout fusion coefficient, and deployment-related parameters are extracted. The deployment-related parameters are sorted according to performance metrics, and their priorities are marked. Based on the parameter priority and theoretical benchmark threshold, the deployment-related parameters are filtered to obtain the deployment parameters to be verified.

7. The intelligent camera vision guidance method for rapid debugging and deployment according to claim 1, characterized in that, The process of verifying the deployment parameters to be verified, and combining them with the theoretical visual task model, transforming them into visual guidance signals to adjust the component box layout, includes: Based on the parameter category and component type, perform on-site verification of the deployment parameters to be verified, and calculate the parameter deviation value; If the parameter deviation value is within the preset deviation parameter range, then the current deployment parameter to be verified is determined to have passed the verification and remains unchanged; No, then replace the current deployment parameters to be verified with the actual measured values; Based on the deployed user layout interface, target objects in the work scene are detected in real time, and target detection information is output. The target detection information is analyzed, evaluation indicators are calculated, and compared with the performance indicators in the theoretical vision task model; If any of the evaluation indicators fails to meet the current performance indicator, the non-compliant indicator is traced back to determine the corresponding attribute parameter and calculate the deviation between the current attribute parameter and the theoretical attribute parameter. Based on the deviation and the component layout control model, a visual guidance signal is generated, and the component frame layout is adjusted by transmitting it in conjunction with the communication protocol.

8. A rapidly deployable intelligent camera vision guidance system, used to implement the method as described in any one of claims 1-7, characterized in that, include: The task construction module is used to acquire and parse visual detection tasks, extract theoretical operating areas and target detection requirements, and build theoretical visual task models. The deviation calculation module is used to parse the user layout interface, obtain the actual operation area, and calculate the parameter correction coefficient and layout fusion coefficient in combination with the theoretical operation area. The algorithm aggregation module is used to select and aggregate corresponding visual detection algorithms according to the target detection requirements, and generate a visual detection algorithm package by combining the basic parameters of the algorithms. The parameter correction module is used to correct the visual detection algorithm package based on the target detection process and the parameter correction coefficients to obtain the detection algorithm correction package. The layout extraction module is used to decompose the detection algorithm correction package based on the layout fusion coefficient and the component layout position, and extract the deployment parameters to be verified. The guidance control module is used to verify the deployment parameters to be verified and, in conjunction with the theoretical visual task model, convert them into visual guidance signals to adjust the component box layout.

9. An electronic device, comprising: One or more processors; Memory, used to store one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors implement the intelligent camera vision guidance method as described in any one of claims 1 to 7.

10. A storage medium storing at least one instruction, at least one program, a code set, or an instruction set, wherein the at least one instruction, the at least one program, the code set, or the instruction set is loaded and executed by a processor to implement the intelligent camera vision guidance method as claimed in any one of claims 1 to 7.

Images