An artificial intelligence-based industrial transfer robot vision intelligent navigation method

Abstract

The application discloses an industrial carrying robot vision intelligent navigation method based on artificial intelligence, relates to the technical field of industrial robot vision navigation and positioning, and remarkably improves the navigation reliability of the industrial carrying robot under an extreme light scene through an adaptive processing framework driven by artificial intelligence; a core light self-adaption feature extraction model fuses brightness saturation estimation and a multi-head attention mechanism, so that the feature extraction process actively avoids the interference of overexposed or underexposed areas, in the positioning optimization layer, a double mechanism combining sparse matching and a sliding window graph optimization is used, a robust kernel function of re-projection error is used to suppress outliers formed by interference such as metal reflection, and the stability of pose solving is enhanced; when the system detects that the number of continuous frame matching points decreases or the error surges, speed reduction and local path re-planning are automatically triggered, a fault response closed loop is formed, and collision or stall caused by instantaneous positioning failure is avoided.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of industrial robot vision navigation and positioning, and particularly relates to an intelligent vision navigation method for industrial carrying robot based on artificial intelligence. BACKGROUND

[0002] In an industrial scene such as an automobile welding workshop, a carrying robot needs to frequently pass through a strong arc light area such as a welding station and a dark area such as a device shadow; such an environment has a dramatic mutation of instantaneous illumination, >100,000 lux to <50 lux, which constitutes a core challenge to a vision navigation system; the robot needs to capture a ground navigation mark or an environmental feature in real time to maintain a centimeter-level positioning accuracy.

[0003] Most of the prior art solutions adopt a 3D vision navigation technology based on a convolutional neural network (CNN), and realize positioning by processing camera images in real time; to cope with illumination changes, some solutions introduce multi-exposure image fusion or local adaptive histogram equalization (CLAHE); however, in the case of extreme illumination mutation, a fixed exposure strategy causes feature loss in an overexposed area or a sharp increase in noise in an underexposed area, and key navigation marks cannot be stably extracted; a traditional algorithm for dynamically adjusting exposure cannot meet the real-time demand of navigation due to a delay in calculation, and cannot update at a rate of >10 Hz, which causes positioning drift.

[0004] Some solutions combine infrared sensors for assistance or a pre-defined illumination compensation parameter library; however, infrared data is easily disturbed in a metal reflection scene, and the parameter library cannot cover random illumination fluctuations such as arc flickering; some solutions also adopt an HDR imaging technology, but the multi-frame synthesis process causes motion blur, and the solutions fail when the robot moves at a high speed; although these methods alleviate the limitations of single exposure, they still cannot fundamentally solve the problem of real-time feature robustness in the case of extreme mutation, which causes the path planning module to frequently trigger deviation correction and reduces carrying efficiency. SUMMARY

[0005] In view of the above-mentioned existing problems, the present application is proposed.

[0006] The present application provides an intelligent vision navigation method for industrial carrying robot based on artificial intelligence, which solves the problem of positioning drift caused by feature loss and insufficient dynamic environmental robustness of existing vision navigation in the case of strong light / shadow mutation in a welding workshop.

[0007] To solve the above-mentioned technical problems, the present application provides the following technical solutions.

[0008] In a first aspect, an intelligent vision navigation method for industrial carrying robot based on artificial intelligence is provided, which includes,

[0009] acquiring an input image of a calibrated monocular camera;

[0010] input the image into a light adaptation feature extraction model to obtain a light robust feature map, the model comprising a feature encoding layer, a brightness saturation estimation subnetwork, and a multi-head attention fusion layer, the brightness saturation estimation subnetwork outputting a pixel-level light weight map and weighting and fusing the multi-head attention on the encoded features;

[0011] perform two-dimensional-three-dimensional sparse matching on the encoded features and three-dimensional key points in an environment semantic map that is pre-constructed and has a metric scale, obtain an initial six-degree-of-freedom pose of the robot based on perspective-n-point pose solving according to a consistency determination, and obtain an optimized pose in a sliding window graph optimization;

[0012] generate an initial path based on the optimized pose and a target point by heuristic search, and dynamically re-plan according to an obstacle mask; when the number of matching inlier points in consecutive T frames is less than a threshold M or the average re-projection error is greater than a threshold ε, reduce the speed of the robot and trigger local re-planning, wherein T, M and ε are pre-set.

[0013] As a preferred scheme of the industrial carrying robot vision intelligent navigation method based on artificial intelligence, the light adaptation feature extraction model is a lightweight Transformer network, which comprises image block embedding, local window multi-head self-attention, and a feedforward layer, and outputs the light robust feature map.

[0014] As a preferred scheme of the industrial carrying robot vision intelligent navigation method based on artificial intelligence, the Transformer network is obtained by training as follows:

[0015] Based on a real scene image library, a synthetic sample of different light intensities and spectra is constructed by using light rendering and arc light spectrum condition generation adversarial network, and sensor noise is superimposed on underexposed samples;

[0016] Different light samples of the same scene are used as positive sample pairs, different scene samples are used as negative sample pairs, a self-supervised contrastive loss is trained, and a saturation suppression regular is introduced to limit the attention weight in the saturation region.

[0017] As a preferred scheme of the industrial carrying robot vision intelligent navigation method based on artificial intelligence, the total loss when training the Transformer network is composed of the weighted sum of the contrastive loss term and the saturation suppression regular term, wherein the saturation suppression regular term restricts the attention weight in the overexposed region according to the soft mask obtained by the brightness-saturation estimation subnetwork.

[0018] As a preferred scheme of the industrial transport robot vision intelligent navigation method based on artificial intelligence, the environment semantic map comprises three-dimensional key points with a metric scale and appearance descriptors, and the two-dimensional-three-dimensional sparse matching adopts consistency determination to eliminate outliers.

[0019] As a preferred scheme of the industrial transport robot vision intelligent navigation method based on artificial intelligence, the cost function of the sliding window graph optimization comprises re-projection error and loop constraint, and a robust kernel function is adopted to suppress abnormal observations caused by highlights and shadows.

[0020] As a preferred scheme of the industrial transport robot vision intelligent navigation method based on artificial intelligence, obstacle detection adopts a reflection suppression segmentation network, the network comprises a main branch for extracting multi-scale features by using a hollow convolution and an auxiliary branch for estimating light flow consistency, a reflection artifact confidence map is obtained by fusing the outputs, and an obstacle mask is generated to update a path cost map.

[0021] As a preferred scheme of the industrial transport robot vision intelligent navigation method based on artificial intelligence, the method further comprises an online adaptive step.

[0022] When the pose confidence is lower than a threshold, only the attention scaling parameter of the illumination adaptive feature extraction model is updated in a small step, and the update amplitude is limited by a preset upper limit and an exponential moving average is adopted.

[0023] As a preferred scheme of the industrial transport robot vision intelligent navigation method based on artificial intelligence, the dynamic re-planning adopts a time elastic band algorithm to smooth the initial path and meet the speed and acceleration constraints.

[0024] In a second aspect, the present application provides a visual navigation device, comprising a monocular camera, a memory and a processor, the memory stores a computer program, and the processor implements the steps of the above-mentioned industrial transport robot vision intelligent navigation method based on artificial intelligence when executing the computer program.

[0025] The present application has the advantages that: the present application significantly improves the navigation reliability of the industrial transport robot in an extreme illumination scene through an artificial intelligence driven adaptive processing framework. The core illumination adaptive feature extraction model fuses brightness saturation estimation and a multi-head attention mechanism, so that the feature extraction process actively avoids the interference of overexposed or underexposed areas, and ensures the stable retention of key environmental features (such as navigation signs and equipment profiles) in a welding workshop where strong arc light and deep shadow appear alternately. This design replaces the traditional fixed exposure strategy, and reduces the risk of positioning drift caused by illumination sudden change from the source.

[0026] In the positioning optimization layer, the dual mechanism of sparse matching and sliding window graph optimization is combined, the robust kernel function of re-projection error is used to suppress outliers caused by metal reflection and other interference, and the stability of pose solving is enhanced. When the system detects that the number of matching points in consecutive frames decreases or the error increases, it automatically triggers speed reduction and local path re-planning, forming a fault response closed loop to avoid collision or stall caused by instantaneous positioning failure.

[0027] In the training stage, the synthetic data generation technology under the condition of light spectrum is introduced, and the model is forced to ignore the light difference and focus on the geometric structure consistency through self-supervised contrast learning, reducing the dependence on high-cost real labeled data. The addition of saturation suppression regularization further constrains the attention weight of the model to the distorted area, improving the physical rationality of feature expression.

[0028] The reflection suppression segmentation network cooperates with the optical flow consistency detection and multi-scale feature extraction to effectively distinguish the metal shelf reflection artifact and the real obstacle, and reduce the false detection rate. The online adaptive mechanism only fine-tunes the attention scaling parameter to restore the stability of the feature with minimum computing power overhead, and adapts to the continuous fluctuations of the factory light. The overall scheme upgrades the existing visual hardware with pure algorithm, and provides a low-cost intelligent navigation solution for high-dynamic industrial environments. BRIEF DESCRIPTION OF DRAWINGS

[0029] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the following will briefly introduce the drawings needed to be used in the embodiment description. Obviously, the drawings in the following description are only some embodiments of the present application, and other drawings can be obtained by those skilled in the art without creating laborious work.

[0030] Figure 1 The flowchart of the artificial intelligence-based industrial transport robot visual intelligent navigation method in embodiment 1. DETAILED DESCRIPTION

[0031] In order to make the above-mentioned purposes, features and advantages of the present application more apparent and easy to understand, the specific embodiments of the present application will be described in detail below with reference to the drawings of the specification.

[0032] In the following description, many specific details are set forth in order to provide a thorough understanding of the present application, but the present application can also be implemented in other ways different from those described herein, and those skilled in the art can make similar generalizations without departing from the connotation of the present application, therefore the present application is not limited to the specific embodiments disclosed below.

[0033] Second, the term "one embodiment" or "an embodiment" as may be used herein means a specific implementation, or example, that can include features that are, optionally, not included in other implementations or examples. Thus, "one embodiment" or "an embodiment" as used herein is meant to be one example out of many implementations or examples that can have a feature in common, or a common desire to be implemented.

[0034] For the sake of implementation and to avoid ambiguity, some terms are explained in this embodiment:

[0035] Monocular camera: refers to an industrial camera containing only one imaging channel, which completes intrinsic and extrinsic calibration. The default intrinsic is represented by a pinhole model plus radial and tangential distortion parameters; the extrinsic is represented by the pose of the camera coordinate system relative to the robot body coordinate system. The default resolution is 640x480 or 1280x720, the frame rate is 30-60 frames per second, and the shutter type is preferably global shutter. The default exposure time is 0.2-5 milliseconds adjustable, and the gain is 0-12 decibels adjustable.

[0036] Calibration: refers to obtaining camera intrinsic, distortion parameters and rigid body transformation of camera relative to robot coordinate system through calibration board or equivalent method. The default uses a checkerboard or dot array calibration board, and at least 20 groups of different pose samples are collected; the extrinsic is obtained through hand-eye calibration or tooling calibration, and the default error is not more than 2 millimeters and 0.2 degrees.

[0037] Light adaptive feature extraction model: refers to a neural network that inputs an image and outputs a light robust feature map, including a feature encoding layer, a brightness-saturation estimation subnetwork, and a multi-head attention fusion layer. The default input is a de-distorted grayscale or three-channel image, the input size is 640x480, the network parameter amount is not more than 10 million, and the single-frame inference delay is not higher than 30 milliseconds (based on embedded GPU).

[0038] Feature encoding layer: refers to a network layer that obtains local feature representation after image blocking or equivalent processing. The default channel dimension is 64-256, which can be adjusted by depth or width multiplier.

[0039] Brightness-saturation estimation subnetwork: refers to a subnetwork that estimates the brightness and saturation risk of each pixel of the input image and outputs a soft mask. The default output range is 0-1, where close to 1 indicates that the pixel is greatly affected by overexposure or strong reflection. The default considers more than 95% gray scale occupancy as high saturation risk, which can be adjusted in the 90-99% range.

[0040] Multi-head attention fusion layer: refers to a module that weights and fuses the encoded features according to the soft mask output by the subnetwork and the global context. The default number of heads is 4-8, and the fused feature map has the same spatial size as the encoding layer.

[0041] Light robust feature map: refers to the feature tensor after fusion, which is used for subsequent matching and pose estimation. The default spatial downsampling factor is 4, and the channel number is 64-128, represented by a floating point number.

[0042] Environment semantic map: refers to a pre-constructed three-dimensional map with metric scale, containing three-dimensional coordinates and appearance description information of key points, and defining a world coordinate system. The default is generated by offline collection of similar visual data or fusion of visual and laser data, with a key point density of not less than 200 per square meter.

[0043] Two-dimensional-three-dimensional sparse matching: refers to the matching of the current frame features with the map key points using nearest neighbor or equivalent strategy, and the elimination of outliers using consistency judgment. The default is to use ratio test or cross consistency test, with a false match rate controlled within 10%.

[0044] Perspective n-point pose solving: refers to calculating the six-degree-of-freedom camera pose based on two-dimensional-three-dimensional correspondence and cooperating with consistency judgment. The output translation unit is meters, and the rotation unit is degrees.

[0045] Sliding window graph optimization: refers to joint multi-frame re-projection constraint, loop constraint and prior constraint for pose optimization within a fixed length time window. The default window length is 10 frames, which can be adjusted between 5-20 frames; the optimization frequency is default 20 times per second.

[0046] Loop constraint: refers to the relative pose constraint added when the historical key frame and the current key frame reach the similarity threshold. The default similarity threshold is 0.7, which can be adjusted between 0.6-0.85.

[0047] Prior constraint: refers to the smooth or constant speed prior added between adjacent key frames to improve observability, which acts on the translation and rotation components by default.

[0048] Heuristic search: refers to a search method that generates a collision-free path using a heuristic cost function, with a default implementation of the classic A*. The default grid resolution is 5 centimeters, and the heuristic is the Euclidean or diagonal distance.

[0049] Obstacle mask: refers to the binary or probability map obtained by thresholding and morphological processing of the reflection artifact confidence map output by the reflection suppression segmentation network, which is used to update the path cost map. The default threshold is 0.5, which can be adjusted between 0.3-0.7; the default inflation radius is 10-20 centimeters.

[0050] Reflection suppression segmentation network: refers to a dual-branch network containing a main branch and an auxiliary branch. The main branch extracts multi-scale environmental features, and the auxiliary branch detects the motion continuity of high light areas based on optical flow consistency. The fusion output is a reflection artifact confidence map. The default inference frequency is consistent with the feature extraction model.

[0051] Pose confidence: refers to the confidence index obtained by comprehensively matching the number of inner points and the re-projection error. By default, it is calculated according to consecutive frames. When the number of inner points in consecutive T frames is less than the threshold M or the average re-projection error exceeds the threshold, it is considered as low confidence. The default T is 3-5 frames, the default M is 30-80 inner points, and the default error threshold is 1.0-2.0 pixels.

[0052] Time elastic band algorithm: refers to a joint smoothing process in time domain and spatial domain for discrete paths, which meets the speed and acceleration constraints and outputs executable trajectories. The default trajectory update time is 10 times per second, and the time prediction window is 2-5 seconds.

[0053] The terms "first", "second", etc. are only used to distinguish different objects and do not represent the order or superiority-inferiority relationship. "And / or" means that any of them can be satisfied alone or simultaneously. The order of steps is not limited: unless there is a direct input-output dependency between steps, the execution order of modules can be parallel or rearranged according to implementation.

[0054] Embodiment 1, refer to Figure 1 As the first embodiment of the present application, the embodiment provides an artificial intelligence-based industrial transport robot visual intelligent navigation method, comprising the following steps:

[0055] Obtaining the input image of the calibrated monocular camera;

[0056] Inputting the image into a light-adaptive feature extraction model to obtain a light-robust feature map, the model including a feature encoding layer, a brightness saturation estimation subnetwork, and a multi-head attention fusion layer, the brightness saturation estimation subnetwork outputting a pixel-level light weight map and weighting and fusing the encoded features with the multi-head attention to reduce the influence of overexposed and underexposed areas on the features;

[0057] The light-adaptive feature extraction model is a lightweight Transformer network, including image block embedding, local window multi-head self-attention, and feedforward layer, and outputs a light-robust feature map;

[0058] The Transformer network is obtained by training as follows:

[0059] Based on the real scene image library, the synthetic samples of different light intensities and spectra are constructed by using light rendering and arc light spectrum condition generation adversarial network, and the sensor noise is superimposed on the underexposed samples;

[0060] Taking different light samples of the same scene as positive sample pairs and different scene samples as negative sample pairs, self-supervised contrastive loss is used for training, and saturation suppression regularization is introduced to limit the attention weight in the saturation area;

[0061] The training process of the Transformer network includes:

[0062] Real industrial scene images are collected, and the generated adversarial process of light rendering and arc light spectrum conditions is used to augment multi-intensity and multi-spectrum samples. Sensor noise is added to underexposed samples. Different lightings of the same scene are used as positive sample pairs, and different scenes are used as negative sample pairs to form a batch.

[0063] The input image is input into a lightweight Transformer, image block embedding, local window multi-head self-attention and feedforward layer to obtain feature representation and attention map for subsequent loss term calculation.

[0064] The contrast loss is used as the main body, and saturation suppression regularization is added to form the total loss:

[0065] L total =L con +α sat L sat ,

[0066] Wherein, L total represents the total training loss, L con represents the self-supervised contrast loss, which is used to close the same scene under different lightings and separate different scenes, α sat represents the saturation suppression regularization weight coefficient, which can be adjusted within [0.1, 1.0] and is dimensionless, L sat represents the penalty term for attention to highlight saturated areas, which is used to suppress the dominance of overexposure and reflection on features.

[0067] In the formula:

[0068]

[0069] Wherein, B represents the batch size, i.e. the number of sample pairs, which is dimensionless, subscript i represents the index of the i-th anchor sample, z i represents the L2 normalized feature vector of the anchor sample, superscript + represents the positive sample pair, represents the positive sample feature of the same scene as the anchor under different lightings, τ represents the contrast temperature coefficient, which is dimensionless, represents the index set of the i-th anchor corresponding to the negative sample, superscript - represents the negative sample pair, represents the negative sample feature, j represents the negative sample index, and cos(·,·) represents the cosine similarity of normalized vectors.

[0070] In the formula:

[0071]

[0072] Wherein, P represents the total number of pixels or feature positions participating in the regularization, subscript p represents the p-th position index, m p represents the saturation-sensitive soft mask obtained by the brightness-saturation estimation subnetwork, which is dimensionless and takes values in [0, 1].p attention weight representing the position, valued [0, 1], a max attention upper bound threshold, dimensionless, max(·) denotes the positive operator, ReLU form;

[0073] The total loss is iteratively minimized with the AdamW adaptive method, and the base update is written as:

[0074]

[0075] where θ represents the trainable parameter vector, and η represents the learning rate, denotes the gradient operator of θ; cosine annealing scheduling is adopted in training, and α is selected with the validation set sat , τ and a max are valued;

[0076] The sliding window evaluates the two types of indicators of the same scene consistency improvement and saturation sensitivity reduction. If the indicators do not improve within a certain number of rounds, early stopping and backtracking to the best weight are performed.

[0077] θ is frozen, and the illumination-robust feature map is output for subsequent two-dimensional-three-dimensional matching and pose estimation;

[0078] Specifically, the training scheme mainly uses contrastive learning, which promotes the stability and discrimination of features under intensity and spectral changes through the pairing of the same scene under different illuminations. After incorporating saturation suppression into the unified target, the network no longer focuses on the distorted area when there are extreme illuminations such as overexposure and reflection, but tends to focus on areas with structural texture and geometric information, thereby improving the reliability of subsequent matching and pose estimation.

[0079] On the data level, the range of illumination distribution is expanded with conditional generation, and noise is injected into underexposed samples to improve adaptability under low signal-to-noise ratio. In the optimization phase, adaptive methods and learning rate annealing are used to make parameters converge on a relatively smooth trajectory, and sliding window index monitoring is used to prevent overfitting.

[0080] Two-dimensional-three-dimensional sparse matching is performed between the features and the three-dimensional key points in the pre-constructed environment semantic map with metric scale, and the initial six-degree-of-freedom pose of the robot is obtained by RANSAC-PnP solution, and the optimized pose is obtained in the sliding window graph optimization.

[0081] Graph optimization is sliding window pose graph optimization, the cost function includes reprojection error and loop constraint, and robust kernel function is used to suppress outliers caused by highlights and shadows.

[0082] The robust target construction of sliding window pose graph optimization includes:

[0083] A set of keyframes are selected within the sliding window, and the inlier observations from 2D-3D matching are collected, along with the corresponding loop constraint pairs and prior information (e.g. pose smoothness prior);

[0084] For each 2D-3D correspondence, the re-projection error vector in pixel plane is computed according to the current window poses, which is normalized by the measurement covariance formed by the observation noise and illumination weight;

[0085] The robustified optimization objective is defined as:

[0086]

[0087]

[0088] where X denotes the set of parameters to be estimated within the window, including the 6-DOF poses of all keyframes, denotes the set of keyframe indices within the sliding window, and subscript k denotes the k-th keyframe index, denotes the set of valid observation indices of keyframe k, and subscript i c denotes the i c -th 2D-3D correspondence, and ρ(·) denotes the robust kernel function that compresses the influence of large residuals, denotes the i c -th 2D-3D re-projection error vector of keyframe k, in pixel unit, denotes the squared Mahalanobis norm with as the measurement covariance, denotes the measurement covariance matrix of the correspondence, in pixel 2 , which fuses the observation noise and illumination weight information, and λ lc denotes the loop term weight coefficient, dimensionless, denotes the set of loop constraint pairs, (φ, ψ) denotes a pair of keyframe indices that form a loop, and e φψ (X) denotes the loop relative pose error vector, in Lie algebra coordinates, and Λ φψ denotes the information matrix of the loop constraint, and λ pri denotes the prior term weight coefficient, dimensionless, and e pri (X) denotes the prior error vector (e.g. generated by pose smoothness / constant velocity model), and Λ pri denotes the information matrix of the prior;

[0089] In the above equation, the robust kernel takes the Cauchy form to suppress outliers caused by high lights and shadows:

[0090] ρ(s) = β 2 ln(1 + s / β 2 ),

[0091] where s denotes the scalar input of squared Mahalanobis norm, dimensionless, β denotes the robust scale parameter, dimensionless, used to adjust the strength of outlier rejection; the prior error e pri (X) Constructed according to the motion prior, such as using constant velocity smoothing constraints between adjacent key frames, and with Λ pri Set the confidence of each degree of freedom;

[0092] An iterative method based on Gauss-Newton or LM is used for parallel solution, and the robust kernel enters the normal equation through iterative reweighting strategy. After each round of update, the observations with large standardized residuals are removed, the window size is maintained, and the loop edges and prior edges are refreshed;

[0093] Specifically, the optimization objective carries the geometric constraints between the image and the map through the re-projection term, and the observation noise and light sensitivity are included in the same dimension through Mahalanobis weighting, so that the observations affected by high contrast have a lower proportion in the cost; after introducing the loop term, the pose graph obtains global consistency in the long range, can maintain topological closure in repetitive areas and occluded environments, and reduces the cumulative drift;

[0094] The prior term is used to maintain the stability of the solution when the local observability is insufficient, especially in the sparse texture or temporary mismatch stage, the smoothing prior can suppress the sudden solution; the robust kernel suppresses the outliers throughout the entire solving process, and its scale parameter determines the compression strength of extreme residuals, combined with the reweighting strategy, the influence of abnormal observations can be adaptively reduced; The overall process organizes observations in windows, unifies various constraints in a robust objective, and forms a closed loop between iterative updates and observation selection, providing a stable six-degree-of-freedom pose sequence for subsequent path planning;

[0095] Based on the optimized pose and target point, an initial path is generated by A* algorithm, and dynamic replanning is performed according to the obstacle mask; When the number of matching inliers in consecutive T frames is less than the threshold M or the average re-projection error is greater than the threshold ε, the robot speed is reduced and local replanning is triggered, where T, M and ε are pre-set;

[0096] The obstacle detection uses a reflection suppression segmentation network, which includes a main branch that extracts multi-scale features using a hollow convolution and an auxiliary branch that estimates light flow consistency, and the reflection artifact confidence map is obtained by fusing the outputs of the two branches, and the obstacle mask is generated to update the path cost map;

[0097] It also includes an online adaptive step:

[0098] When the pose confidence is lower than the threshold, only the attention scaling parameter of the light adaptive feature extraction model is updated in small steps, and the update amplitude is limited by the pre-set upper limit and uses exponential moving average to restore the feature stability;

[0099] Dynamic replanning uses a time elastic band algorithm to smooth the A* path while satisfying velocity and acceleration constraints;

[0100] This embodiment also provides a visual navigation device applicable to industrial scenarios such as automotive welding workshops, including: a monocular camera, a memory, and a processor; the memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions to realize a visual intelligent navigation method for industrial handling robots based on artificial intelligence as proposed in the above embodiment.

[0101] The computer device can be a terminal, comprising a processor, memory, communication interface, display screen, and input devices connected via a system bus. The processor provides computing and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system and computer programs. The internal memory provides an environment for the operation of the operating system and computer programs stored in the non-volatile storage media. The communication interface is used for wired or wireless communication with external terminals; wireless communication can be achieved through Wi-Fi, carrier networks, NFC (Near Field Communication), or other technologies. The display screen can be an LCD screen or an e-ink screen. The input devices can be a touch layer covering the display screen, buttons, a trackball, or a touchpad on the computer device's casing, or an external keyboard, touchpad, or mouse.

[0102] Without changing Figure 1 Under the premise of the overall structure shown, this embodiment is executed in the following order:

[0103] I. Collection

[0104] Input: A calibrated monocular image stream and timestamps.

[0105] Processing: Perform distortion correction, grayscale conversion, or color normalization; perform time synchronization, aligning with the local system clock by default, with an allowable deviation of no more than 5 milliseconds; truncate the brightness of the input image to prevent extreme values ​​from affecting it.

[0106] Output: Normalized image frames, with resolution and frame rate set to default configurations according to the terminology supplement.

[0107] II. Processing and Modeling

[0108] Input: Normalized image frame.

[0109] Processing: First, a soft mask is generated by the brightness-saturation estimation sub-network; the image is processed through a feature encoding layer to obtain local features; the soft mask and the global context are weighted and fused in a multi-head attention fusion layer to obtain an illumination robust feature map; the feature points to be matched and descriptive information are detected or read on the feature map.

[0110] Output: Illumination-robust feature map, set of feature points with corresponding description information, intensity-saturation soft mask.

[0111] III. Matching and Pose Estimation

[0112] Input: Current frame features and map keypoint library.

[0113] Process: Perform nearest neighbor search and adopt ratio test and cross-consistency to reject unreliable matches; estimate initial pose based on perspective-n-point method with consistency check; jointly optimize the graph with multi-frame constraints in a sliding window, and add loop closure constraints if detected.

[0114] Output: Optimized camera 6-DOF pose, number of inlier matches, reprojection error statistics.

[0115] IV. Decision and Control

[0116] Input: Pose, inlier and error statistics, obstacle mask.

[0117] Process: Calculate pose confidence; trigger deceleration and request local replanning when consecutive T frames are below threshold; update cost map according to obstacle mask and generate initial path with heuristic search; perform trajectory smoothing with time-elastic band algorithm and impose acceleration and jerk constraints.

[0118] Output: Velocity command, angular velocity command, local trajectory, confidence alarm signal, and log.

[0119] V. Output and Alarm

[0120] Input: Trajectory and command from control module, confidence alarm.

[0121] Process: Publish to robot control bus at control loop period, record timestamp and key statistics; when confidence is low, attach reason flag (insufficient inliers or excessive error).

[0122] Output: Executable velocity and trajectory commands, diagnostic and alarm information.

[0123] Time and Frequency: Default 30-60 frames per second for acquisition and feature extraction; default 20 times per second for matching and pose estimation; default 10 times per second for trajectory replanning; overall end-to-end perception to control command latency is not higher than 100 milliseconds.

[0124] To ensure implementability and reproducibility, this embodiment provides a set of default configurations and ranges:

[0125] 1. Inner point threshold M: default 50, can be adjusted in the range of 30-80 according to the map density and the lens view angle; select to maximize the success rate of positioning on the validation set.

[0126] 2. Reprojection error threshold: default 1.5 pixels, can be adjusted in the range of 1.0-2.0; recommended to select in the percentile method on the factory measured data.

[0127] 3. Continuous frame number T: default 3 frames, can be adjusted in the range of 3-5 frames; take a smaller value to avoid response lag when the frame rate is low.

[0128] 4. Loop similarity threshold: default 0.7, can be adjusted in the range of 0.6-0.85; prioritize to reduce the probability of false loop.

[0129] 5. Soft mask threshold: default 0.5, used to generate obstacle mask; can be increased to 0.6-0.7 when there are more reflection areas.

[0130] 6. Inflation radius: default 15 centimeters; adjust in the range of 10-30 centimeters according to the robot shape and safety distance.

[0131] 7. Grid resolution: default 5 centimeters; can be reduced to 2-3 centimeters when the channel is narrow, and can be increased to 10 centimeters in large space to improve speed.

[0132] 8. Trajectory constraints: default maximum linear speed 1.0 meters per second, maximum angular speed 90 degrees per second, maximum linear acceleration 1.5 meters per second squared, maximum angular acceleration 180 degrees per second squared.

[0133] 9. Online adaptation: default minimum trigger interval 0.5 seconds, single parameter update step not more than 5% of the initial value, cumulative update not more than 4 times per minute; stop updating and enter conservative mode when the confidence is not restored after three updates.

[0134] 10. Confidence determination and action: determine as low confidence if any condition is met; low confidence action includes reducing linear speed to not more than 0.3 meters per second and increasing re-planning frequency to 15 times per second.

[0135] Parameter setting method: default to select initial values based on historical validation data, then use grid search or item-by-item fine tuning during on-site trial operation; when conflicts occur, take the more conservative value with safety priority.

[0136] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present application and not to limit it. Although the present application has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, which should be covered in the scope of the claims of the present application.

Claims

1. A visual intelligent navigation method for industrial handling robots based on artificial intelligence, characterized in that, include: Acquire the input image from the calibrated monocular camera; The image is input into an illumination-adaptive feature extraction model to obtain an illumination-robust feature map. The model includes a feature encoding layer, a brightness-saturation estimation subnetwork, and a multi-head attention fusion layer. The brightness-saturation estimation subnetwork is used to output a pixel-level illumination weight map; the multi-head attention fusion layer is used to perform weighted fusion of the encoded features output by the feature encoding layer according to the pixel-level illumination weight map and the multi-head attention mechanism to obtain the illumination robust feature map, and read feature points and descriptive information from the illumination robust feature map. The illumination-adaptive feature extraction model is a lightweight Transformer network, which includes image block embedding, local window multi-head self-attention and feedforward layers, and outputs the illumination-robust feature map. The feature points and description information read from the illumination robust feature map are matched with the three-dimensional key points in the pre-constructed environmental semantic map with a metric scale in a two-dimensional to three-dimensional sparse match. The initial six-degree-of-freedom pose of the robot is obtained by perspective-n-point pose solving based on consistency judgment, and the optimized pose is obtained in the graph optimization of the sliding window. An initial path is generated through heuristic search based on optimized pose and target point, and dynamic replanning is performed based on obstacle mask; when the number of matched in-points within consecutive T frames is less than threshold M or the average reprojection error is greater than threshold ε, the robot speed is reduced and local replanning is triggered, where T, M and ε are preset. Obstacle detection employs a reflection suppression segmentation network, which includes a main branch that extracts multi-scale features using dilated convolution and an auxiliary branch based on optical flow consistency estimation. The fused output yields a reflection artifact confidence map, which is then used to generate an obstacle mask to update the path cost map. It also includes online adaptive steps: When the pose confidence is below the threshold, only the attention scaling parameters of the illumination adaptive feature extraction model are updated in small steps. The update magnitude is limited by a preset upper limit and uses an exponential moving average.

2. The visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in claim 1, characterized in that, The Transformer network is obtained through the following training: Based on a real-world image library, we constructed synthetic samples with different light intensities and spectra using lighting rendering and electric arc light spectrum conditional generative adversarial networks, and superimposed sensor noise on underexposed samples. Using samples of different lighting conditions in the same scene as positive sample pairs and samples of different scenes as negative sample pairs, we employ self-supervised contrastive loss for training and introduce saturation suppression regularization to limit attention weights in saturated regions.

3. The visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in claim 2, characterized in that, The total loss during training of the Transformer network consists of a weighted sum of a contrast loss term and a saturation suppression regularization term, wherein the saturation suppression regularization term constrains the attention weights of overexposed regions based on a soft mask obtained from the luminance-saturation estimation subnetwork.

4. The visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in claim 1, characterized in that, The environmental semantic map includes three-dimensional key points with metric scales and appearance descriptors, and the two-dimensional-three-dimensional sparse matching uses consistency judgment to remove outliers.

5. The visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in claim 1, characterized in that, The cost function for the sliding window graph optimization includes reprojection error and lap closure constraints, and a robust kernel function is used to suppress anomalous observations caused by highlights and shadows.

6. The visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in claim 1, characterized in that, The dynamic replanning uses a time elastic band algorithm to smooth the initial path and satisfy velocity and acceleration constraints.

7. A visual navigation device, comprising a monocular camera, a memory, and a processor, wherein the memory stores a computer program, characterized in that: When the processor executes the computer program, it implements the steps of the visual intelligent navigation method for industrial handling robots based on artificial intelligence as described in any one of claims 1 to 6.

Images