37 results about "Robot vision systems" patented technology

A dynamic vision system includes a variable focus liquid lens optical assembly. The dynamic vision system includes a variable lighting assembly. The dynamic vision system includes a control system configured to adjust one or more optical parameters and data collected from the variable focus liquid lens optical assembly in real time. The dynamic vision system includes a control system configured to adjust the variable lighting assembly. The dynamic vision system includes a processing system that dynamically learns on a training set of outcomes, parameters, and data collected from the variable focus liquid lens optical assembly to train a set of machine learning models to control the variable focus liquid lens optical assembly to optimize collection of data for processing by the set of machine learning models.

The invention belongs to the field of machine learning and target detection, and discloses an infrared and visible light fusion pedestrian detection method based on weight autonomous optimization. According to the method, feature extraction is carried out on infrared and visible light images through a convolutional neural network; and through a learnable proportionality coefficient, the extracted features are superposed back to the original image according to respective proportions. In the feature fusion stage, the fusion proportion of the infrared and visible light modes is dynamically adjusted through a learnable fusion weight coefficient. In the training stage, end-to-end optimization is carried out by using a large-scale annotation data set, so that the superposition proportion and the fusion weight alpha of each mode can be adaptively adjusted to adapt to different illumination conditions and scene changes, and the generalization ability of the model is remarkably improved. In a robot vision system for night pedestrian detection, an infrared and visible light modal fusion method based on machine learning weight optimization is adopted, and the accuracy and robustness of pedestrian detection are effectively improved.

The invention belongs to the technical field of image processing, and particularly relates to a GIS early warning method based on cleaning result analysis of a cleaning robot, which comprises the following steps: acquiring original image data acquired by a camera of the cleaning robot; analyzing view angle inclination influence according to the original image data, determining a feature point offset condition, and generating distortion distribution information; performing correction processing on the original image data according to the distortion distribution information to generate corrected first image data; edge features are extracted for the first image data, and contour information related to clean area division is obtained; dividing a cleaning area according to the contour information, and determining cleaning state distribution information of each area block; and generating a GIS blind area distribution map according to the cleaning state distribution information. According to the method, the problems of image distortion, inaccurate region division and the like in a visual system of the cleaning robot are effectively solved, the cleaning efficiency and quality are improved, and a reliable basis is provided for subsequent cleaning path planning.

The invention discloses an environment-friendly operation and maintenance robot vision system, and the system comprises an image collection module which comprises a 3D camera disposed on a mechanical arm of a robot, and is used for collecting a plane image and a depth image of a target object; the image processing module is used for preprocessing the plane image acquired by the image acquisition module and transmitting the plane image to the target detection module; the target detection module is provided with a rotating target detector obtained by training based on a YOLOv8OBB network model, and is used for detecting the image transmitted by the image processing module and outputting a target frame center point, a target frame width, a target frame height and a rotating angle; the data conversion module is used for obtaining a rotation translation matrix of the target relative to the 3D camera; and the result output module is used for converting the rotation translation matrix into a relative motion matrix of the mechanical arm and outputting the relative motion matrix to a mechanical arm control unit of the robot. Based on the scheme of the invention, an efficient and accurate water quality station robot vision system can be obtained.

The invention discloses a robot vision system construction and control method, and the method comprises the following steps: collecting environment image data and robot motion state data, carrying out the real-time preprocessing of the environment image data, generating standardized image data, and carrying out the real-time monitoring of the environment through setting a dynamic environment sensing module. When real-time grabbing operation guided by robot vision is carried out, an environment light interference real-time judgment mechanism is established, differential compensation strategies are set for different working condition scenes, the stability of visual data collection under the complex illumination condition is guaranteed, and meanwhile compensated light parameters are synchronized to an image processing unit; the problem of feature recognition distortion caused by abnormal environmental reflection can be reduced in real time, the accuracy of robot target positioning is ensured, the track error of visual guidance is reduced, and the completeness of a high-precision assembly task is ensured when it is detected that the in-place posture offset exceeds a safety threshold value.

The invention provides an unstacking exception handling system which comprises a processing unit, a display unit, an input unit and a sending unit, simulation software is loaded in the processing unit and used for building a simulation environment according to a real unstacking scene provided by a user, and the simulation environment is used for simulating the unstacking process in the real scene; when the robot vision system detects that the real environment is abnormal in the unstacking process, an unstacking abnormity prompt pops up in a simulation interface of the display unit; the processing unit receives the unstacking abnormity type fed back by the robot vision system; and according to the type of the unstacking exception, an unstacking exception handling option box pops up, and if the user selects that manual handling is needed, an exception handling task is issued to a worker in a real environment. Therefore, the unstacking process in the real environment can be monitored in real time through the virtual stack type in the simulation, so that a worker is reminded in time to deal with stacking abnormity in the dismounting process, and the real stacking process is tracked and managed conveniently.

The invention relates to the technical field of humanoid robots, and discloses a humanoid robot autonomous movement trajectory tracking method, which comprises the following steps: acquiring an environment image of a humanoid robot movement environment by using a robot vision system, and carrying out gray processing and image enhancement processing on the environment image to obtain a target environment image; extracting textural features of the target environment image according to the target environment image, and inputting the textural features into a support vector machine to identify obstacles to obtain positions of the plurality of obstacles in the environment; establishing an environment map of the moving environment of the humanoid robot by adopting a grid method; and constructing an autonomous moving path of the humanoid robot by using the environment map and adopting an ant colony algorithm based on a cosine curve transformation model. The direction of the robot is determined by positioning the target and the obstacle, and meanwhile, the ant colony algorithm based on the cosine curve transformation model is adopted to optimize the moving path, so that the moving path of the humanoid robot is shortest and smooth, the planning efficiency is improved, and the moving time of the humanoid robot is saved.

The utility model discloses a kind of intelligent cutting and welding devices, including linear ground rail, handling robot is installed on the linear ground rail, gripper is installed on the mechanical arm of handling robot, workbench is provided at one side of linear ground rail, two arc ground rails are provided at the other side of linear ground rail.Affinity effect is in at:the utility model is through the cooperation design of arc ground rail, cutting robot, welding robot, linear ground rail, handling robot, vision system, three-jaw chuck and hydraulic jacking assembly, can when titanium sponge reactor bottom flange cutting and welding, realize the reasonable zoning of cutting, handling, welding, operation is convenient, can quickly realize the cutting and welding operation of titanium sponge reactor bottom flange, suitable for all titanium sponge reactor bottom flange cutting and welding occasion, save artificial and time cost, improve operation efficiency simultaneously, reduce security risk.

The application discloses a method and equipment for automatically generating a training data set in a mowing robot vision system, and the method comprises the following steps: S01, acquiring a scene condition control signal and generating a geometric constraint signal; S02, constructing a structured prompt word and inputting the same into a text condition of a text encoder to obtain text encoding; S03, inputting initial noise, the geometric constraint signal and the text encoding into a diffusion generation model, and converting the initial noise, the geometric constraint signal and the text encoding into a preliminary image data set through image decoding; S04, preliminarily screening the preliminary image data set according to semantic consistency and visual quality, performing instance segmentation on the preliminarily screened image data set to obtain an instance mask, and finely screening the instance mask according to a quality feature of the instance mask to obtain a finally generated training image data set. The application can automatically generate a high-quality visual training data set meeting training requirements without relying on real scene collection and manual labeling.

The utility model discloses a feeding and discharging device and a feeding and discharging system.The feeding and discharging device comprises a picking assembly, a mechanical arm, a robot vision system and a distance sensor, and the picking assembly is installed at the tail end of the mechanical arm; the robot vision system is installed on the mechanical arm and used for obtaining an image of the resistor disc and obtaining the first position of the resistor disc on the first plane according to the image of the resistor disc, the distance sensor is used for detecting the distance between the picking assembly and the resistor disc, and the mechanical arm drives the picking assembly to move according to the first position and the distance. And the resistor discs are accurately picked up. According to the feeding and discharging device, through cooperation of the mechanical arm, the picking assembly, the robot vision system and the distance sensor, automatic determination of the positions of the resistor discs and automatic taking and placing of the resistor discs are achieved, manual taking and placing of the resistor discs are replaced, the working time of workers in the high-voltage and noise environment of resistor disc production and machining is shortened, and the working efficiency is improved. And the harm of high-voltage and noise environments to workers is reduced.

The invention discloses a gain-adjustable bionic photoreceptor based on a substrate bias effect and a preparation method thereof. The device comprises an MOS field effect transistor and a photodiode, a substrate of the MOS field effect transistor is in a suspended state, and the photodiode works through a photosensitive PN junction. Under the condition that proper bias voltage is applied, a conducting channel of the MOS transistor is formed. When light irradiates the device, the longitudinal PN junction injects photon-generated carriers into the substrate of the MOS transistor by using a built-in electric field, so that the potential of the substrate is changed, and the threshold voltage of the MOS transistor is reduced. Under the condition that the external bias voltage remains unchanged, the reduction of the threshold voltage of the MOS transistor causes the increase of the drain output current, and the increase of the photoresponse current of the device is realized. The photoreceptor can realize gain adjustment and has an efficient light response characteristic. Similar to a photoreceptor in a human retina, the device has adjustable gain and wide dynamic range of visible light spectrum. Meanwhile, the device is compatible with a standard CMOS technology, large-scale array integration can be achieved, and the device is expected to be applied to an intelligent robot visual system in the future.

The invention discloses a visual sensing information preprocessing method, system and device applied to a nuclear emergency robot and a medium, relates to the technical field of image processing, and solves the technical problem that the image quality of the nuclear emergency robot is poor. The image detail reduction capability in an extreme environment is effectively improved, the structural similarity of a dark area reaches 90% or above, and meanwhile, the irradiation resistance of a visual module is improved to a certain extent on the algorithm level. The framework can be applied to a nuclear emergency robot vision system and can be popularized to other emergency low-illumination radiation field scenes at the same time, and the problem that a traditional vision system is low in returned imaging quality is solved.

The invention relates to a method for intelligently co-producing gas-liquid-solid products by carbonizing camellia oleifera shells, which comprises the following steps: intelligently collecting camellia oleifera shells through a robot, feeding the collected camellia oleifera shells into a catalytic carbonization furnace for catalytic carbonization at 270-350 DEG C, and determining convergence points by using a machine learning-matrix working module and point-to-point iterative learning control, the catalytic carbonization process of the camellia oleifera shells is accurately controlled and monitored by combining temperature and pressure sensing and environment perception, digital-analog and image processing technologies of a robot 3D visual system, so that at least one of a gaseous product, a liquid product or a solid product is produced. According to the method for intelligently co-producing the gas-liquid-solid product by carbonizing the camellia oleifera shells, the camellia oleifera shells are intelligently treated, and gas-state, liquid-state and / or solid-state products can be co-produced.

The invention discloses a coal-fired power plant denitration urea automatic unpacking and feeding system and a control method, and belongs to the technical field of coal-fired power plant denitration. The system comprises a material stacking station, an unstacking station, a bag breaking station and a waste bag recycling station, and through cooperation of an AGV forklift, an unstacking robot, a 3D vision system and a bag breaking machine, full-process automatic operation of urea bags from stacking tray recognition, transferring, unstacking, bag breaking, feeding to waste bag recycling is achieved. The control method is based on the system, and through laser SLAM navigation, visual identification and a distributed coordination algorithm, accurate linkage and safe operation of all devices are ensured. According to the invention, the manual workload and the labor intensity are effectively reduced, the working efficiency is improved, the flying dust pollution is avoided, the stability and the reliability of urea solution preparation in the denitration process are guaranteed, and remarkable economic benefits and environmental benefits are achieved.

The invention provides a honeycomb structure part core lattice defect detection method and system based on robot vision, and relates to the technical field of industrial detection. The method comprises the following steps: shooting a multi-view image of a to-be-detected honeycomb structure part by using a robot vision system subjected to hand-eye calibration, and storing pose parameters of the tail end of an industrial robot in the robot vision system when the image is shot; dividing an independent core grid region in each image by using an image instance segmentation model based on deep learning; utilizing a deep learning image classification model to generate a defect category label of each core grid region; and based on the obtained pose parameters, projecting the core lattice areas in all the images one by one, and judging and removing repeated core lattices in the projection process to obtain a three-dimensional visual core lattice defect detection result of the whole part. According to the invention, accurate identification, positioning and three-dimensional visual presentation of surface defects and bottom welding defects of the honeycomb structure are realized.

The application belongs to the technical field of image processing, and particularly relates to a GIS early warning method based on cleaning result analysis of a cleaning robot, which comprises the following steps: acquiring original image data collected by a camera of the cleaning robot; analyzing the influence of perspective tilt according to the original image data, determining feature point deviation conditions, and generating distortion distribution information; correcting the original image data according to the distortion distribution information, and generating first image data after correction; extracting edge features for the first image data, and acquiring contour information related to cleaning area division; dividing the cleaning area according to the contour information, and determining cleaning state distribution information of each area block; and generating a GIS blind area distribution map according to the cleaning state distribution information. The application effectively solves the problems of image distortion and inaccurate area division in the vision system of the cleaning robot, improves the cleaning efficiency and quality, and provides a reliable basis for subsequent cleaning path planning.

The present application belongs to the technical field of damaged image restoration, and particularly relates to a damaged image restoration method for a binocular vision system, which utilizes the time sequence characteristics of the current view image of the autonomous robot and the spatial information of the adjacent reference view image, models the time sequence information of the current view image based on the conditional generative adversarial network and the improved Transformer, and fuses the spatial information of the adjacent reference view image with the backbone network by using a relevant convolution layer to decode and reconstruct the restoration result; the present application can enable the autonomous robot equipped with the vision system to better cope with the unstructured working environment, ensure the integrity and accuracy of the acquired visual image information, and improve the accuracy and robustness of the robot vision system.

According to the cleaning method and system suitable for the robot dog and the robot vision system, an independent cleaning liquid supply module and an independent high-pressure spraying module are designed, and the cleaning liquid supply module is installed on a robot body of the robot dog or the robot in an embedded mode; the high-pressure spraying module is hidden in a machine body of the robot dog or the robot in a storage mode; a control module and a self-checking module are additionally arranged, the control module is integrated in a main control bin of the robot dog or the robot, and the self-checking module automatically executes a rapid self-checking process when the robot or the robot dog is started; the robot or the robot dog is started, the self-checking module executes the self-checking process, after self-checking is completed, if self-checking is normal, the cleaning system automatically enters a low-power-consumption standby state, whether cleaning is needed or not is judged, and if cleaning is needed, the cleaning liquid supply module and the high-pressure spraying module are controlled to conduct cleaning. According to the invention, a hidden arrangement and modular design mode is adopted, so that the core targets of traceless appearance, quick disassembly and assembly, low consumption, energy conservation and flexible upgrading are realized.

This invention discloses a device and method for measuring the spatial pose guidance accuracy of a robot vision system, relating to the field of robot guidance technology. The device includes a calibration flange mounted on a multi-degree-of-freedom motion platform. Three identical cylinders—a first cylinder, a second cylinder, and a third cylinder—are arranged on the end face of the calibration flange. The laser tracker and vision system can easily fit and identify the characteristic centers of the three cylinder end face circles on the calibration flange. Based on the identified end face circle features, the vision system guides the end effector of a robot tool to the corresponding position. The laser tracker measures the actual pose of the robot tool end effector after guidance, thus accurately measuring the spatial pose guidance accuracy of the robot vision system.

The invention discloses a thermal infrared image registration method based on a region segmentation method, and belongs to the technical field of image processing. According to the method, the problem of insufficient registration precision caused by lens distortion between different cameras is solved through a dynamic feature point distribution and region segmentation technology. The method comprises the following specific steps: extracting feature points of RGB and infrared images and matching; establishing an efficient search structure based on a KD tree; dividing the image into a plurality of rectangular regions, and evaluating feature point distribution of each region; performing proximity search on a few-point region to enhance the quality of feature points; calculating a homography matrix for each region by using an RANSAC (Random Sample Consensus) algorithm; and detecting and correcting the outlier matrix in combination with PCA analysis. According to the method, non-rigid deformation of lens distortion is approximated through region segmentation, the local registration precision is improved, the algorithm robustness is enhanced, and the method is suitable for industrial production and robot vision systems.

The application discloses an environment-friendly operation and maintenance robot vision system, which comprises an image acquisition module, an image processing module, a target detection module, a data conversion module and a result output module.The image acquisition module comprises a 3D camera arranged on a robot mechanical arm and is used for acquiring a planar image and a depth image of a target object.The image processing module is used for pre-processing the planar image acquired by the image acquisition module and conveying the planar image to the target detection module.The target detection module is configured with a rotating target detector trained based on a YOLOv8_OBB network model, is used for detecting the image conveyed by the image processing module, and outputs a target frame center point, a target frame width and height and a rotation angle.The data conversion module is used for obtaining a rotation and translation matrix of the target relative to the 3D camera.The result output module is used for converting the rotation and translation matrix into a relative motion matrix of the mechanical arm and outputting the relative motion matrix to a mechanical arm control unit of the robot.Based on the scheme of the application, an efficient and accurate water quality station robot vision system can be obtained.

A data collection apparatus for calibration of a robot vision system, according to the present invention, comprises a processor that carries out functions of collecting data for calibration of a robot, the functions comprising: acquiring images of a calibration sheet positioned at an arbitrary position at n reference viewpoints serving as a reference for an operating range of the robot; calculating calibration parameters on the basis of the images; applying the calibration parameters to generate and accumulate n mask images at the n reference viewpoints; calculating the position of a new viewpoint reachable by the robot other than the reference viewpoints; and determining a viewpoint direction for generating a new mask image that minimizes an area overlapping the accumulated mask images at the position of the new viewpoint.

This application relates to the fields of robot calibration and computer vision, and discloses a robust calibration method for a line laser robot vision system based on model reconstruction. The method includes: fixing a line laser sensor and initializing a hand-eye matrix; installing a freeform surface calibration object on the robot's end effector to acquire surface point clouds and end effector pose data in multiple poses; transforming the point cloud to the robot's end effector coordinate system based on the initial hand-eye matrix to obtain a reconstructed point cloud; searching for the nearest point from the reconstructed point cloud to the calibration object surface and calculating the point-to-surface distance residual; deriving the Jacobian matrix of the residual with respect to the current hand-eye matrix using a Lie algebra right-multiplication perturbation model; establishing an optimization function with residual minimization as the objective and introducing adaptive weights; solving for the hand-eye matrix error vector; and performing manifold updates on the hand-eye matrix until convergence. This application eliminates the dependence on standard parts, removes intermediate fitting errors, and suppresses measurement posture interference through a robust weighting mechanism, thereby improving calibration accuracy and robustness.

The utility model discloses a double-welding robot internal and external synchronous welding system for a shell ring flange, which relates to the technical field of shell ring flange welding and comprises a fixed seat and a robot welding workstation, a suspension arm and a feeding frame are respectively mounted on the fixed seat, a trolley track system is mounted on the fixed seat, and the robot welding workstation is mounted on the trolley track system. A prefabricated trolley is slidably connected to the middle of the trolley track system, and the robot welding work station is installed on the top of the fixing base through a guide rail. When the internal and external synchronous welding system for the shell ring flange double-welding robot is used, a shell ring to be welded can be placed on the feeding frame, and then the shell ring is moved to the bottom of a robot welding workstation from the feeding frame through a supporting V-shaped plate lead screw lifting machine and a V-shaped supporting plate on the prefabricated trolley; and synchronous welding of the inside and the outside of the shell ring can be realized by matching with a welding robot body and a robot vision system, so that the working efficiency is improved, intelligent manufacturing is realized, and the welding quality is improved.

To provide a robot vision system capable of making calibration at the time of starting a robot unnecessary and re-calibration during an operation unnecessary.SOLUTION: A robot vision system 102 comprises: an operation control unit 164 that controls an operation of an arm 124 of a robot 100; an end effector 126 attached to the arm; cameras 130 and 132 attached to the arm to capture images 168 and 170 near the end effector; and a correction unit 162 that calculates a gain. During a visual feedback operation of the robot using the cameras, the cameras capture the images, the correction unit calculates a moving amount of the arm on the images using a design value containing the attaching positions and the attaching angles of the cameras with respect to the end effector, and calculates a gain on the basis of a ratio of the moving amount of the arm on the images to the moving amount of the arm under the numerical control of the robot, and the operation control unit operates the arm using the gain calculated by the correction unit.SELECTED DRAWING: Figure 4

The application discloses a kind of robot vision systems automatic alignment focusing methods for tool damage detection, the described include: the automatic alignment algorithm of intelligent region of interest (ROI) of fusion YOLOv5 network and improved Laplacian automatic focusing algorithm are proposed.First, search tool position and defogging denoising.Then, with tool center point as key feature, through the optimization of multi-point distribution movement and weighted correction least square method, the alignment pose of robot arm end is calculated.Finally, adaptive dynamic screening ROI focusing window, take the gradient square sum after the convolution operation of this area and Laplace operator as definition value, to determine the focusing pose of robot arm, finally get high-quality tool damage image.The method overcomes the limitations of traditional methods in multi-depth oil dirt dust harsh working conditions and the complexity of the measured target profile, sensitive to reflect definition mutation, improve focusing sensitivity, reduce alignment center point error, effectively exclude background and oil dirt dust interference, improve the efficiency and flexibility of tool damage vision detection system.

The present invention relates to a method for intelligently co-producing gas, liquid, and solid products by carbonizing camellia oil shells. The method comprises intelligently collecting camellia oil shells using a robot and feeding the collected camellia oil shells into a catalytic carbonization furnace for catalytic carbonization at 270-350°C. A machine learning-matrix working module and point-to-point iterative learning control are used to determine the convergence point. The catalytic carbonization process of the camellia oil shells is precisely controlled and monitored, combining temperature and pressure sensing with environmental perception, digital modeling, and image processing technologies of the robot's 3D vision system to produce at least one of gaseous, liquid, or solid products. According to the method, the camellia oil shells are intelligently processed to co-produce gaseous, liquid, and / or solid products.

The application discloses a kind of robot vision system circular arc trajectory guide precision measuring device and method, it is related to robot guide technical field, robot vision system identifies the end face circle of the second cylinder of calibration flange, and generates point cloud based on the end face circle of the second cylinder, and the tool center point of vision system guided robot moves along the circumferential generatrix trajectory of the end face circle of the second cylinder, laser tracker measures actual motion trajectory after being guided by vision system in dynamic measurement mode, there is also the actual motion trajectory point position after being guided by vision system in laser tracking measurement system simultaneously based on the circumferential generatrix theoretical trajectory point position of the end face circle of the second cylinder, i.e., the guide precision of robot vision system spatial circular arc trajectory can be accurately measured and calculated.

The present invention provides a hand-eye calibration method and system for a robot vision system, wherein the method comprises: installing a calibration sphere at the end of the robot, moving the calibration sphere into the workspace of a three-dimensional point cloud camera, controlling the position of the calibration sphere in the workspace, and calculating the coordinates of the center of the calibration sphere in the robot coordinate system; obtaining three-dimensional point cloud information of the calibration sphere in the workspace, and performing point cloud screening to obtain calibration sphere point cloud information; extracting the calibration sphere based on the calibration sphere point cloud information to obtain the coordinates of the center of the calibration sphere in the three-dimensional camera coordinate system; and calculating the hand-eye calibration relationship between the camera and the robot based on the coordinates of the center of the calibration sphere in each calibration in the camera coordinate system and the robot coordinate system, thereby completing the hand-eye calibration of the robot vision system. The present invention is easy to deploy, low-cost, and widely applicable, and is of great significance for promoting the development of the manufacturing industry, reducing enterprise production costs, and improving production efficiency.