2825 results about "Visual positioning" patented technology

The invention relates to the technical field of intelligent detection in the electronic manufacturing industry, in particular to a PCB warping degree intelligent evaluation system integrating visual positioning and multi-modal sensing, which comprises a multi-modal sensing unit, a visual positioning unit, an intelligent evaluation engine and a closed-loop control unit, the multi-modal sensing unit integrates laser displacement, infrared thermal imaging and strain sensors to acquire three-dimensional deformation, temperature and stress data; the visual positioning unit realizes sub-pixel-level positioning by using a high-resolution industrial camera and a feature point matching algorithm, and compensates vibration errors; the intelligent evaluation engine fuses data based on a time-space synchronization protocol, predicts a thermal deformation trend through an improved multi-modal convolutional neural network, and dynamically adjusts a qualified threshold value; and the closed-loop control unit executes sorting and rechecking according to an evaluation result, and optimizes warping and leveling parameters. According to the system, multi-dimensional accurate detection and intelligent control are realized, the PCB warping degree detection accuracy is effectively improved, the process can be dynamically optimized according to the production working condition, and the equipment fault risk is reduced.

The invention relates to the technical field of visual positioning, and provides a visual positioning method and system for precise connector assembly. Performing multi-mode HDR fusion and distortion correction on the original connector image to obtain a multi-layer fusion image matrix; and performing feature region coarse positioning in combination with a composite convolutional neural network to obtain a connector coordinate set, performing geometric feature fine positioning according to the connector coordinate set and the multilayer fusion image matrix to obtain a key point coordinate set, and performing hand-eye calibration and dynamic compensation on the key point coordinate set to generate a pose instruction. And performing vision-force control hybrid assembly processing on the pose instruction according to the sensor feedback data to obtain assembly completion state data. Through multi-modal image fusion, deep learning coarse positioning, precise geometric registration and dynamic cooperation of vision and force control, the speed, precision and stability of precise connector assembly are improved.

The invention relates to the technical field of unmanned aerial vehicle control, in particular to an unmanned aerial vehicle self-adaptive navigation system which comprises an inertial navigation system, a visible light sensor, a data fusion module, a database and a navigation control module. The inertial navigation system provides attitude, speed and acceleration data; the visible light sensor collects an image and obtains environment feature points through deep learning algorithm processing to realize visual positioning; and the data fusion module fuses two kinds of sensor data by using a dynamic weighted Kalman filtering algorithm, and dynamically adjusts the weight according to the environment to optimize the positioning information. The map library and the feature library store environment data to assist in positioning and dynamically update; the navigation control module generates an instruction according to the fused positioning information, and automatically adjusts a flight strategy by considering various factors based on an adaptive control algorithm. The system improves the positioning precision and environmental adaptability, enhances the flight safety and stability, is high in intelligence and autonomy degree, has flexibility and expandability, and can be widely applied to the field of unmanned aerial vehicles.

The invention relates to the technical field of laser engraving, and discloses a laser engraving method and system capable of automatically correcting coordinates of a galvanometer and a camera, and the method comprises the following steps: calculating a target conversion coefficient based on a calibration size and a pixel size of an image collected by the camera, and controlling the galvanometer to perform laser etching on a cross mark on the surface of a PCB (Printed Circuit Board), recording origin data of a galvanometer coordinate system; adjusting the position of the camera to enable the cross center of the view of the camera to coincide with the cross mark, and calculating the coordinate offset; coordinate space transformation is carried out on all the points to be machined, and galvanometer marking coordinate data are obtained; after laser engraving is carried out on the galvanometer marking coordinate data, an engraving area image is collected, the position residual error and the size proportion deviation are calculated, laser engraving is carried out again, and a laser engraving result is obtained. The technical problem that the visual positioning coordinate and the galvanometer marking coordinate in the laser engraving system are not consistent is effectively solved.

The invention discloses an AI-based neurosurgery auxiliary robot visual positioning system, and relates to the field of visual positioning, which comprises the steps of deploying and initializing structured light scanning equipment and near-infrared imaging equipment, carrying out multi-modal image acquisition on a surgical area, and carrying out standardization processing, space-time alignment and fusion on the acquired image. A multi-modal visual acquisition and preprocessing module, a three-dimensional tissue model construction and registration module, a visual-anatomical feature recognition and extraction module, an AI auxiliary positioning and path optimization module, an intraoperative dynamic perception and feedback control module and a target position confirmation and instruction output module are constructed. According to the method, high-precision identification and dynamic modeling of a brain tissue structure are realized, real-time identification, path planning and position correction can be carried out on a key anatomical structure in an operation process, the precision, the intelligent level and the intra-operation response capability of neurosurgery operation are remarkably improved, the operation risk is effectively reduced, and the positioning reliability and the automatic control efficiency are improved.

The invention relates to an unmanned aerial vehicle multi-modal visual positioning method and system in a GNSS denial environment, and the method comprises the steps: constructing a visual inertial mileage calculation method based on deep learning feature extraction and matching, calculating the relative positioning of an unmanned aerial vehicle, and outputting to obtain high-frequency 6-DoF pose information; performing scene matching of infrared RGB image multi-modal fusion based on the Officient-LoFTR, and obtaining absolute positioning in a world system by solving a relative pose between a current key frame picture and a matched satellite image; a factor graph optimization mode is adopted to carry out self-adaptive weight information fusion according to the confidence coefficient of absolute positioning and relative positioning under a world system, and 6-DoF pose information with high frequency and higher precision is output; based on high-frequency and higher-precision 6-DoF pose information, after the unmanned aerial vehicle is recovered, all aerial images and satellite images are fused and updated. The system comprises a calculation module and a fusion updating module. According to the invention, more accurate and high-frequency pose output can be provided.

The invention provides a spatial heterogeneous visual positioning system of a tomato picking robot, and relates to the field of deep learning, and the spatial heterogeneous visual positioning system comprises the following steps: S1, calibrating the positioning of the tomato picking robot; s2, acquiring a tomato image in a non-closed environment through a fixed camera; step S3, obtaining fruit space coordinate information under the field of view of the fixed camera; s4, judging the reachable space of the picking mechanical arm, and moving the picking mechanical arm; s5, carrying out secondary spatial information perception on the target by adopting a palm center camera, and further obtaining spatial coordinate information in a relatively closed environment; s6, performing approximate ball fitting to obtain an optimal picking object; and S7, tomato number judgment and picking pose estimation are carried out. According to the system, the fixed camera and the palm center camera are combined, the characteristics of tomato growth can be effectively handled, the positioning precision and the real-time performance are improved, the higher picking success rate and efficiency are achieved, and damage occurring in the picking process is avoided.

The invention discloses an unmanned aerial vehicle visual positioning flight optimization system based on multi-source data fusion, and belongs to the technical field of unmanned aerial vehicle navigation and control. The system comprises a data acquisition module used for acquiring RGB-D image data and multiple physical data; the environment modeling module is used for constructing a local three-dimensional environment map; the path planning module is used for generating an obstacle avoidance flight path; the instruction generation module is used for converting the obstacle avoidance flight path into an attitude control instruction and an accelerator instruction; the state prediction module is used for obtaining an unmanned aerial vehicle prediction state vector at the next moment; the positioning switching module is used for generating compensated fusion positioning data; and the flight control module is used for executing the attitude control instruction and the accelerator instruction. Through the multi-source data fusion and state prediction compensation mechanism, the problems of positioning interruption and trajectory divergence caused by loss of visual signals and unstable flight caused by sudden change of multi-source coordinate switching in the prior art are solved.

The invention discloses a visual positioning method based on an indoor fine three-dimensional model. The method comprises the following steps: acquiring an indoor original image through a camera, synchronously acquiring the attitude angular velocity of the camera, constructing a first feature data set, and correcting to generate a second feature data set; aligning the second feature data set with a pre-constructed indoor live-action three-dimensional model to generate first position estimation data under spatial boundary constraint; according to the first position estimation data, positioning a feature point region from the dynamic image data acquired in real time, and tracking a point movement track in movement track offset to generate a first feature sequence and the like; according to the scheme, through the technical means of multi-source data fusion, dynamic environment adaptability, a local updating mechanism, path planning optimization and the like, the precision, stability and efficiency of indoor visual positioning and navigation are remarkably improved, and an efficient and reliable solution is provided for positioning and navigation in a dynamic environment.

The invention relates to the technical field of unmanned aerial vehicle navigation and control, and discloses an unmanned aerial vehicle indoor and outdoor seamless navigation method fusing Beidou and visual positioning. Comprising the following steps: S1, synchronously acquiring original observation data of a Beidou satellite navigation system of an unmanned aerial vehicle, an image sequence acquired by a visual sensor and inertial data of an inertial measurement unit; s2, processing the original observation data of the Beidou satellite navigation system to obtain the position of the unmanned aerial vehicle, according to the unmanned aerial vehicle indoor and outdoor seamless navigation method and system fusing Beidou and visual positioning, through a self-adaptive fusion filter module, the fusion weight of the unmanned aerial vehicle and visual positioning is dynamically adjusted according to the Beidou signal quality; smooth transition of indoor and outdoor navigation main sources is achieved, pose jump is effectively avoided, a tight coupling depth fusion algorithm is adopted, the precision and robustness of the system are improved, the continuous and stable flight capacity of the unmanned aerial vehicle in complex indoor and outdoor environments is ensured, and the application scene is expanded.

The invention relates to a cutting path generation system based on visual positioning, cutting equipment and a cutting method. The system comprises a scanning camera control module, an image acquisition and recognition module, an image processing module, a cutting path planning module and a cutting control module. According to the system, by combining a Laplace algorithm and an LBP and GLCM texture extraction technology, the boundary recognition problem under the condition that a pattern and a background are in the same color or low contrast is solved; the path smoothness and the cutting quality are improved by adopting contour optimization and curvature compensation algorithms, the cutting sequence is optimized by combining weighting functions of the area, complexity and overlapping degree, and material deformation is reduced. And motor driving is dynamically adjusted through PID control, and stable control over the speed of the tool bit is achieved. The method has the advantages of being high in recognition precision, high in adaptability, intelligent in path planning and stable in cutting process, the automatic cutting efficiency and the finished product quality of complex patterns or same-color materials are remarkably improved, and the overall cutting precision fluctuation is controlled within + / -2%.

The invention discloses an automatic dovetail welding seam grinding control method based on robot visual positioning, and relates to the technical field of visual positioning. The method comprises the steps that after welding is completed, a dovetail welding seam image is collected and converted into a three-dimensional coordinate through a vision algorithm, and a three-dimensional point cloud is generated; smooth interpolation is performed on the three-dimensional point cloud through a B spline curve method to obtain a parameterized curve, and a control point set is optimized through a genetic algorithm to generate a global optimal polishing path; the polishing robot executes a task according to a path, a tail end sensor collects a real-time path and calculates a deviation value with a global path, and when the deviation value exceeds a preset threshold value, inverse kinematics is triggered to solve and correct the path; the hardness of the dovetail weld is measured through laser-induced breakdown spectroscopy, a comprehensive hardness value is obtained in combination with a matrix hardness database, meanwhile, a visual sensor collects the surface state, and polishing process parameters are dynamically adjusted according to the surface state; after the task is completed, the welding seam angle and flatness are detected. According to the method, the optimal path is generated through visual positioning, and automatic grinding of the dovetail welding seam is achieved.

The invention relates to the technical field of image processing, and discloses a visual positioning system and method for a complex box five-axis numerical control comprehensive processing machine. The system comprises a calibration module for collecting multi-angle images and utilizing mark point calibration coordinates to obtain a parameter mapping table, an extraction module for carrying out edge detection and contour extraction splicing features to obtain a three-dimensional data set, a marking module for calculating the actual position of a box and marking inner cavity machining areas, a recording module for analyzing and determining cutting parameters and deformation of all the areas, and a display module for displaying the cutting parameters and the deformation of all the areas. The processing module generates an obstacle avoidance cutter path and a thin-wall area machining strategy, and the comparison module monitors and compares actual effects to dynamically adjust parameters to obtain compensation data. According to the five-axis numerical control comprehensive processing machine, high-precision visual positioning and intelligent processing parameter optimization of the complex box on the five-axis numerical control comprehensive processing machine are realized, and the processing precision, efficiency and reliability of the complex box are improved.

The invention provides a pathological image visual positioning method and system, equipment and a storage medium, and belongs to the technical field of image recognition, and the method comprises the steps: extracting visual features based on a target pathological image, and determining a semantic feature vector and a knowledge feature vector based on first text description; the target pathological image is a pathological image to be subjected to target area positioning, and the knowledge feature vector is used for representing knowledge information associated with the content of the target pathological image; fusing the semantic feature vector and the knowledge feature vector to obtain a fused text feature; performing cross-modal fusion on the fused text features and the visual features to obtain fused multi-modal features, and obtaining fusion representation based on the fused multi-modal features; and based on the fusion representation, positioning a target area in the target pathological image through a multi-layer perceptron to obtain position information of a bounding box of the target area. The method can improve the capability of accurately and flexibly positioning the pathological image region level.

The invention belongs to the technical field of unmanned aerial vehicle autonomous navigation, provides an unmanned aerial vehicle trajectory planning method and an unmanned aerial vehicle design applying the method, and aims to realize real-time and efficient environmental perception and unmanned aerial vehicle autonomous obstacle avoidance trajectory generation. A group of primitive sets is predefined in a three-dimensional state space to explore the whole search space so as to realize complete coverage of a feasible region, multi-mode perception input of'depth image, current state and target direction 'is adopted, and the depth image is acquired by a depth camera; the current state is obtained by the airborne vision positioning module; multi-modal sensing input is processed by a deep learning network, future expected position, speed and acceleration information is calculated according to output of the deep learning network and serves as input of a bottom layer controller of the unmanned aerial vehicle for trajectory tracking, and finally obstacle avoidance flight in a complex environment is achieved. The method is mainly applied to unmanned aerial vehicle design and manufacturing occasions.

The invention discloses an automatic stacking control system based on visual positioning. The automatic stacking control system comprises a multi-mode visual positioning module, a model building module, a dynamic path planning module, a mechanical arm execution module, an error correction module and a storage module. The multi-mode visual positioning module comprises an RGB camera, a sensor and a laser radar and is used for obtaining three-dimensional space information of goods and a stacking area in real time. The multi-modal visual positioning module is mounted on one side of the stacking area; the model building module is used for building a stacking area, the cargo position and the position of the mechanical arm in a three-dimensional space, when the system is used, information about the position of the stacking area and information about whether new cargoes can be stacked in the stacking area or not can be obtained, then movement at any position in the space can be achieved through the three-axis cooperation mechanical arm, and the three-axis cooperation mechanical arm can move at any position in the space. The application range of the system is widened, and cargoes can be accurately conveyed to the position of a stacking area through the dynamic path planning module and the error correction module.

The invention discloses a semiconductor mechanical arm autonomous navigation and positioning method based on AI vision, and relates to the technical field of mechanical arms, and the method comprises the following specific steps: visual perception and multi-modal information fusion: collecting multi-modal data through configured industrial, depth, light field and polarized light cameras, pre-processing the multi-modal data, and carrying out multi-modal information fusion; realizing information fusion by using a deep learning network containing a feature extraction layer and a fusion layer and an attention mechanism, and outputting a unified visual description; according to the method, a deep learning target detection model is utilized, an attention mechanism is introduced to enhance the attention and extraction capability of small target features, the recognition result is optimized in combination with surface feature information recognized by polarized light imaging, the category and position of a target object can be accurately determined, meanwhile, a mechanical arm motion error model is established, and the recognition accuracy is improved. And the fused visual positioning information is fused with sensor data such as a mechanical arm joint encoder and a gyroscope by utilizing a sensor fusion technology, and a positioning result is optimized and compensated.

The invention relates to a tin ring automatic focusing laser soldering tin fixing and 3D welding spot detecting system, and belongs to the technical field of industrial automatic laser processing. The system is characterized in that a tin ring preparation module identifies pins in real time and dynamically optimizes winding parameters through visual positioning and a self-adaptive winding head, and performs defect identification and pre-repair analysis through multispectral imaging; the soldering tin positioning module is used for switching stations through a rotary table, and precise positioning and clamping of a product are realized by combining laser contour sensing and a focusing compensation algorithm; the laser tin soldering module can visually identify tin materials and automatically call welding parameters, laser power and action time are regulated and controlled in real time by establishing a regional heat conduction model, and gradient cooling is implemented after welding; and the 3D detection module adopts dual-mode scanning, constructs a welding spot three-dimensional model based on fusion data, recognizes microcracks through super-resolution processing, and discriminates the welding spot quality. And automation and intellectualization of the whole process from tin ring preparation to welding spot quality detection are achieved.

The invention relates to the technical field of semiconductor manufacturing, in particular to an automatic loading and unloading control system for semiconductor material plates. A clamping force dynamic control module; the path planning and obstacle avoidance module is used for dividing safety levels of loading and unloading areas on the basis of material plate stacking parameters, and generating an obstacle avoidance track by adopting an improved path search algorithm; optimizing an acceleration curve of the path according to the joint load data of the mechanical arm; the closed-loop verification module is used for carrying out deviation comparison on the actual pose of the tail end of the mechanical arm and the target coordinates, and if the deviation exceeds the limit, the visual positioning and morphology detection module is triggered to cooperatively reset to the path planning and obstacle avoidance module; the visual positioning and morphology detection module and the closed loop verification module are executed in sequence, and output parameters of the previous module serve as input parameters of the next module. Through close cooperation of the modules, efficient and accurate automatic loading and unloading control is achieved, the automation level is improved, and operation safety and accuracy are guaranteed.

The utility model relates to an automatic visual ink-jet line drawing machine. Comprising a conveying device used for conveying materials, a visual positioning device used for shooting and upwards conveying the positions of the materials on the conveying device, a material pressing device used for fixing and flattening the materials on the conveying device and an ink-jet line drawing device used for drawing lines on the materials on the conveying device, all of which are arranged on a rack. The ink-jet line drawing device is located above the conveying device and electrically connected with the visual positioning device. The pressing device comprises a driving source installed on the rack, a driving roller set connected with the output end of the driving source and a pressing net wound around the driving roller set. The driving roller set drives the pressing net to operate and synchronously move in the same direction as the conveying face of the conveying device. The pressing net is arranged above the conveying device and forms a conveying channel used for conveying materials with the conveying device. The automatic line drawing device has the advantages of being capable of automatically completing line drawing, convenient and fast to produce, high in line drawing efficiency and low in production cost.

The invention belongs to the technical field of augmented reality (XR) and computer vision crossing, relates to fusion application in immersive digital performance, and provides a virtual actor reconstruction and immersive presentation system based on 4D Gaussian splash modeling and XR space positioning. The system comprises a set of spherical multi-camera-position high-synchronization camera shooting matrix used for capturing dynamic images of actors; performing dynamic modeling on the multi-angle image through a 4D Gaussian splashing technology, and outputting a virtual actor point cloud model which can be used by XR equipment; vPS visual positioning and an SLAM tracking module are combined, and precise mapping positioning of a performance space is achieved in AR / MR equipment. The method supports the immersive watching of the actor image at the audience end at a 360-degree free visual angle, and realizes the natural presentation of the virtual actor without dead angles and wearing in cooperation with shielding judgment and a real-time rendering engine. The system is widely applicable to on-site entertainment scenes such as immersive theaters, text travel performances, brand activities, concerts and television programs.

The invention provides a spatial heterogeneous vision positioning system and positioning method of a pineapple picking robot, relates to the technical field of picking robot positioning, and realizes high-precision spatial positioning and attitude estimation of the robot in a complex terrain environment by comprehensively utilizing multi-source information such as vision, inertial measurement, multispectrum and structured light. The positioning stability and the real-time performance are obviously improved; by embedding a multi-modal deep learning model, efficient non-destructive detection and intelligent classification of internal defects of pineapple fruits are realized, a mechanical arm is supported to dynamically adjust a picking strategy, fruits with better quality are preferentially selected for picking, and the picking quality and efficiency are improved; and meanwhile, the preset three-dimensional terrain model and the heuristic search algorithm are combined, the motion path of the robot is effectively planned, it is guaranteed that the robot safely and stably moves in the complex environment, and the mechanical damage risk is reduced.

The invention discloses a material weighing and conveying control method and system, and relates to the technical field of self-adaptive control systems. The method comprises the following steps: setting a visual positioning identifier in an unloading area, and configuring an identification code for a material; establishing an information management database, associating the identification codes of the materials with material attributes, and generating a material association attribute table; collecting an identification code of the material, and calling a material association attribute table based on the identification code; and based on the information management database, obtaining a production line state table of each production line and a warehouse state table of the current warehouse, generating a candidate transportation list, determining a comprehensive priority score of each candidate transportation target point, and selecting the candidate transportation target point with the highest score as a final transportation target point. Compared with the prior art in which fixed rules are used for scheduling, the material transportation targets can be dynamically distributed according to real-time production requirements, manual intervention is avoided, the logistics efficiency is improved, high-requirement materials are preferentially distributed, and production interruption caused by material shortage is prevented.

The invention discloses a roadway cleaning robot autonomous navigation and operation system based on multi-sensor fusion. The roadway cleaning robot autonomous navigation and operation system comprises a mechanical system, a control system and a power system. The mechanical system comprises a crawler chassis, a mechanical arm and a big arm turntable; the control system software part presentation layer comprises a user interaction and management module, the business layer comprises a task planning and decision module, and the function layer comprises a robot autonomous walking control module, a roadway deformation detection module, a mechanical arm accessory replacement control module and a mechanical arm roadway cleaning control module. The sensing layer comprises an infrared person detection module, a laser radar mapping positioning module, a mechanical arm visual positioning module and a mechanical arm state feedback calculation module; the hardware part comprises a laser radar, an IMU, a sensor and the like. Various sensor data are fused, accurate positioning, autonomous walking, mechanical arm visual positioning, roadway deformation detection, accessory automatic replacement, roadway automatic cleaning and the like of the robot are achieved, and the intelligent level and the working efficiency of the robot are improved.

The invention relates to the technical field of red tide anomaly detection, in particular to a red tide anomaly detection method and system based on an improved multi-mode Transform. The method comprises the following steps: acquiring a remote sensing image and text data; respectively carrying out data preprocessing according to the obtained remote sensing image and text data; performing visual positioning and text selection based on the preprocessed data; performing cross-modal feature learning on the basis of a hierarchical Transform of a multi-modal capsule mechanism; guiding an attention mechanism based on a semantic path to carry out image-semantic feature alignment optimization; and carrying out multi-modal knowledge distillation on the optimized features. According to the method, an image and text preprocessing module, a visual positioning module, a keyword extraction module and other modules are combined, multi-angle accurate perception of a complex red tide scene is achieved, and the bottleneck that a red tide area is difficult to accurately recognize under the condition that data are single and information dimensions are limited in a traditional method is broken through.

The invention belongs to the technical field of PCB detection, and particularly relates to a probe tip deviation compensation method, system and equipment, and the method comprises the steps: building a reference coordinate system, controlling a probe swing arm to rotate to a vertical position and a horizontal position respectively, and employing a bonding pad combination on a special calibration plate to calibrate the probe tip deviation. An electric signal detection mode is adopted to accurately measure the deviation value of the needle point in the X-axis direction and the Y-axis direction, and the position of the probe is automatically compensated according to the deviation value. The system comprises a high-precision driving module, a visual positioning module and a signal acquisition module, and realizes a full-automatic calibration process. According to the method, a horizontal and vertical double-position detection mode is innovatively adopted, complex angle calculation is avoided, the calibration efficiency and precision are greatly improved, the problem of needle point position deviation caused by mechanical errors and long-term use in a traditional method is effectively solved, the measurement precision of PCB detection equipment is remarkably improved, and the detection precision of the PCB detection equipment is greatly improved. Meanwhile, the maintenance cost and the operation difficulty are reduced, and the method is suitable for the field of high-precision PCB detection.

The invention relates to the technical field of chip mounting, in particular to a die bonder and a die bonding method. The die bonder comprises a track device, a feeding device, a gluing device, a mounting device, a chip supply device and a discharging device. The feeding device and the discharging device are arranged at the two ends of the track device in the length direction correspondingly. The gluing device is arranged at the end, close to the feeding device, of the side edge of the rail device. The mounting device is arranged at the end, close to the discharging device, of the side edge of the rail device, the mounting device and the gluing device are located on the same side of the rail device, and the mounting device comprises a visual positioning component and a mounting component which are independently arranged; and the chip supply device is arranged on the other side of the track device and corresponds to the mounting device. The die bonder provided by the invention is compact in structure, the moving distance of each link is reduced, and the mounting efficiency is improved.

The invention discloses a multi-modal multi-scale retrieval enhancement generation method, system and device applied to external knowledge questions and answers and a medium. The method comprises question perception, multi-modal multi-scale query fusion coding, dense recall and answer generation. Analyzing a key query phrase from the question through a fine-tuned instruction language model, and accurately positioning a region of interest corresponding to the phrase in an image by using an open set visual positioning model; multi-source information is compressed and distilled into an optimal query vector through a deep fusion network integrating multi-head self-attention and an information bottleneck theory; executing a maximum inner product search to recall related knowledge; guiding the large language model to synthesize all information to generate a final answer; the system, the equipment and the medium directly perform feature fusion in the vector space based on the method, so that challenges such as information loss and cascading errors caused by a traditional normal form can be effectively dealt with, high correlation and high accuracy of retrieval knowledge are ensured, and accurate and reliable image-text questions and answers are realized.

The invention relates to an intelligent photovoltaic cleaning method for air-ground cooperation of an unmanned aerial vehicle and a cleaning robot, and belongs to the technical field of photovoltaic cleaning. The method comprises the steps that the real-time pose of the unmanned aerial vehicle is determined in a visual positioning and laser positioning fusion mode; establishing a wind speed estimation model by taking the wind speed as a predictable state quantity; solving an optimal control sequence of the unmanned aerial vehicle in a prediction time domain based on a wind speed estimation model so as to compensate wind disturbance in advance; the track of the unmanned aerial vehicle is corrected to achieve accurate butt joint of the unmanned aerial vehicle and the cleaning robot; and an intelligent decision-making method is used for realizing the timely transfer of the cleaning robot by the unmanned aerial vehicle and the cleaning work of the photovoltaic panel of the cleaning robot. Through centralized control of the central platform, a single unmanned aerial vehicle assists multiple cleaning robots, dynamic task allocation and path planning are achieved, and the problems that desert photovoltaic cleaning efficiency is low and the coverage range is limited are solved.

The invention provides a position correction method and device applied to an unmanned aerial vehicle laser SLAM system, and relates to the field of data processing. The method comprises the following steps: acquiring initial positioning data and environment data sent by sensor equipment located on an unmanned aerial vehicle; determining a first position of the unmanned aerial vehicle according to the initial positioning data; according to the environment data, environment types are determined, and the environment types comprise a dense building area, an open area and a complex area; based on the environment type, a weight combination of positioning modes for the unmanned aerial vehicle is determined, the positioning modes comprise a laser radar positioning mode, an IMU positioning mode and a visual positioning mode, and one positioning mode corresponds to one weight value; performing position correction on the first position according to the positioning mode and the weight combination to obtain a second position; and positioning the unmanned aerial vehicle based on the second position. By implementing the technical scheme provided by the invention, the position correction accuracy of the SLAM system can be improved.