30 results about "Global vision" patented technology

The invention provides a posture recognition method and system based on a posture recognition neural network, and relates to the technical field of posture recognition, and the method comprises the steps: constructing a standard library containing safety and dangerous posture types, and marking a plurality of key points to generate a soft label sample data set; defining the key points as graph nodes, and constructing a feature matrix and an adjacent matrix; a double-branch fusion network model is designed, key point topological structure features are extracted through image convolution branch, image global context features are extracted through global visual branch, and attitude similarity spectral vectors are output after fusion; and finally, the attitude state is judged by calculating the geometric difference degree between the image to be analyzed and the most similar attitude template on the key point distance and the joint angle and comparing the overall difference between the image to be analyzed and the safety / danger camp. According to the method, the dual advantages of data driving and rule verification are combined, the recognition robustness and decision interpretability of dangerous postures are remarkably improved, and the safety risk is effectively reduced.

The invention relates to a mobile welding robot weld path planning method fused with binocular vision, and belongs to the technical field of mobile welding robots. Comprising the following steps of S1, weld joint feature recognition, specifically, weld joint images are collected and processed through a global vision unit fixed to a movable chassis and a local vision unit fixed to the tail end of a mechanical arm, and weld joint feature information is recognized and extracted; s2, generating and fusing double-level tracks; s3, track execution and correction; and through a visual odometer of the movable chassis, the translation drift distance of the movable chassis in the linear direction is collected in real time, the controller controls the mechanical arm to move according to the track according to the final welding track, and weld joint welding is completed. The recognition robustness is improved through double vision, welding seam global positioning and local detail recognition are both considered, interference of welding fume and arc light is effectively resisted, and the feature point leak detection / false detection rate is reduced; the adaptive region adapts to multiple processes, the trajectory precision is optimized by dynamic weight, and the overall precision of trajectory planning is greatly improved.

The invention discloses a robot bolt assembling method and device based on multi-view vision and force sense cooperation, and belongs to the technical field of intelligent control and industrial automation. According to the method, the positions and poses of an assembled workpiece and a threaded hole are perceived cooperatively through global vision and hand-eye vision, a decision module comprehensively processes multi-source vision information, a motion control instruction is generated, and a mechanical arm is guided to complete coarse positioning and fine positioning before assembly. In the assembling process, the force sense sensing module obtains contact force and moment information in real time, the decision making module dynamically adjusts the posture of the tail end of the mechanical arm based on force sense feedback and a force control algorithm, and posture alignment of the bolt and the threaded hole is achieved. And after alignment is completed, the end effector is rotated to drive the bolt to be screwed into the threaded hole under the condition that the tail end posture is kept stable, and assembling operation is completed. The bolt assembling precision and stability are improved, and the method is suitable for multi-specification bolts and complex assembling environments.

The invention discloses a mechanical arm guiding method and system for a track board fine adjustment robot, and the method comprises the steps: controlling a mechanical arm to drive a camera to shoot a calibration board, and building a global vision pose mapping relation from a pixel coordinate system to a tightening shaft coordinate system based on collected coordinate data; during actual operation, the mechanical arm is controlled to move to a preset ideal position to shoot a target image, and the center position and the rotation angle of a fine adjustment claw tightening groove are extracted through a visual recognition algorithm; based on the established visual pose mapping relation, the relative position relation of the target under the pixel coordinate system is converted into the target offset under the tightening axis coordinate system, and the target coordinate for guiding the mechanical arm to operate is synthesized by combining the ideal position coordinate; according to the target coordinate and the rotation angle, the mechanical arm and the end effector are controlled to complete butt joint operation; through the visual pose mapping module and an indirect mapping resolving strategy, the influence of installation errors and tool offset is overcome, and the operation precision, efficiency and reliability of the track slab fine adjustment robot are improved.

The present application relates to the technical field of robot control, and especially relates to a multi-robot optical processing safety control system and method based on machine vision. The method covers the global vision of multi-robot collaborative processing through multiple vision sensors, and tracks multiple feature targets in the workspace in parallel. The high-reflective marker points on the robots in the image are located at the sub-pixel level. After the pixel boundary box is used to determine and issue a primary early warning event or a collision warning event, the pose solution frequency of the involved robot is dynamically adjusted, and the three-dimensional space pose data of the robot is solved. Whether to execute motion intervention or emergency stop is determined according to the three-dimensional space shortest Euclidean distance and the predicted collision time between the involved robots. The present application uses a low-complexity two-dimensional multi-target tracking method for global continuous monitoring, and performs high-precision pose solution and collision detection in stages, so as to realize dynamic optimization allocation of computing resources, avoid monitoring false negatives, and effectively reduce system running load.

The invention discloses an automatic driving vehicle path planning method based on a remote sensing image, and relates to the technical field of image recognition. The method comprises the following steps: an edge server pre-processes an original remote sensing image collected by an unmanned aerial vehicle and received in real time; analyzing the preprocessed remote sensing image, identifying roads and vehicles in the remote sensing image, and marking the vehicles as CAV vehicles and HDV vehicles; constructing a vehicle kinetic equation, analyzing the traffic flow, and selecting a car-following model or a lane changing model; after the lane changing model is selected, the automatic driving vehicle carries out rasterizing processing on the road ahead, and the optimal path of migration to a target grid is calculated. According to the method, through cooperative processing of the unmanned aerial vehicle remote sensing image and edge calculation, the global view of the autonomous vehicle is improved, the unmanned aerial vehicle can accurately obtain a long-distance road topological structure and dynamic obstacle distribution, and planning lag or decision errors caused by a local view blind area are avoided.

This specification provides a wafer cassette docking system that, through the systematic integration of local environmental control, multimodal sensing, and active compliant adjustment functions, achieves an overall performance improvement in the wafer cassette docking process. It can form a stable positive pressure laminar flow air curtain at the initial stage of docking, effectively isolating external particles and providing a continuous ultra-clean environment for the wafer. Employing a multi-sensor fusion scheme consisting of global vision, laser ranging, and feature recognition, it can quickly and comprehensively acquire the six-degree-of-freedom pose information of the wafer cassette, providing a high-precision data foundation for subsequent fine alignment. A micro-motion adjustment platform based on piezoelectric ceramic actuators, combined with real-time closed-loop feedback control, enables nanometer-level active compliant alignment, significantly improving alignment accuracy and reliability. An electromagnetic locking mechanism ensures the stable maintenance of the final pose. The collaborative work of each module improves docking accuracy and efficiency, and also enhances the system's adaptability and stability under different operating conditions.

This invention discloses a staged multi-policy instruction scheduling method and system for VLIW architecture. The method includes: basic step S1. Receiving the symbolic assembly structure (SAS); S2. Configuring three types of scheduling vision interfaces and registering corresponding scheduling policies, including a global vision interface, a loop vision interface, and a linear vision interface; S3. Executing loop vision scheduling, traversing all loop blocks, and concurrently calling the loop vision interface to generate candidate scheduling schemes; S4. Executing linear vision scheduling, after all loop blocks have been scheduled, traversing the remaining unscheduled basic blocks, and concurrently calling the linear vision interface to generate candidate schemes; S5. Performing competitive selection and register allocation on the candidate scheduling schemes generated in each stage; S6. Outputting the optimized SAS. This invention can efficiently adapt to various VLIW processor architectures, improving instruction-level parallelism and code execution efficiency.

The invention relates to the technical field of robot hand control, and discloses a multi-camera-based robot hand positioning and musical instrument playing control method and system, and the method comprises the steps: obtaining a global image through a head camera, and providing a hand rough position; the fingertip camera obtains a local image and provides fine positioning of a target; fusing coarse and fine positioning, calculating a relative position, and planning a motion track; a robot execution mechanism is controlled to execute the track to complete playing; monitoring an actual position, calculating an error, and correcting a joint when the error exceeds a threshold value; the system comprises a head camera module, a fingertip camera module, a visual fusion module, a control module, a robot execution mechanism and a feedback and correction module. According to the invention, by acquiring the high-resolution local image and combining the global rough position information provided by the head camera module, effective combination of rough positioning and fine positioning is realized, and the problem that single global vision is easily blocked by hands or the precision is insufficient due to long distance is effectively overcome.

The invention relates to the technical field of industrial welding robots and automatic welding control, in particular to a thick-wall pipe groove teaching-free welding method and system. The method comprises the steps that a global vision unit and an assembled thick-wall pipe groove area are obtained, and the actual pose of a workpiece and the geometric center line vector of a welding seam are obtained through subarea scanning, point cloud registration and model registration; extracting geometric characteristic parameters of the cross section of the welding seam, matching the groove form code and binding preset welding layer channel information to form a welding seam attribute set; then path planning, multi-layer multi-channel decomposition and welding program generation are executed, and a welding program and a theoretical track are obtained; and finally, correcting subsequent interpolation points on line, scanning the welding seam again under the triggering condition, and updating the welding seam attribute set. According to the method, the welding seam attribute set capable of driving multi-layer and multi-pass welding is constructed, so that the variable cross-section welding problem caused by non-uniform assembly of the thick-wall pipe is effectively solved, and the welding self-adaptive capacity and the welding quality are remarkably improved.

The invention provides a cloud side-end large and small model collaborative management method, system, device and medium, and the method comprises the steps: constructing a cloud side-end collaborative architecture, enabling a cloud side to be used for pre-training a resource management large model, carrying out the quantitative processing of the resource management large model, obtaining a resource management small model, and deploying the resource management small model to a side end and a vehicle end, and the side end is used for storing and finely adjusting the small resource management models, finely adjusting the small resource management models of the side end according to a model fine adjustment period and issuing the small resource management models to the vehicle end, and the cloud end also updates the large resource management model of the cloud end according to a model updating period and carries out quantitative processing and deployment again. According to the method, the advantages of cloud training resources and global view and the real-time response and online adaptability of the side end and the vehicle end are combined, and a large and small model cooperation mechanism is introduced, so that communication resources and computing resources in the Internet of Vehicles are efficiently scheduled, and the requirements of low time delay and high adaptability in a dynamic environment are met.

The key problem of building global vision in vehicle networking is that each vehicle can obtain the overall information of its surrounding environment through the interaction between vehicles and edge servers. However, under the current technical conditions of vehicle networking, data transmission and running efficiency are one of the constraints to achieve the construction of global vision. Therefore, this paper designs a method of optimizing and fusing the field of view of vehicle-mounted cameras based on road segment space division, which includes four steps: (1) data preprocessing: grid connected component construction algorithm based on breadth-first and union set optimization (BGBC); (2) initial allocation: connected component-based greedy allocation algorithm (CCFA); (3) optimization: segmented genetic algorithm optimization guided by greedy strategy; (4) performance index verification through comparative experiments. This algorithm can help reduce the time and energy required for a data packet transmission, while reducing the complexity of running the algorithm and reducing the consumption of edge servers.

The invention discloses a text-guided image editing method and device, a medium and equipment, and relates to the technical field of image editing. Aiming at the limitation of an existing method in understanding an editing instruction, global vision and local entity depth perception are firstly carried out on an original image, and the original image is converted into priori knowledge in a natural language form, so that the knowledge of non-explicit modeling of a diffusion model in image editing is made up, the cognitive ability of an image editing scene is improved, and the image editing efficiency is improved. Therefore, priori knowledge is combined with the editing instruction, multi-angle deep editing reasoning is performed on the incomplete or fuzzy editing instruction, a clearer editing scheme with constraint is generated, the interpretability and the constraint of the editing instruction are enhanced, the adaptability of the model in a complex editing task is improved, and the editing efficiency is improved. And the quality of an image editing result is improved.

This disclosure relates to the fields of industrial automation and machine vision technology, and in particular to a dynamic scanning positioning and grasping system and method based on J1-axis follow-up vision. The system includes: a robotic arm having a J1 axis and a rotating base that rotates synchronously with the J1 axis; an end effector mounted at the end of the robotic arm for grasping a workpiece; a follow-up vision assembly including a global vision unit whose field of view covers the working area of ​​the end effector; and a control assembly communicatively connected to the robotic arm, the end effector, and the global vision unit. This solution addresses the problems in existing technologies such as difficulty in adapting to non-standard sized sheet metal, high reflective interference, high height measurement costs, low operating efficiency, and unstable long-term grasping accuracy.

The invention discloses a teaching-free robot welding system based on panoramic scanning and a welding control method. The teaching-free robot welding system comprises the procedures of remote process pretreatment, on-site workpiece feeding, global vision panoramic scanning positioning, fine positioning vision weld joint deviation correction, intelligent welding path planning and parameter calling, robot automatic welding and workpiece discharging. The welding system is composed of an automatic welding mechanism which comprises an inverted robot, a ground rail walking mechanism and a welding system body. The visual identification system comprises a global visual unit and a fine positioning visual unit; the software system comprises welding intelligent planning software, welding intelligent execution software and an integrated control system; the three parts are cooperatively completed. Workpiece model identification and pose detection are strengthened in the panoramic scanning stage, and the positioning precision is guaranteed; the influence of workpiece errors on the welding seam quality is reduced through dual vision cooperation in the welding seam treatment stage; in the welding process, execution is carried out strictly according to a path autonomously planned by software and called process parameters, and the influence of human factors on the welding quality is reduced.

The utility model provides a kind of integrated vision's laser optical system, including frame and laser optical module, global vision module and coaxial vision module being installed on frame, laser optical module includes laser incidence assembly, Z-axis component, beam combiner component and XY-axis component sequentially distributed along laser incidence direction, beam combiner component includes beam combiner, the side of beam combiner towards Z-axis component is the first coating for laser direct through, the side of beam combiner towards XY-axis component is the second coating of inclination design and for reflected light reflection to coaxial vision module;The field of view direction of global vision module and the irradiation direction in XY-axis component are towards working area.The utility model can satisfy the demand of laser processing in the face of complex workpiece and high-precision marking.

The invention relates to a method for training a machine learning model (200) to classify a state (202) of a target object (204) for behavior planning of an ego vehicle depending on the state (202) of the target object (204), wherein the target object (204) can be detected starting from the ego vehicle, wherein the machine learning model (200) comprises a classifier (206), a state encoder (208), a global vision encoder (210), an object-related vision encoder (212) and a map encoder (214).

The application provides a multi-sensor cooperative wafer pose real-time correction method based on deep learning, and belongs to the technical field of semiconductor intelligent manufacturing. The method adopts a heterogeneous perception architecture combining global vision in the eye-off-hand mode and local laser scanning in the eye-on-hand mode, extracts complementary features from texture images and depth point clouds through a specially designed double-flow multi-modal attention network, and realizes sub-millimeter-level prediction of the six-degree-of-freedom pose of a warped wafer. Based on the prediction result, a variable impedance control model is used to generate a compliant compensation trajectory of a mechanical arm, and the stiffness of the mechanical arm is dynamically adjusted at the contact moment, thereby completely solving the stress damage problem in the wafer grabbing process.

The invention belongs to the field of quality management, and discloses a quality system self-optimization method and system based on PDCA circulation. The method comprises the following steps: collecting multi-source heterogeneous quality data, and constructing a quality knowledge graph comprising six types of nodes including personnel, equipment, materials, methods, environments and measurement data; constructing a causal graph by adopting a causal inference algorithm, calculating an average causal effect of each reason variable on the quality index, and positioning a root dependent variable of quality fluctuation; taking the root dependent variable, the system parameter and the resource state as a state space, and making a decision through a reinforcement learning agent; setting a causal effect prediction network, calculating a causal guidance reward based on the output of the causal effect prediction network, and updating the intelligent agent after weighted combination with an actual reward fed back by the environment; and verifying the security of the optimal disposal scheme in a shadow execution mode, and then deploying the optimal disposal scheme to a production environment for execution. The spanning of the quality system from execution parameter fine tuning to management logic remodeling is realized, and the method has global view and deep self-evolution capability.

This invention relates to the field of robot perception and control technology, specifically to a multimodal perception fusion system and method for a linkage-type dexterous hand. The multimodal perception fusion system of this linkage-type dexterous hand includes an array of tactile sensors distributed on the finger contact surface. In this invention, by setting up a spatially coordinated layout of multimodal sensors and a noise suppression preprocessing module, the accuracy of environmental perception is comprehensively improved. A dense array of tactile sensors is arranged on the finger contact surface to accurately capture the microscopic texture features of the object surface. High-sensitivity pressure sensors are embedded in the joint load-bearing nodes to provide real-time feedback on changes in grasping force. A proximity sensor is integrated into the fingertip to provide millimeter-level distance warning before contact with an object. A three-dimensional spatial positioning system is constructed in conjunction with a global vision sensor. To address sensor signal interference issues, a sliding window mean filtering technique is used to effectively suppress abnormal fluctuations caused by electromagnetic noise.

The application discloses a multi-level visual guidance multi-modal dialogue summary method, relates to the technical field of Internet and artificial intelligence, and uses a pre-trained CLIP model to extract global features and local features of visual information contained in a dialogue, uses a pre-trained model T5 to perform text feature extraction on text of the dialogue, obtains visual features and text features rich in deep semantic information, fuses and aligns global visual information and local visual information with text features through a local multi-modal attention cross module and a global multi-modal attention cross module, and fuses and splices text features guided by global vision and local visual features guided by semantics through a modal fusion module, so that multi-modal dialogue information can be complementary to each other, and the context of the dialogue is paid attention to, thereby improving the quality and accuracy of generated summaries.

The invention discloses an adversarial sample generation and initial momentum optimization method and device for pre-deep search, and belongs to the technical field of image processing, and the initial momentum optimization method comprises the steps: adding a gradient regularization item on a classification loss function, and forming a pre-search loss function; multiple rounds of iterative search are carried out by maximizing a pre-search loss function, a gradient regularization item is used for limiting a search path in a sharp area in which the gradient norm of a classification loss function is higher than a preset condition, and finally a global initial momentum is generated. Searching is carried out in a classification loss function sharp area based on gradient regularization in a pre-searching stage, an initial momentum with global view and direction stability is generated, consistent initial direction guidance is provided for subsequent formal attacks, the problems that gradient directions are inconsistent and local optimum is likely to be caused in countermeasure attacks are solved, and the method is suitable for large-scale popularization and application. Therefore, the black box migration capability of the generated adversarial sample is remarkably improved, and overfitting of the source model is effectively avoided.

This invention discloses a visual representation method based on a bidirectional state-space model—Vision Mamba (Vim). The Vim model first segments the input image into a series of image patches and linearly projects them into a vector sequence, which is then input into the Vim module for efficient sequence modeling. This method is the first to apply the Mamba state-space model to the field of computer vision and introduces a bidirectional state-space modeling approach to optimize the lack of global vision in processing visual data. Simultaneously, it utilizes positional embedding to provide spatial information and location awareness, making the model more robust in intensive prediction tasks such as semantic segmentation, object detection, and instance segmentation. Furthermore, thanks to the efficient design of the Mamba algorithm, Vim exhibits sub-quadratic time complexity and linear memory complexity, showing a significant efficiency advantage compared to visual models based on the Transformer structure. This invention also provides a corresponding visual representation device based on the bidirectional state-space model.

This invention provides a posture recognition method and system based on a posture recognition neural network, belonging to the field of posture recognition technology. The invention constructs a standard library containing safe and dangerous posture categories and generates a soft-label sample dataset by annotating several key points. Key points are defined as graph nodes, and feature matrices and adjacency matrices are constructed. A dual-branch fusion network model is designed, extracting topological features of key points through a graph convolution branch and extracting global contextual features of the image through a global vision branch. After fusion, a posture similarity spectrum vector is output. Finally, the posture state is determined by calculating the geometric difference between the image to be analyzed and the most similar posture template in terms of key point distance and joint angles, and comparing its overall difference with the safe / dangerous categories. This invention combines the advantages of data-driven and rule-based validation, significantly improving the robustness of dangerous posture recognition and the interpretability of decisions, effectively reducing safety risks.

The present application relates to a kind of product warehouse-out identification method based on camera detection, belong to the cross technical field of intelligent warehousing logistics and machine vision.The method includes: obtaining the global vision data of product to be out of warehouse and associating job scene information, generate standardized warehouse-out data set;Construct multi-modal identification verification model, decode bar code information in parallel, carry out feature matching and defect detection simultaneously, output structured verification result and full-quantity process evidence chain;Based on abnormal result, start acquisition failure adaptive compensation mechanism, adjust camera acquisition parameter and update data set, while performing incremental training on model;Aggregation compliance product information and with target warehouse-out order dynamic matching, cyclically execute product warehouse-out identification closed loop process.Multiple dimensions cross verification, environmental adaptive compensation, model rapid iteration and whole-process traceability are realized, the problems of single verification dimension, poor environmental adaptability and insufficient traceability in the prior art are solved.

The invention is suitable for the technical field of agricultural automation equipment, and provides a ridge sealing crop field operation trolley based on a dual-camera recognition system, and the ridge sealing crop field operation trolley comprises a crawler-type chassis, and further comprises a multi-degree-of-freedom adjusting module, a dual-pesticide-chamber pesticide supply module, a dual-camera visual module and a control unit. According to the device, through cooperative perception of the global vision unit and the local vision unit and combination of flexible operation of multi-axis machinery and classified pesticide application, the problems that the field view of ridge sealing crops is limited, the operation precision is low, and the types of weeds cannot be accurately recognized are effectively solved; the device has the advantages of compact structure, high adaptability, high pesticide utilization rate, accurate and efficient operation and the like.

The invention discloses a four-axis unmanned aerial vehicle multi-task loading system integrating a man-machine interaction mechanical arm and global vision. The four-axis unmanned aerial vehicle multi-task loading system comprises a mechanical arm module, a global intelligent vision module, a man-machine interaction module and a multi-task unmanned aerial vehicle loading platform. The mechanical arm module is composed of a clamping jaw mechanism, a first intelligent camera, six bus intelligent steering engines and a hollow connecting plate. The global intelligent vision module is composed of five intelligent high-definition cameras; the man-machine interaction module is operated by using a PC virtual machine and an ROS system and adopts a self-developed four-layer architecture design; the multi-task unmanned aerial vehicle loading platform is composed of four protected rotor wings, model airplane motors, electric speed controllers, four supporting legs, a system box, a storage platform and a connecting frame. According to the method, full-automatic path planning and gesture arm control and remote control flight and gesture arm control are combined, and the method has the advantages of man-machine interaction, global vision, safety and flexibility and is suitable for multi-task scenes.

The invention provides a multi-sensor collaborative wafer pose real-time correction method based on deep learning, and belongs to the technical field of semiconductor intelligent manufacturing. According to the method, a heterogeneous sensing architecture combining global vision of an eye outside the hand and local laser scanning of the eye on the hand is adopted, complementary features are extracted from texture images and depth point clouds through a specially-designed double-flow multi-mode attention network, and submillimeter prediction of the six-degree-of-freedom pose of the warped wafer is achieved. And on the basis of the prediction result, a compliance compensation track of the mechanical arm is generated by utilizing a variable impedance control model, the rigidity of the mechanical arm is dynamically adjusted at the moment of contact, and the stress damage problem in the warping wafer grabbing process is thoroughly solved.

The present application relates to glass curtain wall cleaning, in particular to a glass curtain wall cleaning system and method based on the cooperation of a mother bin and a water gun for targeted cleaning, comprising a mother bin, a negative pressure adsorption type cleaning robot and a central control system, the central control system is integrated in the mother bin, specifically comprising: a stain intelligent recognition and classification unit, based on a deep learning model, the image sent by the enhanced global vision module is subjected to semantic segmentation, the stain area, stain type and stubborn grade are identified; a targeted cleaning decision unit determines the targeted cleaning parameters according to the position of each stain area, stain type and stubborn grade, and plans the water gun spraying path and the robot cleaning path according to the position of all stain areas; a water gun motion control unit calculates the corresponding spray elevation angle compensation value of each stain area according to the water gun spraying path and the laser ranging result, and calculates the angle of each joint of the mechanical arm; the present application can overcome the defects of low cleaning efficiency and poor cleaning effect.

This invention provides an integrated laser optical system, comprising a frame and a laser optical module, a global vision module, and a 3D vision module mounted on the frame. The laser optical module includes a laser incident component, a galvanometer component, a field lens component, and a beam combiner component, sequentially distributed along the laser incident direction. The beam combiner component includes a tilted beam combiner lens. The global vision module is positioned on one side of the beam combiner lens. The side of the beam combiner lens facing the field lens has a first coating for laser reflection, and the side facing the global vision module has a second coating for direct illumination of camera incident light. The laser reflection range of the beam combiner lens, the field of view of the global vision module, and the field of view of the 3D vision module all face the working area. This invention can meet the marking requirements of complex workpieces and high precision from a top-down view of the working area.