4678 results about "Vision based" patented technology

The invention discloses a fine-grained multi-modal prompt learning method based on a visual language pre-training model, and relates to the technical field of visual language pre-training models.The method comprises the steps that a multi-modal prompt learning vector used for learning visual features and text features is constructed, and a fine-grained multi-modal prompt learning model is constructed based on the fine-grained multi-modal prompt learning model; global differences, subtle distinguishing visual features and category difference text features among different visual categories are generated, and a multi-modal prompt learning vector is optimized based on a cosine similarity loss function. According to the invention, through combination of dual-granularity visual prompt and text prompt, the problems that in an existing prompt learning tuning method, cooperative reasoning cannot be carried out in a dual-mode prompt representation space, and fine distinguishing features are difficult to capture are effectively solved; and superior migration generalization ability and few-sample learning ability are verified on a plurality of main image identification reference data sets, and the performance is remarkably improved in downstream tasks.

SOLUTION: A semi-supervised medical image segmentation method based on a visual language model includes the steps of: obtaining a medical image; inputting an unlabeled image and a text description into a visual language model, and obtaining a text-guided mask based on obtained dense image embedding and text embedding; inputting a labeled image into a student model, and calculating supervised loss by using obtained labeled image prediction; respectively inputting the unlabeled image into the student model and a teacher model to obtain unlabeled image prediction and a pseudo label, merging the text-guided mask with the pseudo label, and calculating semi-supervised loss by using the merged pseudo label and unlabeled image prediction; and performing medical image segmentation by using a trained student model on the basis of the supervised loss and the semi-supervised loss.EFFECT: A target segmentation region can be accurately identified by using advantages of text descriptions.SELECTED DRAWING: Figure 1

The invention discloses a three-degree-of-freedom parallel robot trajectory tracking method and system, and relates to the technical field of industry-university-research, and the method comprises the steps: obtaining motion data related to the execution trajectory of a three-degree-of-freedom parallel robot, and carrying out the data preprocessing; based on historical operation data, an intelligent prediction model is constructed to predict trajectory errors and is used for optimizing feedforward compensation of trajectory tracking control parameters; acquiring actual execution track information of the robot based on a visual sensor, and performing error calculation on the actual execution track information and an expected track to perform real-time track correction; dynamically adjusting control parameters in combination with error prediction data and visual feedback data; in the operation process of the robot, the control strategy is continuously optimized, and the intelligent prediction model is updated by using real-time data. Through the intelligent prediction model, visual feedback and adaptive control optimization, high-precision trajectory tracking, rapid error correction, energy consumption reduction and environmental adaptability enhancement are realized.

The invention discloses a patient rehabilitation training data acquisition method and system based on visual identification, and relates to the technical field of rehabilitation training data acquisition, and the method comprises the steps: carrying out the image segmentation of a patient rehabilitation training video through a deep learning model, and separating a patient motion feature region from the patient rehabilitation training video; marking key skeleton nodes in the patient motion feature region, and establishing a three-dimensional coordinate system of the key skeleton nodes; calculating a motion angle and displacement data between adjacent skeleton nodes according to the three-dimensional coordinates of the key skeleton nodes; and performing matching degree analysis on the motion angle and displacement data and a preset rehabilitation standard to generate a rehabilitation training score. According to the method, the deep learning model is adopted to perform image segmentation on the rehabilitation training video, the motion feature region of the patient is accurately extracted from the image, and key skeleton nodes are marked and analyzed in combination with the three-dimensional coordinate system, so that accurate acquisition of rehabilitation training data of the patient is realized.

The invention provides a water level distributed monitoring method and system based on a visual sensing network, and relates to the technical field of water level monitoring, and the method comprises the steps: constructing a water level propagation prediction model through multi-modal comparative learning and a graph neural network, processing a real-time water level image through the combination of adaptive wavelet transform, decoupling representation and a dense connection network, and obtaining a real-time water level prediction model; the water level measurement value and the reliability score are obtained, anomaly analysis and emergency monitoring are carried out based on state estimation, causal discovery and hierarchical reinforcement learning, high-precision and high-reliability distributed water level monitoring can be achieved, abnormal water level changes can be effectively recognized, and timely emergency monitoring parameters are provided.

The invention discloses a sign language translation method and device based on vision and word feature pre-training alignment, and belongs to the technical field of sign language translation.The sign language translation method comprises the steps that vision features of sign language videos are extracted through a vision encoder, and word text features are extracted through a text encoder; performing comparative learning based on vision and word text features to obtain a pre-trained vision encoder; performing word text mask prediction pre-training on the text decoder; constructing the pre-trained visual encoder and the pre-trained text decoder into a sign language recognition model so as to recognize a word text sequence from the sign language video; and accessing the pre-trained large language model in the domain into a sign language recognition model to construct a sign language translation model, and performing combined fine tuning so as to translate the word text sequence into a natural language text. The sign language recognition and translation method can realize more efficient, accurate and reliable sign language recognition and translation, is applied to the fields of intelligent sign language translation, barrier-free communication, sign language education and the like, and provides more accurate and natural language interaction experience for hearing-impaired groups.

The invention relates to the technical field of equipment control, and provides a vision-based collaborative robot end control method and related equipment thereof. Multi-modal dynamic anti-interference filtering and semantic fusion are carried out on original visual data to obtain three-dimensional semantic point cloud data, and coordinate system alignment is carried out on the three-dimensional semantic point cloud data according to a robot-based coordinate system to obtain a target pose parameter set; performing dynamic path planning on the target pose parameter set to obtain a joint space motion track, and performing inverse kinematics solution and motor instruction synthesis on the joint space motion track according to robot D-H model parameters to obtain a motor control instruction, and performing closed-loop feedback optimization on the motor control instruction according to the state data of the end effector to obtain a dynamic correction control instruction. According to the method, through full-link optimization from sensing, planning to execution, the operation precision and safety of the collaborative robot in an uncertain environment are remarkably improved.

The invention discloses a mechanical arm dynamic deviation correction method and system based on visual driving and a medium, and relates to the technical field of mechanical arm control. The method comprises the steps that when the tail end of a mechanical arm enters a preset machining space, an integrated 3D visual sensor is triggered to collect 3D point cloud of a workpiece to be machined; after pose recognition is carried out on the point cloud, the offset is recognized according to the teaching pose and the actual pose, and the initial offset is output; calling the multi-dimensional perception data, performing fusion correction, and outputting a correction offset; performing interference correction through an offset compensation model, and outputting a target offset; parameter adjustment and optimization are carried out according to the target offset, and a joint angle adjustment instruction is output; and performing correction closed-loop feedback according to the updated pose data. The technical problems of precision errors and low efficiency caused by deviation in the operation process of the mechanical arm are solved, and the technical effects that through dynamic deviation correction and multi-sensor data fusion, the operation precision and efficiency of the mechanical arm are improved, and stable operation in a complex environment is ensured are achieved.

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention relates to the technical field of traditional Chinese medicinal material defect detection, in particular to a traditional Chinese medicinal material defect detection method based on vision, which comprises the following steps: regularly acquiring a time sequence image of a traditional Chinese medicinal material sample, acquiring a multi-view image, carrying out pixel alignment on the time sequence image, and carrying out structured organization on the multi-view image according to a shooting direction to associate camera parameters; and generating a registration time sequence image sequence and a multi-view image set. According to the method, the time sequence images of the traditional Chinese medicine samples are collected regularly, pixel alignment is carried out, image offset caused by environment illumination fluctuation and equipment jitter is eliminated, and time-space consistency of dynamic variable quantity calculation is ensured. Structured organization is carried out on multi-view-angle images according to shooting directions, camera parameters are associated, a geometric constraint relation between view angles is established, and the problem that three-dimensional reconstruction precision is insufficient due to view angle isolation in a traditional method is solved.

The invention relates to the technical field of image processing, and discloses an industrial part alignment method and system based on visual analysis and a storage medium. The method comprises the steps that a three-view camera collects an industrial part image, and preprocessing is carried out through gradient magnitude local contrast enhancement to obtain an enhanced image; performing hierarchical feature extraction to identify edge contours and key control points to form a multi-dimensional feature set; and establishing a dynamic reference coordinate system based on the feature set to obtain a part space attitude matrix. And the attitude deviation is compensated through Z-axis offset and rotation coupling error analysis. Posture adjustment is decomposed into a plurality of sub-stages, an alignment track is optimized by adopting a variable speed planning strategy, and accurate alignment of the parts is achieved. The problems that multi-view visual information fusion is insufficient, a special recognition algorithm for geometrical characteristics of the industrial parts is lacked, and Z-axis offset and rotation coupling error compensation is inaccurate in the posture adjustment process are solved, and the precision and stability of alignment of the industrial parts are improved.

A system for dynamic latent space adaptation using spatiotemporal kernel context for multiscale rendering with hierarchical and Lorentzian autoencoders. The Spatiotemporal Kernel Estimator (SKE) analyzes media through motion field, temporal recurrence, frequency band, and scene semantics analyzers to generate adaptive kernel parameters encoding content-specific importance distributions. The system dynamically adapts latent manifold geometry by modifying metric tensor properties according to kernel context, enabling content-aware compression that allocates representational capacity based on visual significance. A multiscale cache implements kernel-adaptive retention policies prioritizing important regions. An adaptive renderer provides intelligent level-of-detail selection based on zoom level and kernel-estimated importance, optimizing processing allocation. The self-optimizing architecture continuously refines kernel context and geometric adaptation based on user interaction and performance feedback, achieving superior compression ratios and perceptual quality. Applications include bandwidth-efficient video streaming, virtual reality, scientific visualization, and cognitive video analytics requiring intelligent context-aware visual processing.

The invention discloses an ROV positioning navigation system based on vision inertial navigation fusion and pose correction, which comprises an inertial measurement value and image preprocessing module loaded on an ROV, the inertial measurement value and image preprocessing module is connected with a vision and inertial navigation data fusion module through a wire, and the vision and inertial navigation data fusion module is connected with a pose correction module. The invention also discloses a positioning navigation method of the ROV positioning navigation system based on visual inertial navigation fusion and pose correction. According to the method, the inertial data provided by the IMU and the environment characteristic information captured by the visual sensor are fused, high-precision estimation of the position and attitude of the ROV can be realized, the sensor data are optimally fused by using the MSCKF algorithm, the influence of the error of a single sensor on the positioning result is reduced, and the positioning precision and reliability of underwater operation are remarkably improved.

The invention relates to the technical field of safety monitoring, in particular to a smart park safety management method and system based on AI visual identification, and the method comprises the following steps: obtaining a video stream captured by a monitoring camera of a smart park, extracting the movement track data of a person through AI, and carrying out the statistics of the stay duration and access times of a target object in each grid; and constructing retention heat distribution of the grid region to obtain a personnel trajectory retention heat value. According to the invention, through the data processing of the monitoring video stream and the combination of the gridding analysis of the personnel motion trail, the accurate capture of the abnormal cruise behavior is enhanced, and through the calculation of the change rate of the access times between the adjacent grids, the statistics of the round-trip condition of the target among the plurality of areas and the combination of the passing authority data, the abnormal screening is carried out. The method improves the recognition capability of abnormal movement modes, achieves the dynamic adjustment of regional safety risk levels, improves the response capability of park security, enables the recognition capability of abnormal behaviors to be enhanced, and reduces the false alarm rate.

The invention belongs to the technical field of fire safety, and discloses a fire-fighting equipment detection early warning method and system based on a visual camera, and the method comprises the following steps: obtaining multi-focal-length image frames of a fixed spray head and a fire extinguisher edge region, judging the continuous change of gray scale in the continuous image frames, and carrying out the detection early warning of the fire-fighting equipment based on the visual camera. Comparing the gray level distribution difference under the long focus and the short focus, identifying the dynamic trajectory of the contour line angular point of the fire-fighting equipment association structure, analyzing the delay of the function action response, marking as an abnormal region, and generating a fire-fighting equipment collaborative early warning information set; according to the invention, by extracting and analyzing the multi-focal-length image frames of the edge areas of important facilities such as a fixed spray head and a fire extinguisher, fine changes of the fire-fighting facilities under differentiated focal lengths can be captured, working states and potential faults of the fire-fighting facilities can be judged, functional abnormalities caused by equipment aging or damage can be recognized in advance, early warning can be carried out in real time, and the safety of the fire-fighting facilities can be improved. And a higher guarantee level is brought to fire safety.

The invention discloses an intelligent welding forming method and system for a green building steel heating radiator, and the method comprises the following steps: carrying out the surface defect recognition of a steel heating radiator base material based on an AI visual inspection system, recognizing a qualified base material, and automatically matching the type of a welding material from a material database according to the material and thickness parameters of the qualified base material. Welding parameters are intelligently matched through AI visual inspection and a neural network model, a laser and friction stir hybrid welding process is combined, traditional manual operation is replaced, the welding efficiency and precision are improved, and the problems of uneven welding seams and the like are solved; welding data are analyzed in real time through a multi-mode AI model, parameters are dynamically adjusted, intelligent defect recognition and repair welding are achieved in cooperation with 3D visual inspection, and the quality stability is improved through whole-process monitoring; smoke dust is treated through an environment-friendly process, acid pickling is replaced with mechanical rust removal, efficient recycling of materials is achieved through waste recycling, a green manufacturing system is constructed, and the sustainable development requirement of green buildings is met.

The invention discloses an unmanned aerial vehicle navigation reasoning method and system based on visual perception and a large language model, and belongs to the field of unmanned aerial vehicle navigation and artificial intelligence, and the method comprises the steps: collecting an image of a target area through a camera carried by an unmanned aerial vehicle; pixel-level perception is carried out on the image through a pre-trained visual perception model, and visual information is converted into structured data; converting the structured data into key statements which can be processed by a large language model by adopting retrieval enhancement generation and combining with a predefined knowledge base; adopting a prompt project and thinking chain technology, and outputting a navigation instruction by the large language model according to the key statement and the current task; when a target of an unknown type is encountered, an active learning process is started, and real-time updating of an edge end knowledge base is realized through human intervention; according to the method, the problems of understanding and decision making of the unmanned aerial vehicle on a dynamic scene in a complex environment are effectively solved, data-driven intelligent support is provided for autonomous flight of the unmanned aerial vehicle, and the method has the advantages of being high in real-time performance, high in environmental adaptability and good in decision making reliability.

The invention provides a robot grabbing pose optimization method based on visual and tactile fusion, and belongs to the field of robot grabbing pose estimation. According to the method, firstly, an initial grabbing pose is predicted based on target object sparse point cloud obtained through vision initial observation segmentation, then a robot plans and executes grabbing to obtain tactile data of the surface of an object, and target object point cloud information of an area where the initial grabbing pose is located is complemented based on the tactile data with object local information; and finally, the initial grabbing pose is optimized according to the complemented point cloud, so that more stable robot grabbing is achieved.

The invention relates to the technical field of fabric detection, and discloses a fabric defect intelligent detection method and system based on AI visual identification. According to the method, motion blur is quantized through motion state data, optical blur caused by fabric motion is eliminated through deconvolution solution, so that motion interference in the fabric transmission process is processed in a targeted mode, self-adaptive balance of the deblurring capacity and the feature retention capacity is achieved, and then based on the optical interference principle, the deblurring capacity and the feature retention capacity are improved. Through a dynamic calibration system combining hardware-level real-time compensation and multi-dimensional optical parameter calibration, dynamic optical parameter calibration of primary correction data is realized, then fabric defect characterization data is extracted to accurately obtain defect features, and finally, a detection-production line control closed loop is constructed through a quality quantitative index and a comprehensive risk value, so that fabric defect detection is realized. The fabric defect detection precision can be improved, so that the problem of high defect missing detection and false detection rate caused by optical data distortion due to movement and environment interference in a traditional method is effectively solved.

The invention relates to the technical field of visual identification, and discloses a belt tearing detection method, device and equipment based on visual identification, and the method comprises the steps: carrying out the dual-channel image collection and preprocessing of the surface of a conveying belt, and obtaining a multi-channel preprocessing image; extracting a thermal difference feature and a texture structure feature of the multi-channel preprocessed image through a double-flow feature network, and inputting the thermal difference feature and the texture structure feature into a quaternion material deformation analysis model for deformation gradient tensor and invariant parameter calculation to obtain a tear feature description vector; performing hierarchical progressive identification and three-dimensional reconstruction analysis to obtain target tearing feature data; and risk assessment is carried out based on the target tear feature data to obtain a tear grade classification result, the interference of ambient temperature drift and non-uniform illumination is effectively eliminated, the calculation efficiency and accuracy of tear detection are improved, and false alarms and missing alarms are reduced.

The invention belongs to the technical field of rail transit, and discloses a rail transit environment foreign matter intrusion detection method and system based on a visual model. The method comprises the following steps: firstly, extracting a suspicious region sub-graph in an original image through preprocessing, and respectively extracting a coarse-grained global feature vector and a fine-grained local feature vector by using an image encoder; meanwhile, text prompt information matched with the image content is generated, and a corresponding text feature vector is extracted; the multi-level similarity of the image feature vector and the text feature vector is calculated, and the respective weight fusion is combined to obtain a comprehensive abnormal score; and finally, dynamically modeling abnormal score distribution based on a Gaussian mixture model to realize foreign matter invasion judgment. According to the method, image and text multi-modal information is fused, the detection precision of foreign matters in a complex scene is improved through multi-scale feature association, the dynamic environment adaptability is enhanced by adopting an adaptive threshold strategy, and the problems that a traditional method is high in false detection rate and insufficient in generalization ability in a complex background of a railway are effectively solved.

The invention relates to the technical field of visual automatic calibration, and discloses a joint manipulator automatic calibration method and device based on a visual system. The method comprises the following steps: carrying out multi-view image acquisition on a joint manipulator of the five-axis robot to obtain original image data; performing occlusion region extraction on the original image data to obtain a joint occlusion region set; on the basis of the joint occlusion region set, disordered image splicing and mark point feature extraction are carried out on the original image data, and joint feature image data are obtained; constructing an adaptive calibration equation based on the joint feature image data, and solving a target transformation relation between a global visual coordinate system and a local visual coordinate system; multi-joint collaborative calibration and joint chain constraint optimization of the five-axis robot are executed according to the target transformation relation, and joint error correction parameters are generated, high-precision joint error correction parameter calculation is achieved, and the positioning precision and the movement precision of a joint manipulator of the five-axis robot are greatly improved.

The invention discloses a water surface target detection method based on a visual language large model, which mainly solves the problem of inaccurate water surface target detection in the prior art, and the implementation scheme comprises the following steps: constructing a water surface target detection data set; extracting multi-scale features of the image by using the backbone network; constructing a dynamic feature aggregation network to fuse the multi-scale features of the image; extracting language features of the target category text; constructing a cross-modal cross attention fusion network, and fusing the visual dynamic aggregation features and the language features; utilizing a dynamic target detection head to obtain a prediction target category and a target frame; forming a water surface target detection model based on visual language multi-modal fusion by using the backbone network, the dynamic feature aggregation network, the text encoder, the cross-modal cross attention fusion network and the dynamic target detection head, and training the water surface target detection model; and obtaining a water surface target detection result by using the trained detection model. The method can effectively detect the water surface target, is high in accuracy and robustness, and can be used for intelligent ship driving.

The invention discloses a visual language model illusion suppression method based on self-adaptive dynamic attention intervention. The method is used for reducing the problem that a visual language model generates wrong associated information. The method comprises the steps that text input, visual input and historical response are acquired, and an attention accumulation vector is initialized; in the layer-by-layer calculation process of the language model, dynamically adjusting a non-normalized attention matrix, enhancing the weight of a visual sensitive attention head, and performing visual Token pruning in a deep network to optimize cross-modal information interaction; and finally, generating an output Token based on the adjusted attention mechanism, and carrying out loop iteration until a complete response is generated. According to the method, a method of combining text deviation correction through self-adaptive attention head modification and visual attention convergence-based Token pruning is adopted, so that the performance of the model in a multi-modal task is remarkably improved, and the illusion phenomenon caused by a language modal dominant reasoning process is effectively relieved.

The invention discloses a deep forgery detection method based on a visual language model, and relates to the field of image forensics. The deep forgery detection method based on the visual language model aims to combine multi-source information to improve the discrimination capability of the model on a real image and a generated image. The method comprises the following steps: firstly, extracting image features through an image encoder of a pre-trained CLIP model; meanwhile, a frequency domain enhanced counterfeit perception adapter is embedded in the image encoder to mine potential anomalies of counterfeit images in the image domain and the frequency domain. Secondly, a manual feature extraction module is provided, discriminative low-dimensional features are extracted from the four aspects of the edge, the texture, the frequency and the symmetry of the image, and the discriminative low-dimensional features are used as auxiliary information input in the forgery detection process, so that the robustness and the interpretability of the model are improved; meanwhile, the text cue words are converted into feature vectors through a text encoder of a pre-training CLIP model; and finally, the model predicts a forgery score by calculating the cosine similarity between the image features and the text features so as to realize the discrimination of the authenticity of the image. According to the method, the problem that the detection capability of the model on the cross-dataset is insufficient is effectively improved.

The invention relates to the field of computer vision and industrial safety monitoring, and discloses a pipe gallery pipeline key connection point leakage monitoring system and method based on AI vision, and the system comprises a data collection module, an intelligent analysis module, a 3D modeling module, a space-time verification module and an alarm prediction module. Real-time detection and three-dimensional accurate positioning of a leakage area are realized in combination with neural network optimization driven by physical simulation parameters; further through optical flow tracking and fluid mechanics verification, false alarm events of non-physical rules are screened out; and based on Bayesian decision and diffusion equation prediction, generating a leakage risk heat map and triggering graded alarm. The method solves the technical problems of low leakage detection precision, high false alarm rate and inaccurate positioning in a complex pipe gallery scene, has the advantages of high environmental adaptability, high anti-interference capability and prospective risk pre-judgment, and is suitable for intelligent safety monitoring of underground pipe galleries and petrochemical engineering pipelines.

The invention discloses a package defect elimination method based on visual inspection, and relates to the technical field of image processing and computer vision, and the method comprises the following steps: setting a polarized light source with a fixed angle on a package production line, and uniformly illuminating a target package area to enable the incident light polarization direction and the package surface reflection direction to form a deviation angle; an industrial camera provided with a polarization filter is used for carrying out image collection on a packaging target area, and the polarization direction of the filter is perpendicular to the polarization direction of a light source so as to restrain reflection interference to the maximum extent. Reflection interference is eliminated through polarized light, and the image quality is improved; by combining a neural network and feature fusion scoring, the defect recognition accuracy and adaptability are improved; a closed-loop elimination control and data tracing mechanism is introduced, the elimination precision is guaranteed, quality responsibility investigation is supported, and the intelligence and reliability of the packaging defect detection and elimination system are integrally improved.

The invention relates to the technical field of computer vision and industrial automation, in particular to a shoe sole three-dimensional gluing contour extraction system and method based on vision single-line laser, and aims to solve the problems that virtual calibration target spots cannot be accurately generated based on shoe sole geometry, the positions and sizes of the target spots are difficult to determine by combining curvature extreme values and principal component analysis in the prior art, and the production cost is low. The problem that a double-branch deep learning model cannot be adopted to fuse feature prediction transformation, and the re-projection error is increased is solved; a virtual calibration target spot is automatically generated based on sole geometry through a feature fusion calibration module, a grid is generated through point cloud processing and Poisson reconstruction, the position and size of the target spot are determined by combining a curvature extreme value and principal component analysis, a corresponding relation is established by utilizing two-dimensional and three-dimensional feature matching, initial alignment is realized through ICP and re-projection error optimization, and the target spot position and size are determined. A double-branch deep learning model is adopted to be fused with feature prediction transformation, iterative optimization is carried out through space consistency errors, and re-projection errors are reduced.

The invention relates to the technical field of machine vision and detection, in particular to a shoe pattern defect detection system based on the visual technology, which comprises an acquisition processing unit, a three-dimensional model unit, a defect detection unit and a defect processing unit. The three-dimensional model unit reconstructs an accurate shoe pattern three-dimensional model in sequence, the defect detection unit divides the model into a plurality of areas, calculates the average curvature, and accurately analyzes and judges the flatness defect degree by adopting a plane fitting algorithm based on a least square method and a curvature difference algorithm based on a sliding window; and the defect processing unit automatically processes the shoes with serious defects, so that high efficiency, accuracy and automation of shoe pattern defect detection are realized, and the quality detection level of shoe production is powerfully improved.

The invention belongs to the technical field of mechanical arm grabbing, and discloses a visual sense-force sense-based mechanical arm instruction grabbing method, which comprises the following steps of: acquiring an RGB-D image of a scene, performing cutting processing on the RGB-D image through OpenCV, and extracting visual features of each object in the RGB-D image; a text command input by a user is coded through BERT, and language features are output; inputting the language features and the visual features into a CLIP model for fusion and alignment, and performing further alignment by adopting a Co-TRM structure to generate bimodal features; generating a point cloud image according to the depth map of the RGB-D image, and based on the point cloud image, combining visual features of an object, and generating a grabbing attitude through Graspnet; the problem that in the prior art, when grabbing is conducted based on vision and instructions, self-adaptive adjustment on grabbing force is not convenient is solved.