5793 results about "Depth map" patented technology

The invention relates to the field of defect detection, in particular to a PCB defect detection method and system based on image recognition. The method comprises the following steps: collecting a multidirectional PCB detection image, carrying out pixel-level registration correction and adaptive pixel stability compensation, and constructing a space-time stability compensation image sequence; performing reverse pyramid structure division on the space-time stability compensation image sequence, performing normalized similarity probability calculation, and constructing an initial region classification result; based on an initial region classification result, depth image visual analysis is carried out, pseudo defect comprehensive elimination optimization is carried out, and a pseudo defect purification high-confidence image is constructed; pCB connection defect identification is carried out on the pseudo defect purification high-confidence image, global defect point distribution marking is carried out, and a defect point space distribution diagram is constructed. According to the invention, high-credibility, high-precision and high-closed-loop PCB defect detection is realized.

The invention discloses a three-dimensional Gaussian sputtering scene reconstruction method based on structure perception refined Gaussian, and aims to solve the problems of Gaussian drift, edge blur, structure artifacts and the like of a reconstruction model due to the fact that sparse point cloud contains outliers, Gaussian morphology and normal are mismatched and a multi-dimensional optimization target is lacked in an existing three-dimensional Gaussian sputtering reconstruction method. A key frame is extracted by collecting target scene video data, sparse three-dimensional point clouds are reconstructed by using an SfM algorithm, a depth map and a normal map are generated through a Lotus model, three-dimensional Gaussian distribution is initialized after the sparse point clouds are filtered, a Gaussian covariance matrix is adjusted by using a normal consistency regular term, and the sparse point clouds are extracted. And after structure attribute analysis is carried out, a comprehensive scoring function is constructed to screen Gaussian points, and finally, a combined training framework including luminosity, normal consistency and structure continuity loss is adopted to optimize and generate a three-dimensional Gaussian scene model. The method is mainly applied to the field of three-dimensional reconstruction and multi-view rendering, and scene reconstruction precision and geometric consistency can be improved.

The invention discloses a multi-view panoramic point cloud splicing method, which comprises the steps of performing joint calibration on a binocular camera and a laser radar through a calibration algorithm, and aligning a coordinate system; a binocular camera is used for collecting a two-dimensional image and carrying out distortion correction, a depth map is generated based on parallax calculation, and three-dimensional point cloud reconstruction is carried out in combination with a triangulation principle; performing deep learning target detection on the two-dimensional image, and screening a line rod assembly area through non-maximum suppression; multi-view point cloud data are collected through a laser radar, point cloud segmentation processing is carried out, and a telegraph pole assembly point cloud subset is reserved; matching the two-dimensional pixel area of the telegraph pole assembly with a laser radar point cloud projection result, and screening point cloud data belonging to the telegraph pole assembly; and carrying out alignment and fusion on the multi-view point clouds through initial registration and fine registration by adopting an improved point cloud splicing algorithm to generate a panoramic point cloud image. According to the method, the point cloud splicing processing time is shortened, and the robustness, precision and integrity of point cloud splicing are improved.

The invention relates to an optimization method and device for sparse view angle three-dimensional Gaussian splash, and belongs to the technical field of three-dimensional reconstruction in computer vision, and the method comprises the steps: collecting a sparse view angle image; a multi-view stereoscopic vision model based on deep learning generates a geometrically consistent depth map for the sparse view image, converts the depth map into point clouds and fuses the point clouds to obtain dense point clouds; sampling dense point clouds by adopting voxel-guided farthest point sampling to obtain initialized point clouds, and constructing a three-dimensional Gaussian field; rendering the three-dimensional Gaussian field through an enhanced geometric renderer to obtain a rendering depth and a rendering normal; constructing a multi-level geometric regularization loss function, and optimizing the three-dimensional Gaussian field; and performing optimization adjustment on the three-dimensional Gaussian field based on a shape-scale constraint criterion and a two-stage adaptive opacity constraint strategy to obtain an optimized three-dimensional Gaussian field. According to the method, the problems of initialization failure, insufficient geometric supervision and element out-of-control of 3D Gaussian splashing under the sparse view angle are solved.

The invention discloses a robot grabbing posture generation method and a related device, and the method comprises the steps: obtaining an RGB image, a depth image and a position coordinate of a target object, and generating a spatial calibration matrix through data preprocessing; target object segmentation is carried out on the calibrated RGB matrix according to the target point coordinate sequence, a segmentation mask is generated, and object mass center coordinates are calculated; generating a three-dimensional point cloud by using the calibrated depth matrix, the segmentation mask and the camera parameters, extracting a plane point set and a non-plane point set through an RANSAC algorithm, and analyzing to obtain an axial feature vector and a plane normal vector; finally, grabbing parameters are calculated according to the feature vectors and the centroid coordinates, and a grabbing posture transformation matrix is generated through vector operation. According to the technical scheme, under the condition of not depending on a preset model library, the grabbing posture of an unknown object is generated by analyzing the geometrical characteristics of the object, the accuracy problem in the process of converting two-dimensional image information into three-dimensional grabbing parameters is solved, and the adaptability and reliability of a robot grabbing task are improved.

The invention relates to the technical field of multi-target tracking, in particular to a dynamic multi-target tracking and trajectory prediction method. Comprising the following steps: acquiring a color image and a depth image, and preprocessing; constructing a dynamic threshold double-flow neural network, extracting features from the preprocessed color image and depth image to obtain a visual feature set and a depth feature set, fusing the visual feature set and the depth feature set through an adaptive attention mechanism to obtain a fused feature set, and combining a dynamic threshold to obtain an observation information list of a target; updating the observation information list through a self-motion decoupling mechanism to obtain a real information list; establishing space association and time association between the target and the track based on the real information list to obtain a newest track set; and analyzing target characteristics, environmental constraints, social behaviors and historical trajectories, generating a prediction trajectory of each target and a corresponding multi-factor score, generating a time-varying influence graph, and performing planning and decision making by adopting a double-layer decision-making mechanism.

An aero-engine augmented reality virtual-real fusion method based on depth prior scene reconstruction comprises the following steps: firstly, collecting data of an aero-engine look-around RGB image and a depth image, calculating a camera frame pose by using an SFM algorithm and obtaining a sparse point cloud, and optimizing the camera pose by using an ICP algorithm and the point cloud converted from the depth image to obtain an aligned prior dense point cloud; gaussian points are initialized based on the priori dense point cloud and the sparse point cloud, and random generation of 3D Gaussian is limited by using the boundary of the dense point cloud as geometric constraint; through the three-dimensional reconstruction of the adjacent multi-view enhancement strategy, the spatial and visual consistency is improved, and an aero-engine model is reconstructed; then understanding the reconstructed aero-engine model based on semantic segmentation, identifying and marking core components, and finally performing virtual-real fusion visualization on the reconstructed aero-engine through augmented reality to guide training, assembling and maintenance operations. According to the invention, rapid high-quality three-dimensional reconstruction and augmented reality virtual-real fusion visualization of the aero-engine are realized.

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention relates to a roadside signboard identification method based on unmanned aerial vehicle three-dimensional point cloud lightweight semantic segmentation, and belongs to the technical field of intelligent traffic. The method comprises the following steps: optimizing a point cloud acquisition path through adaptive flight control; adopting an RSPAE algorithm to enhance local geometric features of the point cloud; converting the point cloud into a three-channel fusion image (a depth image, an intensity image and a local depth variance image); extracting multi-scale features by using a double-branch neural network, and fusing the aligned features through a GFM module and a CFM module; the lightweight decoder recovers a high-precision semantic segmentation map; and generating a final identification result by combining geographical registration and multi-frame redundancy suppression. And state evaluation and anomaly detection are realized based on an MLP scoring device and a mahalanobis distance. According to the method, the segmentation precision, the reasoning speed and the positioning precision are remarkably improved in a complex scene, the method is suitable for deployment of embedded equipment, and the problems of low efficiency and high omission ratio in the prior art are solved.

The invention provides a posture recognition algorithm for any object under a monocular camera and an application system, and the algorithm comprises the steps: S1, constructing a target three-dimensional model, carrying out the multi-view annular shooting image collection of a target, and generating a dense grid model through feature extraction, matching, posture calculation and a multi-view geometric method; s2, generating an image depth map, and predicting depth information of a target in a motion process based on a monocular image sequence; s3, extracting a target image mask, and generating a target area mask graph through an image encoder, a prompt encoder and a mask decoder; and S4, executing attitude estimation, performing attitude initialization, correction and screening by combining the three-dimensional model, the depth map and the mask map, and outputting a six-degree-of-freedom attitude result of the target. According to the method, the target is subjected to annular shooting modeling through the method based on multi-view geometry, the three-dimensional model of the target is generated, attitude estimation is achieved in combination with the image mask and the depth map, the generalization ability of an attitude estimation algorithm in an actual scene is improved, and the application range of the attitude estimation algorithm in the actual scene is widened.

The invention discloses a robot dynamic environment adaptive sensing and navigation system based on a three-dimensional laser radar, relates to the technical field of robots, and solves the technical problems that comprehensive environment information is difficult to obtain, and the weight is difficult to adjust by fusing weather types and sensor confidence coefficients. Comprising the following steps: generating original point cloud data by combining a bionic compound eye type laser radar with a silicon photon integrated chip; marking point clouds based on a KITTI data set, performing preprocessing, training a Transform model to output a dynamic obstacle mask, and filtering background point clouds; a weather detection model is constructed, the weight is dynamically adjusted according to the weather type and the sensor confidence coefficient, and position and attitude estimation is fused; laser radar point cloud constructs a geometric map, a camera depth map generates a dense map, and semantic tags are mapped to generate an environmental semantic map; and converting the environmental semantic map into a three-dimensional grid map, and planning an obstacle avoidance path based on the grid map by using an A * algorithm.

The invention belongs to the technical field of three-dimensional imaging and measurement, and particularly discloses a laser scanning three-dimensional imaging method and device, a medium and equipment, and the method comprises the steps: obtaining laser scanning data of a target object, and the laser scanning data comprises depth data including reflection intensity, a scanning angle, a timestamp and flight time; preprocessing the laser scanning data, and fusing a time sequence and scanning path information to obtain multi-channel input features adaptive to a neural network; constructing a three-dimensional geometric reconstruction model, and training the model; inputting the multi-channel input features into a trained three-dimensional geometric reconstruction model to obtain a dual-channel output tensor containing a dense depth map of the target object and a corresponding uncertainty map; converting the dual-channel output tensor comprising the dense depth map of the target object and the corresponding uncertainty map into a three-dimensional point cloud under world coordinates; and post-processing the three-dimensional point cloud to obtain a three-dimensional image of the target object.

The invention belongs to the technical field of image analysis, and discloses an old people health state prediction and alarm method based on AI image analysis, and the method comprises the steps: obtaining a depth image, a color image and a pressure image of an activity environment of old people, carrying out the combined analysis of the depth image and the pressure image, recognizing a terrain semantic region in a monitoring region, and carrying out the prediction of the health state of the old people; and carrying out dynamic segmentation on the gait contour of the old people. And the terrain gait coupling degree is calculated by analyzing the depth change characteristics and the motion period characteristics of the gait sub-regions, so that the abnormal gait sub-regions are screened out. And for the abnormal regions, the terrain gait coupling degree, the dynamic texture features of the color image and the pressure distribution features of the pressure image are fused, and a cross-modal health risk vector is constructed. Based on the vector, the health state of the old people is accurately predicted, and a corresponding alarm signal is generated. According to the invention, precise evaluation and timely early warning of the health state of the old people are realized, and intelligent guarantee is provided for the home safety of the old people.

The invention provides an interactive robot grabbing method based on a large language model, and the method comprises the steps: obtaining a task operation sequence according to a two-dimensional image of a current scene, dialogue interaction information and the large language model, the task operation sequence comprises a target object, component information of the target object, target grabbing position information and action information; obtaining a target mask area according to the two-dimensional image and the target capture position information in combination with an image segmentation model, and obtaining a three-dimensional point cloud image with a target capture position according to the target mask area and the depth map of the current scene; and according to the action information, the three-dimensional point cloud picture with the target grabbing position and a grabbing prediction model, a target grabbing pose is obtained, and according to the target grabbing pose, a robot is controlled to grab the target object. According to the method, the analysis capability of the robot on the complex semantic instruction is improved, and the adaptability of the robot in a refined operation task is also improved.

The invention discloses a self-adaptive three-dimensional scene reconstruction method and system based on a single panorama, and belongs to the field of computer vision processing. The method comprises the following steps: firstly, generating a depth map through an indoor scene panorama and constructing an initial three-dimensional grid; then generating a multi-view image and a shielding mask thereof based on view conversion, forming a training sample pair for fine tuning of the diffusion completion model, and injecting scene prior information for the model; then extracting a grid boundary contour and calculating a central axis, and adaptively constructing a camera pose set; after the integrity of the grid is optimized through iteration completion, the grid is converted into a 3D Gaussian sputtering field, and Gaussian point parameters are optimized through multi-view data; in the optimization process, an up-sampling strategy of rendering error feedback and edge Gaussian points is adopted, and finally a scene model with complete geometric textures is output. According to the method, a high-quality three-dimensional scene is reconstructed from a single panorama through a structure self-adaptive completion and Gaussian refining technology, and the method is particularly suitable for immersive roaming reconstruction of indoor scenes.

A dynamic adaptive depth camera occlusion detection method, system and device, and a storage medium, the method comprising: step T1: constructing a three-frame time sequence analysis window, continuously obtaining infrared speckle image sequences at different moments and corresponding dense depth map sequences, and establishing a pixel-level space-time alignment mapping relationship; t2, calculating a motion vector between adjacent frames, and executing the step T3 when the average motion intensity exceeds a dynamic threshold value; wherein the dynamic threshold is determined by the dense depth map; t3, positioning a speckle feature point in the infrared speckle image, generating an analysis window by taking the speckle feature point as a center, calculating a gradient change rate, and when the gradient change rate is smaller than a gradient threshold value, marking the region as a primary candidate region; and T4, carrying out space-time consistency comparison on the primary candidate region and two adjacent frames, and reserving a shielding region of which the variation value is smaller than a variation threshold value in three continuous frames. The method is high in precision, good in real-time performance and high in stability.

The invention provides a mountainous area tunnel construction safety intelligent monitoring and early warning method and system, and relates to the technical field of construction safety monitoring, and the method comprises the steps: collecting visible light and depth images of tunnel surrounding rock, and carrying out the segmentation and extraction of crack features through a depth attention network after image preprocessing and data fusion; extracting parameter time sequence data based on the crack spatial position and the type feature; determining fracture evolution characteristics and critical state parameters by combining wavelet transform and stress-rate coupling analysis; and adopting deep reinforcement learning to calculate the instability probability and generate early warning information. According to the invention, intelligent identification, instability prediction and risk early warning of tunnel surrounding rock cracks are realized, and the safety monitoring accuracy and early warning timeliness are improved.

The invention provides a multi-view three-dimensional point cloud reconstruction method and device based on DPE-SE depth estimation, and relates to the technical field of computer vision and three-dimensional reconstruction. The method comprises the following steps: acquiring multi-view image data; preprocessing the image; inputting the preprocessed image into a DPE-SE-based depth estimation model, carrying out key constraint on an edge region through a semantic edge guiding mechanism, carrying out adaptive propagation updating on a weak texture region, realizing accurate depth estimation, and generating a multi-view depth result; then geometric consistency check and multi-scale depth fusion are performed on a multi-view depth result, and a dense depth map is constructed; and finally, performing three-dimensional back projection reconstruction and point cloud optimization processing, and outputting high-quality point cloud data containing three-dimensional coordinates and confidence information. According to the method, the problems of edge mismatching and depth voids are remarkably improved in complex illumination, weak texture and shielding environments, the continuity and structural integrity of the point cloud are improved, and technical support is provided for unmanned aerial vehicle surveying and mapping, building detection and digital twin modeling.

Provided is a defect detection method and device, computer equipment and a storage medium. The method includes: acquiring an RGB image, a depth image and a sample label of a detection object sample; performing feature map extraction and feature map fusion on the RGB image and the depth image by a feature extraction network of the defect detection model, to obtain a fused feature map; performing defect detection based on the fused feature map by a feature reconstruction network of the defect detection model, to obtain a defect score map, wherein the defect score map being obtained by fusing a global defect score map which is generated based on a global defect detection network with a local defect score map which is generated by a local defect detection network; and updating parameters of the defect detection model based on the defect score map and the sample label.

The invention belongs to the technical field of image processing and robots, and particularly relates to an image processing method and device and a robot, and the method comprises the steps: obtaining a structured light coding image, a polarization imaging image and a binocular depth image; constructing a multi-dimensional feature vector by fusing decoded contour phase, polarization analysis material characteristics and depth image data; dynamically calibrating the multi-dimensional feature vector according to the ambient light and the pose of the robot, correcting color deviation through an adaptive white balance algorithm, compensating a camera installation error, and obtaining a calibrated descriptor; based on the calibrated descriptor, utilizing an improved Hough transformation and template matching algorithm to identify the object category attitude, and calculating six-degree-of-freedom pose parameters through a triangulation method; and according to the six-degree-of-freedom pose parameters, a collision-free grabbing path is generated in combination with a robot kinematics model and obstacle information. Therefore, the problems of insufficient image analysis capability, single compensation mechanism, lack of path optimization capability and the like in the prior art are solved.

The invention discloses a three-dimensional scene synthesis method fusing depth estimation and a double-view video, and relates to the technical field of image fusion. The invention discloses three-dimensional scene synthesis integrating depth estimation and a double-view video. Comprising the following steps of: acquiring a left image sequence and a right image sequence which are synchronous; generating an initial depth map through a depth estimation model; generating a mask of a sheltered region through region sheltering detection; positioning the sheltered region in an image of a current frame; and performing weighted fusion on the updated depth map through the depth confidence map, and outputting a three-dimensional scene of the current frame through the fused depth map. According to the method, a shielding detection mechanism and a shielding area mask are constructed, space alignment is carried out on a fusion depth map of a previous frame and an image of a current frame, and depth completion of a shielding area is realized in combination with a residual fusion strategy.

The invention relates to the technical field of image recognition, and discloses a flame-retardant material surface defect image recognition method, device and equipment and a storage medium, and the method comprises the steps: carrying out the multi-angle image collection and preprocessing of a flame-retardant material combustion test sample, and obtaining a standardized multi-view image data set; performing multi-model feature extraction and feature matching processing to obtain a material defect representation vector set and a defect semantic feature set; constructing a self-adaptive connection structure and a hierarchical defect map; performing multi-layer information transmission and topological relation explicit modeling through a depth map neural network to obtain a defect node depth representation set and a material defect relation matrix; domain invariant feature extraction and structural consistency constraint are carried out, material-independent defect type distribution and defect severity quantification results are obtained, defect features of all angles of the surface of the flame-retardant material can be comprehensively captured, remote interaction characteristics between defects are extracted, and the discrimination capability of defect feature representation is enhanced.

The invention discloses a three-dimensional scene reconstruction method and device based on large model geometric prior, and a medium, and aims to solve the problems that a conventional 3DGS is liable to have artifacts and detail loss in geometric discontinuity, data redundancy and illumination variation scenes, and predicts a dense depth map and a normal map from a monocular image by using a pre-trained large model. The position and form of the Gaussian kernel are constrained as additional geometric priori; a primitive adjustment strategy based on kernel density estimation is introduced in the training stage, small Gaussian primitives with similar structures and adjacent spaces are combined into a large Gaussian primitive, the rendering quality is kept, redundancy is reduced, and the volume of the model is reduced; an exposure coefficient is adaptively estimated for each input image, an exposure compensation image loss function is constructed, and floating artifacts caused by illumination differences at shooting moments are eliminated. Experiments show that compared with the prior art, the method improves the three-dimensional reconstruction precision and real-time rendering quality of complex illumination and less-texture areas in a public data set and an unmanned aerial vehicle aerial photography scene.

The invention discloses a crop growth analysis system based on machine vision, and relates to the technical field of agricultural intelligent monitoring and analysis, the system comprises a multi-source acquisition module, a disturbance identification module, a growth behavior modeling module, an environmental adaptability correction module and a feedback prediction module; wherein the multi-source acquisition module can synchronously acquire visible light, near-infrared and depth images, and generates a joint index in combination with environmental data; the disturbance identification module identifies abnormal areas such as wind disturbance and shielding based on the residual image and the image structure network; the growth behavior modeling module is used for extracting internode change, leaf surface tension and bifurcation angle characteristics by using a three-dimensional structure time sequence alignment and hidden change rate coding network; the environment adaptability correction module constructs a crop-environment double-domain mapping relation; the feedback prediction module outputs personalized intervention suggestions in combination with deviation analysis and crop variety embedding vectors; the system has the advantages of high modeling precision, strong adaptability, fast feedback response and the like, and is suitable for intelligent planting management of various crops.

The invention relates to an aircraft surface damage area segmentation method based on adaptive multi-modal fusion, belongs to the technical field of nondestructive testing, and solves the technical problem of low segmentation precision of an aircraft skin curved surface falling damage area. Obtaining a grey-scale map, a depth map and a point cloud map of the aircraft surface damage skin; synthesizing the grey-scale map, the depth map and the point cloud map, and performing preprocessing, feature extraction, feature conversion, image display and projection and weighted fusion on the synthesized grey-scale map, the synthesized depth map and the synthesized point cloud map to obtain a total edge probability map of skin damage; and performing targeted screening, continuity detection, connectivity detection, convex hull construction, marking and skin falling damage area division on the total edge probability graph. The method is used for aircraft skin health monitoring, maintenance guarantee and operation and maintenance.

The invention discloses an operation and maintenance manipulator intelligent control method and system based on visual identification, and relates to the technical field of intelligent manipulator control, and the method comprises the steps: collecting RGB image data and depth image data of an operation and maintenance operation area, and obtaining a standardized image matrix and a mapping relation matrix; inputting the standardized image matrix into an improved ResNet residual network model, generating a comprehensive feature descriptor, and calculating a spatial position coordinate and an attitude angle of the target equipment based on the mapping relation matrix; based on the current joint angle state of the manipulator, an improved Jacobian matrix inverse kinematics algorithm is used for solving a target angle sequence of each joint, a preset operation mode library is matched based on the comprehensive feature descriptor, and a grabbing force parameter and a motion speed parameter are determined; and converting the target angle sequence into a control instruction, and sending the control instruction to each joint driver of the manipulator to drive the manipulator to complete action planning. According to the invention, full-process automation from environment perception to task execution is realized.

The invention provides a coal quantity detection method and device based on laser radar and vision fusion, and belongs to the technical field of coal quantity detection of underground coal mine conveying belts. According to the technical scheme, the coal quantity detection method based on laser radar and vision fusion comprises the following steps: S1, sensor joint calibration; s2, data acquisition; s3, identifying a coal quantity area; s4, complementing the depth image; and S5, calculating the coal quantity. The invention further provides a coal quantity detection device which comprises a speed sensor, a laser radar, an industrial camera, a computer and a human-computer interface. The method has the beneficial effects that through multi-sensor data fusion and a deep learning algorithm, underground complex environment interference is overcome, and high-precision real-time detection of the coal quantity is realized; the detection device has the explosion-proof and dust-resistant characteristics, and the detection method comprises environment self-adaptive correction and lightweight calculation, and can be widely applied to intelligent mine transportation systems.

The invention discloses a double-station robot sorting optimization method and system based on digital twinning and a terminal, double improvement of sorting efficiency and safety is achieved by constructing a digital twinning driven collaborative operation system, and the method comprises the steps that firstly, a laser radar and a polarization camera are used for forming a composite sensing unit; geometric morphology, material reflection characteristics and spatial pose data of a target object are synchronously obtained, and are input into a digital twin engine after time synchronization processing; an engine constructs a virtual sorting scene containing a material attribute database based on a physical rendering technology, sub-millimeter-level space registration is achieved through feature fusion of a binocular stereoscopic vision depth map and geometric parameters, high-precision digital twin stations are generated, a system plans a space-time constraint trajectory of a double-station robot in a virtual environment, and a target object is obtained. And an integrated discrete event simulation engine performs operation time sequence conflict prediction, and when a space overlapping risk is detected, an alternative path containing a dynamic obstacle avoidance strategy is generated through a trajectory re-planning algorithm.

The invention relates to the technical field of augmented reality and industrial safety monitoring, in particular to an AR glasses dangerous state dynamic labeling method and system based on multi-source perception, and the method comprises the steps: synchronously collecting data such as a depth image and gas concentration through multi-source equipment such as a depth camera and a gas sensor; extracting rotation invariant point cloud features through an improved dangerous target detection model, and fusing multi-modal data to generate a dynamic danger level; based on VIO, realizing accurate spatial registration of annotation and a real scene, and generating a self-adaptive annotation style according to a danger level; and multi-device labeling pose synchronization is realized through an entropy weighting ICP algorithm, and labeling transparency and size are dynamically optimized in combination with a user fatigue value and environment illumination. According to the method, the comprehensiveness of danger identification and the accuracy of marking are improved, team collaborative operation is supported, the method is suitable for scenes such as electric power inspection and dangerous chemical transportation, and the operation safety and efficiency are guaranteed.

Disclosed in the present invention is a fruit stem pose recognition method used for a pomelo fruit picking apparatus, the method comprising the following steps: preprocessing image data of a scene to be recognized, so as to obtain preprocessed image data, the image data comprising an RGB map and a depth map; inputting the preprocessed image data into an optimized instance segmentation model to obtain a depth map of a single pomelo fruit; on the basis of the depth map of a single pomelo fruit, recognizing an elliptical fruit pose feature region, and further using the Hough ellipse detection algorithm to obtain an ellipse tilt angle of the fruit pose feature region; and, by means of the ellipse tilt angle of the fruit pose feature region, calculating and obtaining the position and orientation of a pomelo fruit stem. The method integrates a plurality of advanced technologies to achieve optimization of a whole process from image data preprocessing to fruit stem pose accurate recognition, thus improving the recognition accuracy and working efficiency of automated pomelo fruit picking apparatuses, and reducing the mis-picking rate and the fruit damage rate.