Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

10577 results about "Image frame" patented technology

Skeleton detection and fall detection method based on improved spatio-temporal adaptive graph convolution

Disclosed in the present application is a skeleton detection and fall detection method based on improved spatio-temporal adaptive graph convolution. The method comprises the steps of: S1, collecting image data to acquire data of each image frame; S2: using a pre-trained yolov5 target person detection model to detect whether a target person is present in the data of each image frame, and if a target person is present, turning to step S3, and if no target person is present, ending the process; S3: for each detected target person, using a Deepsort target tracking algorithm to perform target tracking to obtain a tracking result, calculating the similarity to obtain the result of target association, and updating trajectory information of each target person; and S4: performing pose recognition on each target person on the basis of the trajectory information, using a spatio-temporal adaptive graph convolutional network to extract a feature vector of a pose, and using a classifier to perform human body behavior classification and recognition, in order to determine whether the target person has experienced a fall incident. The method achieves higher accuracy and robustness.
Owner:NANJING HOWSO TECH

Building construction monitoring method

The invention provides a building construction monitoring method, which comprises the steps of obtaining multi-view video image data of a construction site, and performing fusion processing on the multi-view video image data by adopting an image splicing algorithm to obtain a panoramic three-dimensional scene model of the construction site; acquiring an image frame in the panoramic three-dimensional scene model, and analyzing the image frame by adopting a pre-trained work type identification model to obtain position information of operating personnel and equipment of each work type; acquiring position information of the operating personnel of each type of work, and analyzing postures of the operating personnel by adopting a pre-trained human body posture estimation model to obtain action state information of the operating personnel; and acquiring the position information and the action state information of the operating personnel of each type of work, and judging the specific operation state of the operating personnel of each type of work in combination with a pre-established work type operation knowledge base.
Owner:GUANGDONG DEHONG CONSTR ENG CO LTD

Crop whole growth cycle identification method and system based on deep learning

The invention provides a crop whole growth cycle identification method and system based on deep learning, and the method comprises the steps: obtaining a multispectral image sequence of a target crop in a continuous time period through an image collection device, and carrying out the standardized illumination adjustment processing of each image frame in the multispectral image sequence, obtaining a standard illumination image set corresponding to the multispectral image sequence; carrying out crop region segmentation processing on each image, extracting a local feature region related to a target crop, generating a standardized crop image set containing the local feature region, inputting the standardized crop image set into a pre-trained multi-task deep learning model, extracting combined features through parallel convolution branches, and carrying out image segmentation processing on the combined features; and executing cross-stage correlation analysis in the full connection layer, outputting a multi-task classification result corresponding to the target crop growth cycle, and generating a stage identification report corresponding to the target crop full growth cycle. According to the invention, the crop whole growth cycle identification precision and the agricultural management efficiency can be improved.
Owner:HUAYUNSHENGDA(BEIJING)METEROLOGICAL TECH CO LTD

Low power proximity-based presence detection using optical flow

In various examples, low power proximity based threat detection using optical flow for vehicle systems and applications are provided. Some embodiments may use a tiered framework that uses sensor fusion techniques to detect and track the movement of a threat candidate, and perform a threat classification and / or intent prediction as the threat candidate approaches approach. Relative depth indications from optical flow, computed using data from image sensors, can be used to initially segment and track a moving object over a sequence of image frames. Additional sensors and processing may be brought online when a moving object becomes close enough to be considered a higher risk threat candidate. A threat response system may generate a risk score based on a predicted intent of a threat candidate, and when the risk score exceeds a certain threshold, then the threat response system may respond accordingly based on the threat classification and / or risk score.
Owner:NVIDIA CORP

Digital human real-time interaction system and digital human real-time interaction method

The invention discloses a digital human real-time interaction system and method. The interaction system comprises terminal equipment and a cloud server. The terminal device receives voice input by a user and transmits the voice to the cloud server. The cloud server comprises a video acquisition and processing module, an intelligent question and answer and voice generation module, an audio processing module and an image rendering module. A video capture and processing module captures original video data indicative of an object of a digital figure and generates successive image frames. The intelligent question and answer and voice generation module converts the voice into a text; performing semantic analysis on the text, understanding the intention of the user, and generating a corresponding reply text; the reply text is also converted into a reply voice. The audio processing module processes the reply voice and the continuous image frames to generate first video data. And the image rendering module fuses the first video data with the virtual background, and performs real-time processing on the fused video to generate a response video. And the terminal equipment outputs the response video.
Owner:BEIJING HONGLIAN 95 INFORMATION IND CO LTD +1

Bridge bolt monitoring image recognition method and system based on deep learning

The embodiment of the invention discloses a bridge bolt monitoring image recognition method and system based on deep learning, and belongs to the technical field of bridge image processing. The method comprises the following steps: acquiring a multi-angle monitoring image set of a bridge bolt part through an image acquisition device, and preprocessing the monitoring image set to generate a standardized target image sequence; positioning a bolt monitoring area for each image frame in the target image sequence based on a pre-trained positioning model, and generating an area coordinate set containing bolt position information; inputting the region coordinate set into a state recognition model, extracting structural state features of each bolt monitoring region, and generating a looseness degree prediction result of the bolt part according to the structural state features; according to a matching result of the looseness degree prediction result and a preset safety threshold range, a bolt state abnormity alarm signal is output; and associating the bolt state abnormity alarm signal to the corresponding monitoring image frame, and generating a monitoring report containing a visual mark.
Owner:GUIZHOU QIANTONG ENG TECH CO LTD

Unmanned aerial vehicle area boundary automatic identification and obstacle avoidance method and device

The invention relates to an unmanned aerial vehicle area boundary automatic identification and obstacle avoidance method and device, and the method comprises the steps: obtaining environment perception data of an unmanned aerial vehicle sensor, carrying out the type division of the environment perception data, and obtaining corresponding image data, point cloud data and continuous image frames; performing boundary recognition processing on the image data to obtain a corresponding boundary recognition result, and performing obstacle feature extraction on the point cloud data to obtain corresponding obstacle features; performing long-range dependency feature prediction on the continuous image frames through a preset Transform model to obtain corresponding motion prediction information; performing obstacle avoidance path planning on the three-dimensional environment map according to the dynamic boundary information to obtain a corresponding obstacle avoidance path; and performing decision analysis on the obstacle avoidance path, the dynamic boundary information and a preset task target to generate an automatic obstacle avoidance decision. According to the invention, environment information can be updated in real time and a corresponding obstacle avoidance decision can be made.
Owner:SHENZHEN TIANYING BROTHERS UAV INNOVATION CO LTD

Self-adaptive camera pose intelligent adjusting system

The invention relates to the technical field of intelligent control, in particular to a self-adaptive camera pose intelligent adjusting system which comprises a dynamic recognition module, a pose adjusting module, a mechanism control module, an information filtering module and a feedback association module. According to the method, the foreground pixel brightness gradient and the boundary continuity in the image frame are extracted, the target displacement path can be recognized, the tracking information is constructed, the target coherent perception under rapid scene change is guaranteed, the attitude parameters and the target motion direction and speed are combined, the offset correction instruction is generated, the response stability of the dynamic target is improved, and the target tracking accuracy is improved. The execution structure feedback is used for correcting control output, reducing the driving error accumulation risk, eliminating non-target disturbance through image boundary definition and motion consistency, improving data purity and combining a view angle execution state and path information to realize synchronous control of tracking and view angle adjustment; and the response efficiency and the following robustness of the system in a dynamic environment are enhanced.
Owner:SHENZHEN STARCAM TECH

Systems and methods for object detection of unmanned aerial vehicles

A system to detect, classify and track small objects in an image frame. The system includes an image data system and image analytic system. The image data system receives real-time image data from an image capturing system and inputs the image data to image analytic system. The image analytic system performs the object detection, classification and tracking of objects in an image frame using a combination of techniques.
Owner:EPIRUS INC

Emotion recognition method based on visual language pre-training and multi-modal collaborative fusion

The invention discloses an emotion recognition method based on visual language pre-training and multi-modal collaborative fusion, and belongs to the technical field of multi-modal emotion recognition. According to the method, a visual encoder is pre-trained by using a large-scale visual-text pair data set through a visual language pre-training normal form, so that the visual feature extraction capability of the visual encoder is improved; image frames, skeleton nodes and audios are extracted from the video and are respectively conveyed into encoders of various modes, and features of the multiple modes are extracted; the skeleton node features and the features extracted from the video frames are fused through a cross attention mechanism, so that visual features are further enhanced through information such as motion postures of people; finally, through a learnable query vector and a cross attention mechanism, the visual features and the audio features are fused into the query vector; in the prediction stage, sentiment classification is carried out by using query vector features obtained by learning.
Owner:TIANJIN UNIV

Park safety monitoring method and system based on dynamic visual identification

The invention relates to the technical field of monitoring analysis, in particular to a park safety monitoring method and system based on dynamic visual identification. The method comprises the following steps: acquiring a plurality of park monitoring videos based on park monitoring equipment; performing three-dimensional illumination migration rendering on the plurality of park monitoring videos, and performing low-frame smooth completion so as to generate a time sequence monitoring image frame sequence; carrying out spatial position positioning calculation on the park monitoring equipment, carrying out topological association network reconstruction, and constructing an omnibearing monitoring perception topological map; performing dynamic object visual detection on the time sequence monitoring image frame sequence, and performing frame-by-frame optical flow tracking analysis to generate a dynamic target optical flow trajectory; and carrying out multi-time-point sampling on the dynamic target optical flow trajectory, and then carrying out mutation point identification so as to generate an optical flow trajectory mutation point. According to the invention, efficient park safety monitoring analysis and high-precision safety early warning are realized.
Owner:珠海九洲现代服务有限公司

Video and Audio Multimodal Searching System

A multimodal search system using a video query is described. The system can receive video data captured by a camera of a user device. The video data can have a sequence of image frames. Additionally, the system can receive audio data associated with the video data captured by the user device. Moreover, the system can process, using one or more machine-learned models, the sequence of image frames to generate video embeddings related to the sequence of the image frames. The video embeddings can have a plurality of image embeddings associated with the sequence of image frames. Furthermore, the system can determine one or more video results based on the video embeddings and the audio data. Subsequently, the system can transmit, to the user device, the one or more video results.
Owner:GOOGLE LLC

Three-dimensional dynamic scene reconstruction method and apparatus, and storage medium

The present disclosure relates to the field of computer vision and discloses a three-dimensional dynamic scene reconstruction method and apparatus, and a storage medium. The three-dimensional dynamic scene reconstruction method comprises: acquiring synchronized videos of a plurality of viewpoints of a dynamic scene; computing matching points between video images of different viewpoints, and estimating intrinsic and extrinsic parameters of each camera; obtaining a Gaussian splatting point set {p0} on the basis of a sparse point cloud constructed according to the depth of each matching point; for the first image frame of each video, using {p0} to perform static training thereon, to obtain a Gaussian splatting point set {p}; for the remaining image frames, dividing {p} into a static point set {S} and a dynamic point set {D}, performing dynamic training on {D}, and constructing a dynamic Gaussian splatting point set {P} from {p}, {S}, and the final {D}; and, in view of the intrinsic and extrinsic parameters of each camera, rendering {P} using a Gaussian splatting rendering pipeline, to obtain rendered images at different moments from new viewpoints.
Owner:TSINGHUA UNIVERSITY

Pixel calibration method and system of high-precision liquid crystal display screen

The invention relates to the field of display screen pixel calibration, in particular to a pixel calibration method and system of a high-precision liquid crystal display screen. The method comprises the following steps: acquiring a real-time playing animation image of a high-precision liquid crystal display screen based on an internal sensor of the display screen; carrying out image scene distortion feature mining on the real-time playing animation image so as to obtain a comprehensive scene distortion feature; performing frame-by-frame pixel optical tracking on the real-time playing animation image, and performing frame-by-frame rolling offset vector calculation so as to construct a pixel point time sequence offset trajectory diagram; carrying out regional dynamic optical flow estimation on the pixel point time sequence offset trajectory diagram, carrying out dynamic trajectory distortion modeling based on comprehensive scene distortion characteristics, and constructing a pixel point offset trajectory distortion model; and carrying out dynamic color change perception on the pixel point offset track distortion model so as to generate a self-adaptive pixel point track offset compensation value. According to the invention, efficient and accurate pixel calibration is realized, and the dynamic visual effect is improved.
Owner:SHENZHEN E B CO LTD

Efficient 3D Gaussian scene reconstruction method based on multi-modal depth distribution supervision

The invention discloses an efficient 3D Gaussian scene reconstruction method based on multi-modal depth distribution supervision, and compared with the prior art, pixel-level depth dense point clouds are directly extracted in a Gaussian initialization stage, a multi-modal depth distribution map is generated, random initialization of dense Gaussian positions according to COLMAP sparse point clouds is effectively avoided, and the reconstruction efficiency is improved. The problems that under the condition of few views, point clouds obtained by COLMAP are too sparse, provided geometric constraints are insufficient, and meanwhile randomly initialized Gaussian blocks the point clouds to move to the correct position due to the local minimum value are solved. According to the method, an image segmentation algorithm based on edge detection is applied to the image frame, a large number of Gaussian primitives which are far away from edge points and are similar in depth are fused, the 3D Gaussian number is effectively pruned, and memory consumption of a GPU is saved. According to the method, extra isotropic regularization loss is introduced into global gauss, so that the problem of Gaussian distribution in which artifacts are generated along the line-of-sight direction and the height is lengthened is avoided.
Owner:CHENGDU UNIV OF INFORMATION TECH

Three-dimensional pose estimation using two-dimensional images

In various examples, systems and methods for pose detection model training for predicting three-dimensional pose estimates using two-dimensional image data are provided. The occupant pose detection model may be trained using multi-view image sensor training data that includes image frames that capture a pose of a training subject within a machine interior using multiple synchronized optical image sensors placed around the machine interior that produce a set of captured image frames of the training subject from different viewpoints. Based on the multi-view image sensor training data, the occupant pose detection model may generate a set of individual, predicted 3D pose estimates for the training subject from a captured image frame from each of the respective optical image sensors. To adjust the occupant pose detection model during training, a loss feedback may be generated that comprises a pose alignment loss, a pose depth loss, and / or a ground truth kinematic loss.
Owner:NVIDIA CORP

Fire-fighting equipment detection early warning method and system based on visual camera

The invention belongs to the technical field of fire safety, and discloses a fire-fighting equipment detection early warning method and system based on a visual camera, and the method comprises the following steps: obtaining multi-focal-length image frames of a fixed spray head and a fire extinguisher edge region, judging the continuous change of gray scale in the continuous image frames, and carrying out the detection early warning of the fire-fighting equipment based on the visual camera. Comparing the gray level distribution difference under the long focus and the short focus, identifying the dynamic trajectory of the contour line angular point of the fire-fighting equipment association structure, analyzing the delay of the function action response, marking as an abnormal region, and generating a fire-fighting equipment collaborative early warning information set; according to the invention, by extracting and analyzing the multi-focal-length image frames of the edge areas of important facilities such as a fixed spray head and a fire extinguisher, fine changes of the fire-fighting facilities under differentiated focal lengths can be captured, working states and potential faults of the fire-fighting facilities can be judged, functional abnormalities caused by equipment aging or damage can be recognized in advance, early warning can be carried out in real time, and the safety of the fire-fighting facilities can be improved. And a higher guarantee level is brought to fire safety.
Owner:WEIFANG PING AN FIRE ENG CO LTD

Underwater vision-inertia-acoustic odometer positioning method fused with DVL

The invention discloses an underwater vision-inertia-acoustic odometer positioning method fused with a DVL, and belongs to the technical field of robot navigation positioning. According to the method, firstly, an underwater image acquired by a camera and carrier motion data acquired by an IMU and a DVL are used as input information to carry out time alignment and space coordinate system conversion; performing feature point extraction on the image to construct a visual re-projection residual error, and dynamically adjusting a weight ratio of visual information based on the number of feature points and the re-projection residual error; calculating IMU data increment between adjacent frames based on pre-integration to construct an inertial residual error; calculating the displacement change between image frames according to the speed information measured by the DVL to construct a DVL residual error; establishing a nonlinear least square optimization function based on Bayesian maximum posteriori estimation; vision, inertia and acoustic speed measurement data are fused at the rear end through a sliding window mechanism, a multi-constraint joint optimization model is constructed, and the robustness and accuracy of a vision-inertia-acoustic odometer system in the underwater environment are improved.
Owner:HOHAI UNIV

Material intelligent transportation and safety monitoring system and method for shield construction

The invention relates to the technical field of tunnel engineering construction, and discloses an intelligent material transportation and safety monitoring system and method for shield construction, and the system comprises a visual perception unit, a sensor network module, an AI analysis center module, a safety decision module and a human-computer interaction interface. According to the invention, data acquisition is carried out through the visual perception unit and the sensor network module, multi-target detection operation is carried out on image frames through the AI analysis center module after target identification and track prediction, target types, space coordinates, contour boundaries and confidence coefficients are identified and extracted, and safety judgment and early warning output are carried out. According to the invention, by integrating the multi-view camera equipment and the UWB, GNSS and other sensors and adopting a deep learning target detection algorithm, high-precision identification and continuous tracking can be carried out on construction site personnel, equipment, segments and other key objects, and accurate input is provided for subsequent risk analysis.
Owner:CHINA RAILWAY 11TH BUREAU GRP CORP LTD +1

Structural perception three-dimensional scene reconstruction method and device

The invention discloses a three-dimensional scene reconstruction method and device based on structure perception. The method comprises the following steps: constructing a sparse point cloud generation model according to a diffusion model based on a Unet architecture; inputting the information into a sparse point cloud generation model, and generating camera internal and external parameters and sparse point cloud data of the RGB image frame under the corresponding view angle; constructing a rough three-dimensional structure grid according to the sparse point cloud data; carrying out random sampling on the rough three-dimensional structure grid, and carrying out Gaussian field initialization on the Gaussian primitives according to a random sampling result to obtain an initialized Gaussian field; the loss between the current image frame and the RGB color image is calculated, parameters of Gaussian primitives of all the RGB image frames in the information are iteratively optimized in sequence according to the loss, all the optimized Gaussian primitives are combined, and a reconstructed three-dimensional scene is obtained. According to the invention, the spatial consistency and accuracy of reconstruction results are improved.
Owner:CHENGDU UNIV OF INFORMATION TECH

Network edge monitoring and early warning method based on video image AI analysis

The invention discloses a network edge monitoring and early warning method based on video image AI analysis, and the method comprises the following steps: S1, obtaining video data, processing the video data, and generating an image frame sequence; s2, analyzing an image frame sequence, and extracting space and time features; s3, taking the target as a hypergraph node, constructing a hyperedge based on space and time features, and dynamically adjusting hyperedge connection by using a genetic algorithm; s4, constructing a multi-layer hypergraph Transform network, extracting multi-scale spatial-temporal characteristics, and calculating a semantic relationship between nodes; s5, setting a butterfly optimization algorithm initial population, and dynamically optimizing model parameters through global and local search; s6, constructing an anomaly detection model, identifying an abnormal behavior, and feeding back a result to optimize model parameters and hyperedge selection; and S7, deploying the model at an edge node, triggering early warning when an abnormal behavior is detected, and pushing information to a management platform. According to the invention, through video image AI analysis, accurate detection and real-time early warning of abnormal behaviors in a network edge scene are realized.
Owner:SHAANXI VIDEO BIG DATA CONSTR & OPERATION CO LTD

Image-based three-dimensional occupant assessment for in-cabin monitoring systems and applications

In various examples, image-based three-dimensional occupant assessment for in-cabin monitoring systems and applications are provided. An evaluation function may determine a 3D representation of an occupant of a machine by evaluating sensor data comprising an image frame from an optical image sensor. The 3D representation may comprise at least one characteristic representative of a size of the occupant, (e.g., a 3D pose and / or 3D shape), which may be used to derive other characteristics such as, but not limited to weight, height, and / or age. A first processing path may generate a representation of one or more features corresponding to at least a portion of the occupant based on optical image data, and a second processing path may determine a depth corresponding to the one or more features based on depth data derived from the optical image data and ground truth depth data corresponding to the interior of the machine.
Owner:NVIDIA CORP

Light guide plate defect detection method and system based on neural network

The invention discloses a light guide plate defect detection method and system based on a neural network, and particularly relates to the technical field of machine vision detection, and the method comprises the following steps: aiming at the problem of image instability of a light guide plate in a dynamic transmission or rotation process, continuously collecting an image sequence and extracting time domain features; and performing interference judgment in combination with the inter-frame consistency prediction coefficient and a first threshold to realize accurate identification of the abnormal image frame. For an abnormal image frame, further correcting the recognition credibility of the abnormal image frame by adopting a confidence adjustment and fusion mode, and meanwhile, introducing a frequency domain transformation and image enhancement strategy to compensate detail loss caused by motion blur; according to the method, inter-frame consistency analysis, confidence fusion regulation and control and frequency domain fuzzy recognition and compensation mechanisms are introduced, abnormal judgment and image quality restoration of the light guide plate image in the dynamic scene are realized, the recognition accuracy and stability of the neural network model on the defect type, position and confidence are improved, and the false detection and omission ratio is effectively reduced.
Owner:深圳市鸿卓电子有限公司

Nighttime unmanned aerial vehicle object tracking method fusing hybrid attention mechanism

Disclosed is a nighttime unmanned aerial vehicle (UAV) object tracking method fusing a hybrid attention mechanism. According to the method, a night vision image enhancement model is employed to enhance various nighttime image frames in a night vision video sequence of UAV before performing image object tracking and recognition, to improve the accuracy of object tracking for nighttime images. With the aid of a spatial channel hybrid attention mechanism, the night vision image enhancement model can adaptively learn spatial channel features, enhancing generalization ability, and enabling the network model to focus on more effective features. Additionally, a denoising processing module is introduced to remove noise hidden in the nighttime images, improving the object positioning accuracy and tracking performance of UAV in nighttime scenarios, and reducing the failure probability of object tracking by UAV in nighttime scenarios.
Owner:CHONGQING UNIV OF TECH

Systems and methods for improving detection of fetal congenital heart defects

Systems and methods are provided for aiding the detection and diagnosis of critical heart defects during fetal ultrasound examinations, in which image data (e.g., motion video clips and / or image frames) is analyzed with machine learning algorithms to detect and identify morphological and / or flow abnormalities indicative of critical CHDs. The results of the analyses are presented for review to the clinician, optionally with an overlay, for the selected image frames that identifies the abnormalities with graphical or textual indicia. The overlay further may be annotated by the clinician and stored to create documentary record of the fetal ultrasound examination.
Owner:BRIGHTHEART SAS

Unmanned aerial vehicle visual navigation method and device based on deep learning

The invention relates to an unmanned aerial vehicle visual navigation method and device based on deep learning, and the method comprises the steps: obtaining a visual image collected by an unmanned aerial vehicle visual system, carrying out the feature extraction of the visual image, and obtaining image feature representation; performing type analysis on the image feature representation to obtain scene data, depth information and continuous image frames; performing scene identification on the scene data to obtain an identification result of the obstacle and the passable area; performing optical flow estimation on the continuous image frames and the depth information through a preset optical flow algorithm to obtain a pose estimation result; obtaining a task demand of the unmanned aerial vehicle, and obtaining a navigation control strategy with environmental adaptability; and optimizing the self-adaptive mixed learning model according to the feedback information in combination with a preset attention mechanism and a reward shaping condition to obtain an optimized self-adaptive mixed learning model. The adaptive capacity of the unmanned aerial vehicle in a complex and dynamic environment can be improved.
Owner:SHENZHEN TIANYING BROTHERS UAV INNOVATION CO LTD

Coal mining equipment obstacle avoidance method and system based on machine vision

The invention relates to the field of excavation equipment obstacle avoidance based on image analysis, in particular to a coal mine excavation equipment obstacle avoidance method and system based on machine vision, and the method comprises the steps: obtaining the frequency domain characteristic parameters of the current electromagnetic noise during the operation of underground electromechanical equipment, and synchronously collecting the current original image frame output by a visual sensor; after a current de-noised image frame is obtained after dynamic de-noising processing, an obstacle contour feature point set is extracted through multi-scale edge detection, and an obstacle position confidence map is obtained according to space-time consistency analysis; obtaining point cloud coordinates of an image data loss area caused by electromagnetic interference, and mapping the point cloud coordinates to an image coordinate system to obtain a three-dimensional geometric complemented image; and constructing a dynamic occupation grid map of the underground environment according to the three-dimensional geometric complementation image and the obstacle position confidence map, and planning an obstacle avoidance path of the mining equipment according to the dynamic occupation grid map. And the obstacle avoidance capability of the mining equipment is improved, and safe and efficient coal mining operation is guaranteed.
Owner:TIANCHEN COAL MINE OF ZAOZHUANG MINING GRP

Concrete crack three-dimensional reconstruction method and system based on multi-modal fusion and medium

The invention discloses a concrete crack three-dimensional reconstruction method and system based on multi-modal fusion and a medium, and relates to the technical field of structural engineering detection, and the method comprises the following steps: obtaining a structural image frame sequence and a structural laser radar point cloud frame sequence of a target structure; obtaining a crack mask image frame after crack segmentation; an integral structure de-noised point cloud picture is obtained; obtaining a visible point cloud coloring point cloud map and a visible point cloud semantic segmentation point cloud map; obtaining an overall three-dimensional coloring point cloud map and an overall three-dimensional semantic segmentation point cloud map; obtaining an overall three-dimensional point cloud marking map; and performing three-dimensional attribute measurement on the crack based on the three-dimensional point cloud marking map to obtain three-dimensional geometric information of the crack. According to the method provided by the invention, a multi-frame and multi-modal fused crack structure reconstruction framework is designed, the method can adapt to crack detection of various three-dimensional structures, and simultaneous detection of crack width information, crack position and crack trend information can be realized based on an overall three-dimensional point cloud marking map.
Owner:CENT SOUTH UNIV

Space interaction accurate identification method and system based on multi-modal fusion

The invention provides a space interaction accurate recognition method and system based on multi-modal fusion, and relates to the technical field of artificial intelligence, and the method comprises the steps: collecting human skeleton, visual image and voice information, generating a space-time attention map through employing a feature extraction network, enhancing visual features, and executing feature complementary correction. Fusing the three types of modal information to obtain unified feature representation; when the recognition confidence is low, the space-time convolution generative adversarial network guided based on the action causal relationship graph complements the missing image frame, and the accuracy and robustness of space interaction recognition are improved.
Owner:ZHONGTIAN ZHILING (BEIJING) TECH CO LTD

Point-line feature fusion-based transformer substation inspection robot positioning and mapping method

The invention discloses a transformer substation inspection robot positioning and mapping method based on point-line feature fusion. According to the method, a binocular camera is used for collecting point-line features in an image frame, and high-quality long-line features are quickly obtained based on an adaptive criterion optimized EDLines algorithm; the spatial depth of point-line features is obtained through triangularization of a binocular camera, and redundant line features are filtered based on a length response threshold strategy of the spatial depth of a transformer substation. Based on a point-line pole feature pole plane constraint method, IMU information and visual information are combined, gyroscope zero offset is independently estimated and optimized, IMU pre-integration information is updated, and a more accurate initial posture of the inspection robot is obtained. A point-line feature dictionary based on a transformer substation scene and a large-scale equipment target detection model in a transformer substation are made, closed-loop detection is performed through transformer substation equipment road signs and point-line features, and rapid relocation correction is performed on sliding windows of a current key frame and a previous fixed key frame number. And finally, the pose of the substation inspection robot and a substation scene map are optimized through global BA, and the problem of large-scale scene error drift accumulation is solved.
Owner:HANGZHOU DIANZI UNIV