7733 results about "Object detection" patented technology

The invention provides a construction scene prediction method and device fusing an image, a text and a BIM modal, and relates to the technical field of intelligent construction prediction management. According to the method, the BIM semantic graph is constructed by extracting the BIM semantic information of the BIM model, and the target detection recognition of the construction site video is carried out in combination with the YOLO model to obtain the target detection result; performing cross-modal alignment with the CLIP to realize deep fusion of the multi-modal data of the image, the text and the BIM to obtain a multi-modal heterogeneous graph; and inputting the multi-modal heterogeneous graph into a space-time sequence model for prediction, outputting prediction results of construction scenes at a plurality of moments in the future, and dynamically mapping the prediction results to a digital twinborn platform to realize risk early warning and visual display. The construction dynamic change can be captured in real time, the construction progress and risk can be accurately predicted, and the intelligent level of construction management is improved.

Various embodiments of the present disclosure relate to operator assistance based on extracting natural language characters from one or more sensed objects. For instance, particular embodiments may generate a natural language utterance based on extracting natural language text in a nearby traffic sign. In an illustrative example, particular embodiments may detect, via object detection and within image data, one or more regions of the image data depicting the traffic sign. Particular embodiments can then extract one or more first natural language characters represented in the traffic sign based at least on performing optical character recognition within the one or more regions of the image data in response to detecting the one or more regions of the image data depicting the traffic sign.

The invention provides an improved YOLOv11s safety helmet wearing detection model and an optimization method thereof, and relates to the technical field of computer vision target detection. According to the improved YOLOv11s safety helmet wearing detection model and the optimization method thereof, the improved YOLOv11s safety helmet wearing detection model comprises the following modules: a multi-modal fusion module, a space-time analysis module, a domain adaptation module, a topological optimization module and a dynamic architecture module; and the multi-modal fusion module is used for realizing feature decoupling by adopting channel separation convolution based on input RGB and near-infrared images, fusing visible light and thermal radiation features through a dynamic weight distribution algorithm, implementing affine transformation alignment on multi-scale features by utilizing a spatial transformation network, and generating a multi-modal feature graph. Through fusion of visible light and near infrared spectrum features and implementation of dynamic weight distribution, complementarity of target texture and thermal radiation features under a complex illumination condition is enhanced, and the problem of feature distortion of single-mode data in a strong backlight or low-illumination scene is solved.

The invention discloses a manipulator grabbing method based on deep learning target detection and image segmentation, and relates to the technical field of artificial intelligence and robotics.The manipulator grabbing method comprises the following steps that a scene image to be processed is collected, the image quality is improved through the multi-light-source fusion image enhancement technology, and recognition errors caused by uneven illumination are reduced; and inputting the enhanced image to a pre-trained deep learning model, executing a target detection task, and outputting an initial bounding box and a category label of the target object. According to the method, through multi-light-source image enhancement and high-precision image segmentation, the accuracy of target recognition and contour extraction is remarkably improved, and the capture failure rate caused by image misjudgment is reduced. And meanwhile, geometric consistency verification and a multi-factor grabbing scoring mechanism are introduced, dynamic screening and collision pre-detection are conducted on the paths, the grabbing stability and safety of the mechanical arm in the complex environment are effectively guaranteed, and the intelligence and robustness of the whole system are remarkably improved.

The invention discloses a multi-view panoramic point cloud splicing method, which comprises the steps of performing joint calibration on a binocular camera and a laser radar through a calibration algorithm, and aligning a coordinate system; a binocular camera is used for collecting a two-dimensional image and carrying out distortion correction, a depth map is generated based on parallax calculation, and three-dimensional point cloud reconstruction is carried out in combination with a triangulation principle; performing deep learning target detection on the two-dimensional image, and screening a line rod assembly area through non-maximum suppression; multi-view point cloud data are collected through a laser radar, point cloud segmentation processing is carried out, and a telegraph pole assembly point cloud subset is reserved; matching the two-dimensional pixel area of the telegraph pole assembly with a laser radar point cloud projection result, and screening point cloud data belonging to the telegraph pole assembly; and carrying out alignment and fusion on the multi-view point clouds through initial registration and fine registration by adopting an improved point cloud splicing algorithm to generate a panoramic point cloud image. According to the method, the point cloud splicing processing time is shortened, and the robustness, precision and integrity of point cloud splicing are improved.

The invention relates to the technical field of construction progress monitoring, and discloses a construction progress monitoring method and system based on big data. The method comprises the following steps: forming a space-time alignment data set through multi-source data acquisition, filtering and quality evaluation; performing feature extraction and registration to generate a digital model; target detection classification is performed to form a completion state table; progress evaluation is achieved through component-task mapping; trend analysis and risk identification are performed to generate a prediction result; decision reference is provided for personalized information screening and augmented reality display. Through multi-source data acquisition, fusion and intelligent analysis, accurate perception, objective evaluation, scientific prediction and visual presentation of the actual state of the construction site are realized, so that a comprehensive, accurate and prospective construction progress monitoring method is provided, the construction period delay risk is effectively reduced, and the construction management efficiency is improved.

The invention discloses a video analysis-based electric power operation personnel behavior identification and early warning system and method, which realize accurate identification and real-time early warning of electric power operation personnel behaviors by combining a video analysis technology with multi-modal data fusion, and effectively improve the safety management level of an operation site. Compared with a traditional safety supervision mode, the method employs a mode of combining deep learning target detection and time sequence behavior analysis, improves the recognition accuracy of operators and safety equipment, fuses the data of equipment worn by the operators with video data, improves the detection precision, and improves the safety supervision accuracy. And misjudgment caused by illumination change, shielding or complex environment is reduced. Besides, high-risk violation behaviors such as no safety helmet wearing, no safety belt fastening, violation climbing, high-altitude object throwing and the like are accurately recognized through the violation detection module, and different levels of alarm measures are adopted according to the severity of the violation behaviors in combination with an early warning feedback mechanism, so that the pertinence and response efficiency of early warning are improved.

Vehicle computing systems may receive and fuse long-wave infrared data with radar data to detect and track objects in low-visibility driving environments. In some examples, radar data including position and velocity data may be projected over infrared image data to detect, classify, and track infrared-emitting objects. Machine-learned transformer models with attention also may be trained to output object detections based on combined infrared and radar data. The fusion of infrared and radar data may be used individual and / or may be synchronized with other sensor modalities. In some examples, the fusion and analysis of infrared and radar data may be used in specific low-visibility driving environments, using low-light, fog, and shadowed areas of the environment, to detect and track infrared-emitting and / or moving objects such as pedestrians and animals that may be obscured from detection by other sensor modalities.

The invention discloses an operating room intelligent monitoring method and system based on monitoring video recognition, and relates to the technical field of medical safety supervision, and the method comprises the steps: collecting a video through an operating room camera array, and constructing an operation region panoramic sequence through a residual neural network and an optical flow field; a double-branch target detection network is adopted to extract the position of a medical worker, the body position of a patient and the characteristics of surgical instruments, and a surgical scene model is constructed; performing trajectory tracking based on skeleton key point extraction and Kalman filtering, and constructing a dynamic graph of the operation process; generating an operation process state report by using the time sequence diagram convolutional network and a multi-head attention mechanism; and comparing the operation specification library through a knowledge distillation algorithm, and carrying out grading recording on abnormal events. According to the invention, efficient abnormal event tracking and recording functions are realized, technical support is provided for operation quality control and safety management, and the overall performance of operating room intelligent monitoring is improved.

A system to detect, classify and track small objects in an image frame. The system includes an image data system and image analytic system. The image data system receives real-time image data from an image capturing system and inputs the image data to image analytic system. The image analytic system performs the object detection, classification and tracking of objects in an image frame using a combination of techniques.

The invention provides a robot obstacle avoidance and navigation method based on multi-modal fusion and a visual language model, and the method comprises the steps: collecting different modal data in real time through a plurality of sensors, and carrying out time synchronization processing and normalization processing; extracting features of different modal data and fusing the features through an intermediate layer; and inputting the fused multi-modal data into a visual language model, generating a semantic map of the environment by utilizing semantic segmentation of the model and a target detection result, generating an action strategy in combination with a natural language instruction and a visual analysis result, and converting the generated action strategy into a control signal which can be executed by the robot to realize closed-loop control. According to the method, the visual language model and the multi-modal sensor fusion technology are combined, and the perception, decision making and real-time response capabilities of the mobile robot in a complex dynamic environment are improved.

The invention provides a data center inspection robot monitoring analysis method and system based on machine vision, and relates to the technical field of inspection robots, and the method comprises the steps: obtaining a video stream and environment parameter data collected by an inspection robot, and carrying out the detection and recognition of an abnormal state through a deep learning target; and constructing a multi-modal data fusion analysis model to form a knowledge graph to generate a root cause analysis result, and planning an inspection path based on priority scores. According to the invention, intelligentization and precision of data center monitoring are realized, and inspection efficiency and fault diagnosis accuracy are improved.

A multimodal intelligent agent system for dynamic environmental monitoring and user-centered support, consisting of: a multimodal sensor module configured to continuously acquire environmental and behavioral data from multiple input modalities, including at least one visual sensor, at least one acoustic sensor, at least one environmental conditions sensor, and at least one proximity or motion detection sensor, each generating modality-specific data streams representing visual images, audio waveforms, physical environmental parameters, and motion signatures within a monitored environment; a data preprocessing and fusion subsystem that is operationally coupled with the multimodal sensor module and configured to normalize, temporally align, and transform the modality-specific data streams into high-dimensional feature embeddings using a variety of encoders, wherein the visual encoder uses convolutional or vision transformer architectures, the audio encoder uses a spectral-temporal feature extractor, and the sensor encoder transforms raw analog data into context vectors suitable for multimodal alignment; a multimodal processing unit consisting of a transformer-based large language model (LLM) trained on paired multimodal datasets and configured to perform semantic fusion, context abstraction, and inference across the aforementioned aligned multimodal feature embeddings to generate a contextual understanding of environmental and behavioral states; an adaptive agent controller coupled to the multimodal inference processing unit and configured to instantiate, manage, and terminate a variety of task-specific intelligent agents, each agent being a software unit configured to perform a specialized function selected from meeting summarization, behavioral analysis, misplaced object detection, or environmental anomaly identification, with the agents dynamically interacting with the inference engine to retrieve contextually relevant multimodal embeddings for task execution; a personalization and adaptive learning subsystem consisting of a user preference database and a neural memory structure configured to update and refine model parameters based on user-specific interaction history, thereby enabling personalized output generation, prioritization of recommendations, and long-term behavioral adaptation; and An output generation interface is operationally connected to the adaptive agent controller and configured to produce multimodal output in textual, visual, and auditory form. The interface is capable of displaying human-readable summaries, notifications, and visual reconstructions of identified entities or environmental states.

Disclosed are techniques for combining data using transformer-based machine learning models. In some examples, a first transformer is used to combine a first dataset with a second dataset. The results are then combined with a third dataset, using a second transformer. Each dataset can represent data from a different sensor modality. The transformers compute scores based on queries and apply the scores to values. The first dataset can be used to generate queries for the first transformer, and the values for the first transformer can be derived from the second dataset. Similarly, the third dataset can be used to generate queries for the second transformer, and the values for the second transformer can be derived from the output of the first transformer. The output of the second transformer is therefore a combination of all three datasets and can be used for object detection, for example, determining three-dimensional boundaries of objects.

The invention relates to the field of computer vision, and discloses a photovoltaic panel defect category detection algorithm based on feature pyramid and cascade group attention, comprising the following steps: S1, data acquisition, labeling and preprocessing; s2, carrying out model training by using an improved YOLOv11 algorithm; and S3, predicting the input infrared picture, accurately positioning the defect and marking the category. According to the photovoltaic panel defect category detection algorithm based on the feature pyramid and the cascading group attention, the feature pyramid and the cascading group attention can be detected according to the internal circuit structure of the photovoltaic panel and an infrared imaging picture; accurately positioning, marking and classifying the defects on the photovoltaic panel; a newest YOLOv11 target detection algorithm is adopted and improved, and part of C3k2 modules are replaced by SCC3k2 modules, so that information capture on space and channels is more accurate, and adverse effects caused by redundant information extraction are reduced; and meanwhile, different input segmentation is provided for each attention head in the attention module, and the diversity of the characteristics input to the attention heads is enhanced by cross-head cascade output characteristics.

The present invention relates to the field of ship design, and in particular to an automatic three-dimensional modeling method, system and device based on a two-dimensional ship CAD drawing. The method includes obtaining and parsing ship CAD drawing data, extracting metadata from the drawing, and identifying primitives; generating images through rendering technology for unidentified complex primitive areas, and performing preprocessing; inputting the preprocessed images into an object detection model, generating detection frames and category annotations using a deep learning algorithm, and performing coordinate conversion to accurately calibrate the positions of primitives; extracting geometric information, structural attributes and material data based on object detection results and primitive information, performing parametric expression conversion, and generating parametric model data of the ship structure; inputting the generated parametric data into a modeling instruction template, generating a ship CAD modeling instruction sequence, and executing the instructions through a CAD system to automatically construct a three-dimensional model of the ship. The present invention can realize the fully automated conversion from two-dimensional CAD drawings to three-dimensional modeling, thereby improving the efficiency and accuracy of ship design.

The invention relates to the technical field of material detection, in particular to an optimization method for industrial pure iron metallographic specimen defect detection based on YOLOv8obb. The method comprises the following steps: acquiring an industrial pure iron metallographic sample defect image; preprocessing the metallographic image; constructing a YOLOv8obb defect detection network, and optimizing the defect detection network to obtain an optimized defect detection model; according to the model, a DynamicConv is used as a convolutional layer of Bottleneck in a C2f module, a SimAM attention mechanism module is introduced into a backbone network, a tiny target detection head is added, and an NEFPN feature pyramid network is used as a neck multi-scale feature fusion network; and the preprocessed sample data image pair is used for performing defect detection model training, and the trained optimized defect detection model is used for performing industrial pure iron metallographic image defect detection, so that the detection precision and the detection efficiency are improved.

The invention discloses a chip defect weak supervision semantic segmentation method based on YOLO and a diffusion model, and the method comprises the steps: collecting a chip defect weak supervision data set, carrying out the weak supervision manual marking, and constructing a chip defect image weak supervision data set; constructing and training a target detection model based on the chip defect image weak supervision data set; obtaining a to-be-detected chip defect image, inputting the to-be-detected chip defect image to the trained target detection model for defect position prediction, and outputting a chip defect area positioning image; inputting the chip defect area positioning image into a diffusion model for image reconstruction, and outputting a chip reconstruction image; and performing difference value comparison and thermodynamic diagram analysis on the to-be-detected chip defect image and the chip reconstruction image, and outputting a pixel-level precision chip defect semantic segmentation result. According to the method, the model is trained by using the low-precision weak supervision annotation data, so that high-precision semantic segmentation is realized, the training cost is reduced, and the practicability of the model is improved.

Object detection and tracking systems may use machine-learned transformer models with self-attention for detecting, classifying, and / or tracking objects in an environment. Techniques described herein may include receiving sensor data generated by different sensor modalities of a vehicle, determining different bounding shapes based on the different sensor modalities, and using a machine-learned transformer model to determine associated and / or combined bounding shapes. The machine-learned transformer model may receive a variable number of input bounding shapes representing any number of objects and various sensor modalities. Multiple stages of the transformer may be used to determine associated bounding shapes and to assign attributes for the associated bounding shapes, based on the individual bounding shapes of the different sensor modalities and / or previous bounding shapes for objects detected and tracked in a previous scene in the environment.

The invention belongs to the field of defect detection, and particularly relates to an industrial PCB defect identification method based on a sample generation model. Comprising the following steps: constructing an original PCB defect image data set with labels, and preprocessing original PCB defect images to obtain preprocessed images; inputting the preprocessed image into a sample generation model for processing to generate a defect sample; combining the original PCB defect image data set and the defect sample, and performing adaptive size filling and high-frequency noise injection processing to obtain an enhanced defect data set; training the YOLO-ADF target detection model by adopting the enhanced defect data set to obtain a trained YOLO-ADF target detection model; sampling the trained target detection model to carry out industrial PCB defect identification; according to the invention, the accuracy of small target defect detection is improved, the omission ratio is reduced, and the detection accuracy and the model robustness are improved.

The invention discloses a target detection method based on Mama feature fusion, and relates to the technical field of image target detection. According to the method, the innovative implementation of the VSSA module is utilized, a selective scanning mechanism of the state space model is applied to 2D visual data processing, the long-distance dependency relationship in the image is effectively captured through state space modeling in four directions, the limitation of a traditional state space model in the two-dimensional visual data processing process is solved through the multi-direction processing strategy, and the processing precision of the 2D visual data is improved. The model can comprehensively perceive spatial dependency relationships in different directions in an image, the VSSA adopts learnable state space parameters to dynamically model a feature sequence, the ability of the network to understand a complex space structure is enhanced, the method is particularly suitable for processing scenes needing long-distance context information, and in addition, the method is combined with MTMHSA, so that the complexity of the network is reduced. And the fusion capability of different levels of features in target detection is further enhanced. Through the innovation, the model can better understand the target in the image, and the positioning and classification precision of the target is improved.

The invention relates to a multi-article identification and distinguishing method, system and device and a medium. The method comprises the steps of obtaining multi-view image data of articles in a target area; performing object detection and instance segmentation on the multi-view image data by using a deep learning algorithm to obtain preprocessed image data; acquiring a first spatial position of the object in the target area by using a depth sensor, and performing spatial correction by combining the preprocessed image data to obtain depth information of the object in the target area; in combination with the preprocessed image data and depth information, performing secondary positioning on articles in the target area through a multi-modal data fusion algorithm, optimizing the distinguishing between the articles in the target area by using spatial correlation, confirming the identity information of each article in the target area, and generating an identification result; and matching the identification result with the article state in a preset database to generate a matching result, and feeding back the matching result to the user side. The method has the effect of improving the recognition precision of the borrowing machine.

The invention provides a cross-domain target detection method and system based on semi-supervised learning in extreme weather, and the method comprises the steps: enabling a student model to learn bounding box regression knowledge of a trained teacher model through distillation, so as to achieve the regression capability in extreme weather; performing instance-level feature adversarial training on a pre-training detection model which is obtained in a distillation stage and has a regression capability in extreme weather by using an intermediate-state image which is generated by a diffusion model and is closer to a target domain and has smaller noise, and determining a new teacher model and a new student model based on the detection model after the adversarial training; in the fine tuning stage, a dynamic category pseudo label screening strategy is utilized to screen pseudo labels generated by a new teacher model, and a screened target domain pseudo label is utilized to train a new student model to obtain a target detection model used for detecting a to-be-detected image in extreme weather. The phenomena of leak detection and error detection in extreme weather scenes in the prior art can be reduced, and the original reasoning speed is kept.

The present invention relates to the technical field of deep learning and computer vision. Disclosed is an unmanned aerial vehicle image-based small object detection method for target areas. The present invention crops images of obvious small objects in certain target areas, and annotates the small objects of different categories to form a raw training and testing dataset, so as to ensure the accuracy of data required in the early stage of the algorithm and further ensure the scientificity of the algorithm; uses the computing capability of an improved YOLOv7 detection model to collect image features of different degrees in the dataset, the improved YOLOv7 detection model using YOLOv7 as a basic model and adding to a neck network an MS-CET module, which is constituted by an improved self-attention mechanism and convolution module SPPCSP, and a BHC-FB module, which is constituted by bidirectional mixed convolution modules NConv and RPConv connected in parallel; and finally fuses different feature layers as a final judgment basis of an unmanned aerial vehicle for small object detection in the target areas, to further check the accuracy of the algorithm and criteria for dataset selection, thereby improving recognition accuracy.

The invention discloses an improved YOLOv11 power transmission line defect detection method, which introduces an attention mechanism to optimize the feature extraction capability, improves the performance of a model in a complex scene, remarkably enhances the understanding of the model for local and global information, and improves the target detection precision. Spatial offset is carried out on a feature map in a detection head by adopting feature extraction, feature association of different scales is enhanced, the detection precision of a small target is improved, redundant calculation is reduced, the calculation efficiency and performance of a model are further optimized, diversified requirements in a complex environment are met, and the method is suitable for popularization and application. The feature learning ability of a self-integrated attention head optimization target detection model is introduced, and the detection precision and robustness are improved under the scenes of small object detection and complex backgrounds. The method effectively improves the detection precision and training efficiency, can achieve excellent performance in a high-precision requirement and a real-time task, and meets the requirements of safety risk management and control in the power industry.

The invention discloses a construction man-machine safety intelligent detection and early warning method based on deep learning. The method comprises the following steps: S1, introducing a wavelet convolution improved YOLOv11 target detection algorithm framework; s2, coordinate extraction of the instance segmentation module; s3, designing a monocular depth estimation network; s4, training the monocular deep network; and S5, converting the two-dimensional coordinates into the three-dimensional coordinates. According to the invention, the real distance between the worker and the equipment is obtained and real-time safety early warning is carried out; obtaining a man-machine actual distance, and judging the accurate position and posture of the machine by combining angle information of the construction machine with edge contour detection; target detection in a complex construction scene is realized by optimizing a network structure; the S2R-DepthNet depth estimation network is used for performing depth estimation on a single RGB image through a self-supervised learning method, the three-dimensional information acquisition process is simplified, and dependence on expensive hardware is eliminated.

Systems and methods for detecting features, outputting feature representations, determining measurements, and combinations thereof using machine-learned algorithms are provided. In some embodiments, first sample data from a first characterization modality and second sample data from a second characterization modality are provided to a machine-learned algorithm. Improved feature detection, feature representation, and measurement may be realized by using multimodal data with a machine-learned algorithm. In some embodiments, a first characterization modality is an interferometric modality and a second characterization modality is a spectroscopic modality. A first characterization modality may be optical coherence tomography and a second characterization modality may be a diffuse spectroscopy modality, such as near-infrared spectroscopy. Sample data may be intraluminal and / or vascular data useful in characterizing a vascular system of a subject, such as a human.

The invention discloses an interactive action detection method based on multi-level features, and the method comprises the steps: S1, fusing the information of a low-level feature map, a middle-level feature map and a high-level feature map through a cascading fusion mode, and obtaining a fused feature map; s2, acquiring local detail features from the fused feature map, performing global modeling to obtain global features, fusing the global features with the local detail features to generate global context features, performing character detection, object detection and interaction detection in parallel through multi-task branches, outputting character features, object features and interaction features, and outputting the character features, the object features and the interaction features. S3, gradually fusing multi-level contexts through attention interaction of a unitary relationship, a pairwise relationship and a ternary relationship, generating text embedding for an interaction category by utilizing a pre-training text encoder, and aligning the text embedding with visual features to improve the accuracy of fine-grained interaction classification; and S4, explicitly modeling a complex interaction relationship, and finally predicting and outputting a final prediction result by using FFN.

The invention discloses a shaving board surface defect detection method based on deep learning, and the method comprises the steps: collecting shaving board surface images, and screening out shaving board images with surface defects; manually marking the surface defects of the shaving board in the screened shaving board image, constructing a shaving board surface defect data set, and dividing the shaving board surface defect data set into a training set, a verification set and a test set in proportion; an improved YOLOv11 target detection model is constructed; inputting the training set and the verification set into an improved YOLOv11 target detection model for training; performing comprehensive evaluation and analysis on a training result by using an evaluation index; and testing the trained improved YOLOv11 target detection model by using the data of the test set. According to the method, various types of shaving board surface defects can be accurately predicted, the detection precision of tiny defects and complex defects on the surface of the shaving board is also improved, meanwhile, the detection precision and efficiency of the shaving board surface defects are greatly improved, and the labor cost is greatly reduced.

The invention provides a multi-modal three-dimensional target detection method based on an attention mechanism, and the method comprises the steps: obtaining image data and LiDAR point cloud data, carrying out the preprocessing of the image data and the LiDAR point cloud data, carrying out the feature extraction of the processed image data and the processed LiDAR point cloud data, and carrying out the feature extraction of the processed image data and the processed LiDAR point cloud data; and fusing the extracted image features and the point cloud features based on a multimodal feature fusion algorithm of bidirectional information interaction, optimizing the fused features, eliminating redundancy and noise, performing refined processing on the fused features in a specific region of interest, inputting the fused and optimized features into a target detection module, and performing target detection on the target. Through candidate box generation, classification and regression tasks, classification, positioning and identification of a multi-modal three-dimensional target are completed, based on a non-maximum suppression algorithm, a redundant detection box is removed, and a detection result is output, including a category label of the target, a bounding box coordinate and a confidence score. According to the invention, the efficiency and detection precision of multi-modal fusion are improved, and the method is convenient to use.