267504results about "Character and pattern recognition" patented technology

A system and method are disclosed for generating hyperspectral images from multi-modal sensor data including RGB, LiDAR, thermal, and near-infrared inputs. Training data includes hyperspectral images and corresponding multi-modal measurements. Spectral band grouping is performed based on correlation coefficients. A multi-modal decomposition network with cross-modal attention mechanisms generate reconstructed hyperspectral images by fusing complementary sensor information. A fine-tuning network creates reconstructed RGB images. A comprehensive quality assurance system analyzes spectral consistency, cross-modal coherence, and fusion artifacts to generate quality metrics. Missing data compensation strategies handle corrupted sensor inputs using information from other modalities. The system includes temporal integration for video sequences and multi-resolution processing for different sensor resolutions. Quality metrics guide network weight adjustments to improve reconstruction accuracy while maintaining robustness to sensor failures and environmental variations.

The present disclosure provides an underwater detection method for contact leakage of a tunnel joint of a dam culvert, including: obtaining an underwater image sequence of the culvert by an underwater robot; preprocessing and registering the underwater image sequence; extracting particle information appearing in the underwater image sequence based on the registered underwater image sequence; constructing a three-dimensional fluid velocity distribution map based on the particle information, and determining leakage situation according to the three-dimensional fluid velocity distribution map; and superimposing the three-dimensional fluid velocity distribution map on a preconfigured three-dimensional culvert model, and rendering and displaying it. The present disclosure can adapt to the complex and changeable underwater environment inside the culvert, it has a high sensitivity to fine leakage in the deep layer, it can realize the accurate measurement and trend prediction of leakage, and it can automate the detection process to reduce manual dependence.

Provided are a digital twinning method and system for a scene flow based on a dynamic trajectory flow, which belong to the field of traffic control. The method includes: extracting and identifying a target semantic trajectory with a detecting and tracking integrated multi-modal fusion and perception enhancement network; extracting road traffic semantics, so as to obtain a highly parameterized virtual road layout top view; obtaining a road layout traffic semantic grid encoding vector based on the virtual road layout top view; constructing a target coupling relation model; constructing a traffic force constraint model; constructing a long short term memory trajectory prediction network; predicting a motion trajectory of a target with the long short term memory trajectory prediction network, so as to obtain the predicted motion trajectory; and obtaining a digital twin of the scene flow based on trajectory extraction, semantic identification and the predicted motion trajectory.

Provided is a medical image segmentation method based on multi-scale feature fusion. The method includes acquiring medical image data of a same type, and sketching a region of interest (ROI); preprocessing sketched image data; constructing a multi-scale feature extraction module, a multi-scale feature fusion module, and an encoder-decoder deep learning network model; performing 5-fold cross-validation on the network model; and evaluating a medical image segmentation result output by the model with an evaluation index including a dice, an accuracy, a precision and a recall. This application extracts multi-scale features comprehensively from an encoder, a decoder, and connection between the encoder and the decoder in the network, and effectively learns multi-scale information in the image. The multi-scale features are fused by an interactive module, so this application relieves feature conformity caused by direct multi-scale fusion, thereby improving performance of the medical image segmentation.

The invention belongs to the technical field of multi-modal data processing, and particularly relates to a multi-modal document retrieval enhancement generation method based on a large model, which comprises the following steps: receiving query content input by a user for a multi-modal document; processing the query content by adopting an embedded model, generating a query vector representing user query semantic information, and storing the query vector in a vector database; analyzing the multi-modal document to obtain long text information, segmenting the long text information into data blocks by adopting a recursive partitioning strategy, and numbering and marking the data blocks; carrying out vectorization processing on the data blocks by adopting an embedded model to generate document vectors, storing the document vectors into a vector database, and constructing a hierarchical index structure; retrieving in a vector database based on the query vector, and returning a retrieval result; and processing a retrieval result by utilizing a large language model to generate response content conforming to the query intention of the user. According to the method, the multi-modal document can be effectively analyzed and processed, and the accuracy and comprehensiveness of analysis are improved.

The invention discloses an ancient building risk prediction management and control method and system based on a large model, and relates to the technical field of ancient building protection. An ancient building risk prediction management and control system based on a large model comprises a multi-source data acquisition module, a multi-modal data fusion module, a large model risk prediction module, a risk assessment and early warning module, a digital twinborn dynamic management and control module and a repair decision module. According to the method, the structure state data, the visual image data and the environment data are fused, so that the state of the historic building is comprehensively perceived; the self-supervised learning pre-trained ViT-GPT model is combined with the LSTM time sequence model, so that the time sequence analysis capability and the multi-modal data processing capability of risk prediction are remarkably improved, the future damage probability and the risk trend of the historic building can be accurately predicted, and the defects of the prior art in long-time span risk prediction are overcome.

A system for detecting sensor position change (DCPC) on movable vehicle components includes two or more sensors on at least one of the movable vehicle components. The sensors detect optical information within a distinct field of view (FOV) about an environment surrounding the vehicle. The distinct FOV of each sensor at least partially overlaps with an FOV of at least one other sensor. The system includes controllers that execute an application for DCPC. The DCPC application acquires overlapping optical information from the sensors, computes a conditional correspondence probability distribution for feature points in the overlapping optical information, computes a normalized joint entropy of the feature points, determines that the sensor is in a position manageable by the DCPC, continuously monitors positions of each of the sensors, selectively dynamically aligns the sensors, and ensures that the sensors are calibrated for vehicle perception tasks including advanced driver assistance system (ADAS) functions.

The invention provides a battery film production automation control method and system based on visual inspection, and relates to the technical field of production control, and the method comprises the steps: collecting a film image, carrying out the fusion, generating a fused film image, carrying out the noise reduction, constructing a feature point matching relation, removing wrong point pairs, carrying out the geometric correction, carrying out the sub-pixel-level alignment, and obtaining a registration image; edge enhancement is carried out based on an improved Frangian filtering algorithm, an edge region is repaired and segmented to obtain a film edge contour, three-dimensional reconstruction is carried out, three-dimensional point cloud coordinates are solved, a film three-dimensional contour model is reconstructed, a film defect detection model is constructed, and a film defect distribution diagram and defect confidence are obtained through forward propagation; performing association analysis with a production process, constructing a feature mapping model, generating a defect knowledge base and matching a process parameter optimization strategy, generating a dynamic instruction and sending the dynamic instruction to a station control unit, monitoring operation data, performing closed-loop feedback control, and obtaining a current optimal control scheme.

Using a pre-trained and fixed Vision Transformer (ViT) model as an external semantic prior, a generator is trained given only a single structure / appearance image pair as input. Given two input images, a source structure image and a target appearance image, a new image is generated by the generator in which the structure of the source image is preserved, while the visual appearance of the target image is transferred in a semantically aware manner, so that objects in the structure image are “painted” with the visual appearance of semantically related objects in the appearance image. A self-supervised, pre-trained ViT model, such as a DINO-VIT model, is leveraged as an external semantic prior, allowing for training of the generator only on a single input image pair, without any additional information (e.g., segmentation / correspondences), and without adversarial training. The method may generate high quality results in high resolution (e.g., HD).

The present invention discloses an interactive behavior understanding method for posture reconstruction based on skeleton and image features. The steps are as follows: constructing and preprocessing the data set, extracting skeleton and image features, fusing and reconstructing these features, and conducting experimental evaluation and validation. This method retains the purity of skeleton features for human behavior information extraction and uses image features to retain effective environmental information, complementing the model feature information. Skeleton features are extracted using a graph convolution network, enhancing the relevance of input skeleton point information for accurate feature extraction. Effective image features are quickly and accurately extracted through the Vision Transformer network combined with a multi-head attention mechanism.

The present invention relates to the technical field of big data processing, and discloses a multi-source heterogeneous data fusion and processing method based on big data, including the following steps: data preprocessing, cleaning, complementing and feature extraction of multi-source heterogeneous data, providing standardized and high-quality input for subsequent processing; dynamic knowledge graph modeling, constructing a dynamic knowledge graph based on the semantic relationship of multi-source data, and generating a unified semantic representation; cross-modal alignment and deep fusion, semantically aligning multi-modal data, and generating a unified comprehensive representation through deep fusion of features; real-time stream processing, dynamically updating and processing real-time updated stream data to achieve real-time fusion of data; adaptive optimization, dynamically adjusting fusion strategy parameters, and improving the adaptability and robustness of the fusion strategy. The present invention realizes efficient fusion, real-time processing and semantic consistency expression of multi-source heterogeneous data through dynamic knowledge graph modeling, cross-modal alignment, real-time stream processing and adaptive optimization.

Disclosed in the present invention is a salient contour matching-based method for target measurement in a severe imaging environment. The method specifically comprises: (1) acquiring a binocular image of a target; (2) establishing a global-local joint constraint-based background light estimation model, and removing a scattering effect of a medium in an imaging environment to obtain a restored left eye image and a restored right eye image; (3) learning an original image, and on the basis of a residual between a network reconstructed image and the original image, obtaining target localization prediction maps of the left eye image and the right eye image; and (4) respectively extracting contour lines of the target in the left eye image and the right eye image, constructing feature matching descriptors of contour points, performing stereo matching on the two sets of contour lines by minimizing matching cost, and performing three-dimensional reconstruction on the contour lines in light of calibrated intrinsic and extrinsic parameters to complete the measurement of a key size. According to the present invention, the key sizes of different targets in a severe environment can be accurately measured, thereby providing an effective solution for the problem of measuring the sizes of targets in a severe environment.

The invention discloses a pumped storage power station construction anomaly detection method and system based on unmanned aerial vehicle image analysis, and relates to the technical field of engineering construction monitoring. Planning a construction area acquisition path, obtaining a multi-view image, processing multi-modal sensor data, and generating a construction site feature tensor; inputting the construction site feature tensor into a dense connection convolutional network for processing, and establishing a four-dimensional construction dynamic model in combination with a heterogeneous graph attention module with a causal reasoning capability and a gating circulation unit; calculating a characteristic distance between the mixed knowledge model and the four-dimensional construction dynamic model by adopting a double-layer contrast learning network, and calibrating an abnormal region and uncertainty thereof; and performing causal reasoning traceability analysis on the detected abnormal region, and screening an optimal compensation scheme through a case-enhanced Bayesian analyzer. According to the method, effective fusion of CAD drawings and expert knowledge is realized through the deep cross-modal knowledge distillation network, and a more comprehensive and normative knowledge basis is provided for anomaly detection.

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.

The present invention relates to the technical field of image fusion. Disclosed is a method for fusing infrared light and visible light images. According to the present invention, design is carried out on the basis of a spatial channel attention mechanism and a gradient aggregation residual dense block, a strong texture and weak texture of a feature are retained by integrating Sobel and Laplacian operators while combining the advantages of ResNeXt and DenseNet, and channel information and spatial information of a feature map are refined by means of introducing a spatial and channel attention mechanism, thus the information capture capability of the feature map is increased, and refined spatial and channel feature maps are fused by using a pooled fusion block, to obtain a high-quality fusion feature.

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor that the system validates above a mathematical threshold.

The invention discloses a target identification tracking method and system based on multi-source fusion imaging, and the method comprises the steps: obtaining and preprocessing multi-modal environment monitoring data, carrying out the image enhancement based on the preprocessed multi-modal environment monitoring data, and obtaining the image-enhanced multi-modal environment monitoring data; performing multi-modal feature extraction on the image enhancement multi-modal environment monitoring data, and introducing a cross attention mechanism to fuse the extracted multi-modal features to construct cross-modal fusion features; constructing a cascade target recognition model, and inputting cross-modal fusion features to perform target recognition on the current monitoring scene; tracking a scene recognition target in the current monitoring scene to generate scene target tracking information, and storing the scene target tracking information in a preset track memory pool; and when a tracking target disconnection condition occurs during target tracking, obtaining a mismatched tracking trajectory, and performing tracking trajectory disconnection repair in combination with the trajectory memory pool. Therefore, the identification accuracy and tracking reliability of the target in the monitoring scene are improved.

The present invention provides a three-dimensional reconstruction method integrating a laser radar and oblique photography, comprising: S1, acquiring an oblique image and image data and preprocessing same to obtain a first oblique image, wherein the image data comprises POS data; S2, acquiring coordinates of a ground control point and high-precision POS data; S3, correcting the oblique image to obtain a second oblique image; S4, performing aerotriangulation on the second oblique image, and obtaining an effective matching point pair by means of improved SIFT; S5, performing combined adjustment on the basis of the effective matching point pair and performing dense matching to obtain a dense point cloud; S6, acquiring a laser point cloud and preprocessing same; S7, unifying the coordinates of the laser point cloud and the dense point cloud, and using improved ICP to obtain a fused point cloud; and S8, constructing a triangulated irregular network and performing texture mapping to generate a three-dimensional model. In the present invention, on the basis of the advantages of high resolution and strong anti-interference capability of a laser radar, defects in an oblique photography three-dimensional model are mitigated, and meanwhile, an oblique image is corrected and an algorithm used in feature matching is improved, thereby improving the precision, integrity and efficiency of the three-dimensional model.

The invention belongs to the technical field of visual special effects, and discloses an AI visual special effect dynamic generation system fusing multi-modal perception. Through cooperative work of six core modules of multi-modal input processing, importance analysis, parameter configuration, parameterized rendering, parameter optimization and rendering and output, deep fusion of special effects and contents is realized. An image sequence, audio data and scene parameters can be processed at the same time, a multi-modal feature data set is constructed, a scene key visual area is scientifically recognized, a visual importance distribution map is generated, and the special effect parameter configuration and rendering process is guided. An improved neural radiation field technology and a physical constraint model are adopted to ensure the consistency and reality of the special effect at different visual angles; and a multi-dimensional quality evaluation and parameter automatic optimization mechanism is introduced, so that the visual expressive force and the artistic value of the special effect are guaranteed. The creation threshold is remarkably reduced, the cooperative expression ability of the special effect and the content is improved, and the special effect becomes a powerful tool for enhancing the narration and enhancing the emotion.

The embodiment of the invention provides a scene space three-dimensional model dynamic modeling method based on multi-modal data, and the method comprises the steps: fusing the feature information of a video sequence, point cloud data and IMU data, and building a space-time corresponding relation through employing a particle filtering algorithm. Scene semantic understanding is realized based on a Transform multi-modal scene analysis network, scene scores are calculated in combination with spatial density, depth occlusion and shape change features, and an optimal modeling algorithm is selected through a strategy network of deep reinforcement learning training. Meanwhile, scene dynamic features are analyzed by using a deep neural network and a graph neural network, adaptive grid subdivision and geometric constraint reconstruction of a dynamic region are realized, and a reconstruction threshold is dynamically adjusted according to a scene semantic type. According to the method, the three-dimensional reconstruction quality and efficiency of a large-scale complex scene are effectively improved.

The present invention belongs to the technical field of land change detection. Disclosed are a land change detection method combining matrix decomposition and adaptive propagation, and a system. A deep learning semantic segmentation model combining matrix decomposition and adaptive propagation is used for detecting a land change. The model is of a twin double-branch structure, which can receive both an input of a pre-temporal remote sensing image data set and an input of a post-temporal remote sensing image data set and can share network parameters, and which has the advantage of end-to-end feature learning. In the present invention, an attention method based on matrix decomposition is used to replace a conventional attention method based on manual design; moreover, in view of adaptive propagation technology for pixel offsets of neighboring points, a deep learning twin differential semantic segmentation network is established to realize automatic and precise land utilization / land coverage change detection on the basis of high-resolution optical data.

The present disclosure belongs to the technical fields of computers and defect detection. Provided are a defect detection method and a related device. The method comprises: obtaining an image to be processed; processing said image by means of a feature extraction module in a deep-learning defect detection model, wherein the feature extraction module comprises an intermediate feature layer, the intermediate feature layer comprises a dropout layer, and the dropout layer is configured to randomly drop features which are extracted by means of the intermediate feature layer processing said image, such that a plurality of features subjected to random dropout are obtained after said image is processed by means of the feature extraction module; by means of a detection head in the deep-learning defect detection model, processing the plurality of features subjected to random dropout, so as to obtain a plurality of category prediction results and a plurality of position prediction results of defects in said image; and obtaining category detection results, position detection results, category uncertainties and position uncertainties of the defects in said image according to the plurality of category prediction results and the plurality of position prediction results of the defects. (FIG. 1)

The invention discloses a multi-source remote sensing image semantic segmentation method based on Transform, Mama and a diffusion model, belongs to the technical field of electric digital data processing, graphic data reading and image data processing, and is used for extracting image information of a remote sensing image. Comprising the following steps: acquiring remote sensing images, constructing an image information extraction network based on the remote sensing images, and respectively inputting the remote sensing images in a training set, a verification set and a test set into the constructed remote sensing image information extraction network based on Transform, Mama and diffusion models. According to the method, the RGB features and the depth features are subjected to Transform feature extraction, and a cross Mama method is provided to perform cross enhancement on the RGB features and the depth features; according to the method, a row and column pooling Transform module is used for replacing a traditional Transform module, and detail texture information and large-scale structure information of a feature map in the vertical direction and the horizontal direction are extracted at the same time; according to the invention, a cross Mama attention feature fusion module is used to replace a traditional Mama module, and the modeling capability for complex scenes and diversified targets is enhanced.

The present invention discloses an edge-guided RGBD underwater salient object detection method with multi-attention, which includes the steps of: S1 performing channel-dimension preprocessing on RGB image and depth map and serially connecting them to obtain an RMD image; S2 inputting the RMD image into a trained salient object detection model including an encoder structure and a decoder structure; and S3 obtaining salient images and edge images output from the salient object detection model as object detection results, in which the encoder generates feature maps with reduced resolution layer by layer using the RMD image based on a residual neural network, and the decoder structure includes an edge detection branch used to fuse the feature maps layer by layer, and a saliency detection branch used to fuse the feature maps and edge feature maps layer by layer. The present invention preprocesses the input image with underwater imaging features, and uses an edge prediction image to guide multi-layer salient object detection, thereby enhancing edge details of the salient image.

A multi-task joint perception network model and detection method for traffic road surface information can simultaneously detect a lane line and a drivable area. A coordinate attention mechanism is integrated into a traditional feature extraction network to ensure that a feature extraction effect is enhanced while a calculated amount is not increased. In a neck network, a dilated convolution residual module is proposed to enhance performance of prediction of details by the network, and a decoder part shares features of the drivable area into lane line detection to enhance a lane line detection effect under complex road conditions. In a training stage, there is provided a alternating optimization training method to improve integral segmentation performance of the model. The multi-task joint perception network model and detection method realizes quite high accuracy and excellent speed performance in a challenging BDD100K dataset.

The invention provides a live broadcast room content identification and intelligent distribution method and system based on multi-modal fusion, and relates to the technical field of machine learning, and the method comprises the steps: collecting video, audio and bullet screen data of a live broadcast room, and carrying out the feature extraction; performing cross-modal interaction, feature alignment and feature reconstruction on the multi-modal features by using the multi-modal feature fusion model to generate cross-modal fusion features; and carrying out content identification based on a pre-trained content classification model to obtain a category label and a time sequence feature. Furthermore, a live broadcast room content portrait and an association network are constructed, and a live broadcast room real-time score is generated and a live broadcast room display weight and a push strategy are dynamically adjusted in combination with user historical watching behavior data and a deep reinforcement learning model. Through multi-modal data fusion and deep reinforcement learning, accurate identification and personalized recommendation of the live broadcast room content are realized, and the user experience and the platform operation efficiency are effectively improved.

Since defect areas in collected hydro turbine defect images account for a small proportion and there is a significant color difference between surrounding background and defects, which creates too much redundant information and greatly affects detection speed, a method for detecting defects in a hydro turbine top cover based on an improved YOLOv8 model involves first cropping collected defect images to obtain cropped images, then expanding the cropped images by image enhancement techniques such as image flipping and image blurring to obtain a training dataset. A defect detection network based on YOLOv8-CBAM is constructed and trained to generate a defect detection model for detecting the defects in the hydro turbine top cover. The method achieves high precision in defect detection for the hydro turbine top cover.

Systems and methods of sensor data fusion including sensor data capture, curation, linking, fusion, inference, and validation. The systems and methods described herein reduce computational demand and processing time by curating data and calculating conditional entropy. The system is operable to fuse data from a plurality of sensor types. A computer processor optionally stores fused sensor data that the system validates above a mathematical threshold.

The invention provides a circuit board welding fault identification method and system based on machine vision, and relates to the technical field of visual identification, and the method comprises the steps: obtaining an image of a circuit board welding region through employing a structured light projection system and a multi-camera imaging system; realizing accurate positioning of a defect region in the region of interest by utilizing three-dimensional shape information and through morphological operation and an edge detection algorithm; for the accurately positioned defect area, obtaining high-precision three-dimensional point cloud data of the defect area, extracting a multi-scale curvature feature descriptor, and fusing the multi-scale curvature feature descriptor with visual features to form a multi-feature descriptor; inputting the multi-feature descriptors into a defect classification and recognition module based on ensemble learning to obtain the types and severity of the welding defects; and according to a defect identification result, the three-dimensional space distribution and severity of the welding defects are visually displayed.

The present application relates to a weld seam surface defect detection method and system. The weld seam surface defect detection method comprises: performing multi-angle and multi-surface acquisition of target surface images; then, reconstructing normal maps of the target surface images by means of a high-performance algorithm, and clarifying normal features of defects; and finally, on the basis of the surface normal features, aiming at transverse texture defects and height mutation defects, respectively designing a transverse texture defect detection algorithm based on two stages and rotated rectangle extraction and a height mutation defect detection algorithm based on adaptive dual-threshold segmentation. The present application overcomes texture noise and pseudo-defect interference of ground surfaces, effectively solves the problems of difficulty in feature extraction and positioning of ground surfaces of steel rail weld seams, and has the system operating efficiency superior to that of existing detection methods, thus satisfying requirements of actual production.