73551 results about "Computer vision" patented technology

The present invention relates to the technical field of image fusion. Disclosed is a method for fusing infrared light and visible light images. According to the present invention, design is carried out on the basis of a spatial channel attention mechanism and a gradient aggregation residual dense block, a strong texture and weak texture of a feature are retained by integrating Sobel and Laplacian operators while combining the advantages of ResNeXt and DenseNet, and channel information and spatial information of a feature map are refined by means of introducing a spatial and channel attention mechanism, thus the information capture capability of the feature map is increased, and refined spatial and channel feature maps are fused by using a pooled fusion block, to obtain a high-quality fusion feature.

The invention relates to the technical field of camera linkage alarm, and discloses a camera linkage alarm method and system for intelligent environment monitoring, and the method comprises the steps: synchronously collecting the video stream data of a plurality of cameras and the multi-parameter monitoring data of an environment sensor; executing multi-path feature analysis and cross-modal feature fusion to obtain environment-vision joint features; environment scene self-adaptive classification is executed, and environment-vision joint features are analyzed through an anomaly judgment model to obtain an abnormal event judgment result; executing multi-camera collaborative verification and vision-environment data association analysis to obtain multi-dimensional abnormal event description information; and selecting a multi-camera linkage response strategy and coordinating the plurality of cameras to carry out three-dimensional monitoring on the abnormal area to obtain a linkage response execution result, so that a false alarm mode can be identified and an abnormal judgment rule can be dynamically adjusted, continuous optimization of the performance of the plurality of cameras is realized, and the false alarm rate and the missing report rate of the plurality of cameras in long-term operation are reduced.

A multi-modal information tagging method, apparatus and device, and a storage medium and a product. An information tagging effect is effectively improved by means of determining visual feature information of visual information by means of a multi-modal tagging system, performing multi-modal fusion processing on the basis of the visual feature information, the visual information, recognized text information and descriptive text information to obtain image-text fusion features, determining image-text correlation information between the visual information and both the recognized text information and the descriptive text information, on the basis of the image-text fusion features and the image-text correlation information, determining a tagging result for information to be tagged, and performing multidimensional information tagging on the basis of multi-modal information fusion and cross-modal correlations.

The present application discloses a face image restoration method based on a state space model, which includes inputting the to-be-restored face image to a restoration model to obtain a restored face image; the restoration model includes an encoder and a decoder; the encoder sequentially includes a first image fusion module, a first multi-scale state space module, a second image fusion module, a second multi-scale state space module, a third image fusion module, and a third multi-scale state space module; the decoder sequentially includes a fourth multi-scale state space module, a first multi-scale attention fusion module, a fifth multi-scale state space module, a second multi-scale attention fusion module, a sixth multi-scale state space module, and a third multi-scale attention fusion module. This method ensures the consistency of face semantic information while restoring the detailed texture, especially low quality face images in real degraded scenes can achieve better restoration results.

The invention discloses a video content intelligent management method and system based on multi-modal analysis, and belongs to the field of video content management.The method comprises the steps that multi-modal features of video content are extracted; performing fine structure segmentation on the video content based on the multi-modal features, and segmenting the video into fine segments related to semantics; the fragments obtained after fine structure segmentation are subjected to tagging, and metadata is generated for each fragment; constructing a multi-modal semantic index of the video content based on the generated metadata; according to the method, a semantic retrieval function is provided by utilizing multi-modal semantic indexing, fine fragments required by a user are accurately recalled in combination with text label matching, measurement numerical value scoring, cross-modal vector retrieval and filtering based on a shot language and a shooting technology label, and the user queries and positions a specific time point or the fine fragments in a video through a natural language. According to the invention, the accuracy of video content analysis is improved, the user experience is optimized, and the operation complexity is reduced.

An agricultural vehicle includes cameras and a LIDAR sensor operably coupled to the agricultural vehicle, and an imaging controller operably coupled to the LiDAR sensor and the cameras. The imaging controller includes at least one processor, and instructions that cause the processor to receive image data from the cameras, receive LiDAR data from the LiDAR sensor, analyze the image data from each of the cameras to generate labeled image data, analyze the LIDAR data to generate labeled LiDAR data, and fuse the labeled image data with the labeled LIDAR data. Related agricultural vehicles and methods are also disclosed.

The invention discloses a multi-modal medical image data intelligent processing system, relates to the field of medical image analysis, and is applied to multi-modal medical image whole-process analysis of CT, MRI, PET, ultrasound and the like. According to the system, different modal image features are extracted and fused through a cross-modal manifold fusion network; a semantic guidance dynamic registration engine optimizes registration parameters to ensure that the registration error is less than or equal to 1.5 mm; the multi-task collaborative diagnosis network realizes multiple tasks such as disease classification; the clinical knowledge embedding and interpretable module generates a structured report and is in butt joint with an HIS system. Meanwhile, the model is optimized through a federated learning architecture, the adaptability of newly added data is improved by more than or equal to 20%, and intelligent processing and analysis of multi-modal medical images are realized.

The invention belongs to the technical field of virtual person generation, and particularly relates to a digital person video generation method based on a multi-modal large model, and the method comprises the following steps: 1, constructing a multi-modal data system; 2, multi-modal large model training and adaptation are carried out; 3, constructing a digital human three-dimensional model; step 4, performing semantic analysis and modal mapping; 5, generating a time sequence action and a mouth shape; step 6, building and rendering a virtual scene; step 7, audio and video synchronous rendering and synthesis; step 8, quality optimization and defect repair; and step 9, performing user interaction and iterative optimization. Through technical innovation and engineering, the core pain point in digital human video generation is solved, efficient, vivid and customizable content production capacity is provided for virtual anchors, intelligent customer service, enterprise training and other scenes, and the AI digital human technology is promoted to be applied to large-scale business from experiments.

The invention relates to the field of road compactness monitoring, in particular to a road compactness real-time regulation and control method and system based on digital twinning, and the method comprises the steps: collecting vibration acceleration, temperature and position data, and generating a time-space aligned multi-source fusion feature data set after processing; inputting a pre-trained LSTM model to construct a dynamically updated digital twinborn body, and outputting a three-dimensional compaction energy spectrum; dispersing the atlas and calculating parameters, and generating a compaction degree deviation matrix; a regulation and control instruction is generated and issued based on matrix optimization; and according to the measured data and the predicted value residual error, triggering re-optimization and calibrating the sensor. According to the method, the problems of non-uniform compaction quality and over-high energy consumption caused by poor adaptability, decision lag and error coupling of a traditional static model are solved.

An inspection application may display, on a user device, a user interface including at least a portion of a vehicle inspection report including a plurality of inspection categories. The inspection application may configure the user device to obtain inspection information associated with a vehicle, the inspection information comprising photographs, videos, audio, and / or text. The inspection application and / or a network-accessible inspection assistant system may generate a prompt including at least a portion of the inspection information and information indicating potential vehicle defects. The prompt may be transmitted to a large language model that returns a response indicating any potential vehicle defects identified in the inspection information. The vehicle inspection report may then be updated to indicate the potential vehicle defects identified by the language model.

The invention discloses an underwater robot autonomous obstacle avoidance and path planning system based on a multi-modal sensor, particularly relates to the field of underwater robot navigation, solves the problems of multi-modal sensing and path planning in a complex environment, and unifies the feature space of a multi-source heterogeneous sensor based on a physical field correction mode of an acousto-optic fluctuation rule. Fusion deviation caused by physical property difference of sonar and visual data is overcome, and obstacle characterization precision and dynamic environment adaptability are remarkably improved; a dynamic confidence decision-making mechanism is combined with a path curvature-moment constraint model, the environmental perception credibility and the robot motion performance are synchronously fused into a path generation process, the feasibility of motion control is ensured while the geometric collision risk is avoided, and the contradiction between the safety and the performability in path planning is solved; the global path optimization eliminates the hidden danger of local path sudden change through iterative smoothing under double constraints of physical field and dynamics, and forms an optimal navigation scheme considering obstacle avoidance efficiency, energy consumption and motion stability.

A ground visual question-answering method based on dynamic dual-level visual information fusion includes using a dual-level multiscale network, which is divided into language-guided pixel-level features and region-level features. These two scale branches are combined to predict the final textual answer and ground answer. Furthermore, a question-guided dynamic region-level feature localization network is proposed to locate visual information guided by the question and adaptively assign masks of different sizes to ground answers, thereby enhancing the accuracy of locating and segmenting small targets. Additionally, a cross-modal aggregation module is designed to fuse features from both levels, enhancing the fusion of pixel-level and region-level features to improve the segmentation effect of ground answer masks' edges. The ground visual question-answering system built by the language-guided adaptive dual-level feature fusion network in this invention can effectively improve the accuracy of the entire model while answering questions and generating answer ground masks simultaneously.

The invention discloses a heterogeneous double-mechanical-arm tail end pose mapping method, system and device and a storage medium, and the method comprises the following steps: S1, obtaining the angular velocity, position and tail end pose information of each joint of a main mechanical arm, and calculating the spatial position and pose of the tail end of the main mechanical arm; s2, motion trail data in the teaching process of the main mechanical arm is collected, a long-short-term memory network model is established, the pose of the future time step length is predicted, and the motion trail of the auxiliary mechanical arm is planned; s3, data information is transmitted, and the problem of communication delay is solved; s4, the pose information of the tail end of the main mechanical arm is solved, the joint angle is adjusted, and the error is adjusted in combination with proportional-integral-differential control; s5, the position and speed signals are filtered, a mechanical arm dynamic model is established, an adaptive control law is designed, and a trajectory smoothing algorithm is adopted; and S6, establishing a virtual obstacle map, and limiting a safe working space. According to the method, comprehensive and accurate mapping and prediction of the tail end poses of the heterogeneous double mechanical arms are achieved.

The present invention provides an intelligent robot for power inspection and an inspection method thereof, comprising a quadruped robot, wherein a laser radar, a voiceprint sensor and an infrared sensor are provided at the front end of the quadruped robot; a pan-tilt base is provided at the middle of the quadruped robot, and a camera is provided on the pan-tilt base; a six-degree-of-freedom mechanical arm is also provided at the tail of the quadruped robot, and a mechanical claw is provided at the end of the mechanical arm. The robot flexibly plans tasks based on the layout of the substation, sets inspection cycles for different equipment according to their importance, and automatically generates accurate task plans, thereby avoiding the tediousness and omissions of manual planning and improving planning. The advanced path planning algorithm enables the robot to quickly plan the optimal path, integrates multiple positioning technologies to ensure accurate navigation, and effectively reduces travel time. Multiple inspection modes are carried out in parallel, non-stop inspections are dynamic screening, routine inspections are comprehensive inspections, and non-preset inspections are flexibly targeted at specific points, which greatly improves the coverage efficiency of the inspection and avoids detection blind spots.

The invention relates to an AI real-time intelligent guidance and self-adaptive obstacle avoidance method for multi-source sensing fusion of an unmanned aerial vehicle, and the method comprises the steps: collecting flight data through a visual sensor and an infrared sensor, obtaining a timestamp and space coordinate information, calculating a sampling frequency difference, and generating a fusion data set of time-space alignment through a self-adaptive interpolation algorithm; image and heat source features in the fusion data set are extracted, and a multi-dimensional feature set is generated, so that whether an obstacle exists or not is judged, and related information is extracted; and analyzing environment data, calculating an environment complexity index, generating a risk level in combination with obstacle information, predicting a dynamic weight of the sensor, and adjusting a working mode of the sensor. And then based on the obstacle condition, calculating a comprehensive risk value and predicting a motion trend through a path risk model, and finally generating an optimal obstacle avoidance path and a flight instruction, thereby realizing high-efficiency processing of sensor data, accurate risk assessment and real-time obstacle avoidance planning, and ensuring stable and safe flight of the unmanned aerial vehicle in a complex environment.

The invention discloses a multi-modal large model assisted laparoscope soft tissue registration surgical navigation method and system. The method comprises the following steps: acquiring a preoperative CT (Computed Tomography) or MRI (Magnetic Resonance Imaging) image and an intraoperative laparoscope image, and converting to obtain a multi-modal feature sequence; inputting the obtained multi-modal feature sequence into a large model for fusion to obtain a cross-modal matching matrix; random sampling is carried out by using an RANSAC random sampling consensus algorithm, an error between corresponding point sets of two data is measured by using an Euclidean distance, an analytical solution is obtained through SVD decomposition, and a feature matching error is introduced to obtain an optimal rigid body transformation pose; performing non-rigid deformation local fitting adjustment on the optimal rigid body transformation pose residual error by adopting a parameterized deformation model in combination with learning prediction deformation, and performing loop execution to obtain a soft tissue organ real-time registration image; automatic, high-timeliness and high-precision real-time registration of soft tissues is realized under the multi-mode and multi-variable conditions, and reliable navigation and positioning support is ensured to be provided in a complex operation environment.

The invention discloses a children MPP (Mycoplasma Pneumoniae) auxiliary diagnosis system based on multi-modal time series data modeling, and aims to improve the auxiliary diagnosis, clinical typing and severe prediction capabilities of mycoplasma pneumonia and reduce the workload of doctors. The MPP dynamic representation method based on multi-modal time sequence modeling is innovatively provided, multi-modal data such as images, electronic medical records and inspection indexes are fused, the time sequence feature embedding technology is adopted, and the evolution trend of the disease course can be effectively captured. Furthermore, deep fusion of different modal data is realized by using a progressive cross-modal semantic alignment strategy and a modal attention gating mechanism, and the problem of insufficient diagnosis accuracy caused by incompatibility of data modals in a traditional method is solved.

Methods for counter-scaling windows and rearranging objects within those counter-scaled windows in a 3D computer-generated environment is disclosed. The windows can be counter-scaled in accordance with changes to their depth within the 3D computer-generated environment. Objects within the window can be presented at their actual size and with their correct scale relative to each other and with respect to the 3D computer-generated environment, and can maintain their actual size and scale relative to each other and the environment regardless of the depth of the window in the environment. Some embodiments of the disclosure rearrange the layout of the objects to present more or fewer objects in the window as the size of the window changes due to counter-scaling. Depth-dependent window resizing and object rearrangement while preserving actual object size and scale within the 3D computer-generated environment can provide a more authentic viewing experience and can encourage increased user interaction.

A device, system and method comprising computer vision techniques for fire prevention / detection and risk assessment, as well as for determining deviations from an ideal operational state. The present invention includes for example systems and methods which leverage data collected by camera systems composed of infrared and visible light sensors to detect and / or prevent a fire from starting, and additionally, use this data to determine a risk assessment for the building. The present invention also provides for example a system and method for monitoring and controlling safety risks in indoor industrial environments by determining deviations from an ideal operational state using computer vision techniques and game-theoretic competitive ranking frameworks.

The invention relates to the technical field of target detection, in particular to a robot three-dimensional environment sensing method and device based on deep visual learning, and the method comprises the steps: collecting visual data based on a sensing system carried by a robot, carrying out the synchronous processing, extracting the spatial structure characteristics of a synchronous visual data stream and a preliminary semantic segmentation map, and carrying out the recognition of a target image; combining the spatial structure features with the preliminary semantic segmentation map to generate a geometric semantic feature map; extracting and optimizing local, regional and global features of the geometric semantic feature map, and performing three-dimensional modeling processing according to a multi-scale feature tensor to generate a three-dimensional geometric model; and carrying out fusion optimization on the structure information of the three-dimensional geometric model and the geometric semantic feature map, and carrying out verification processing based on a verification framework to obtain three-dimensional environment perception data. Through deep visual learning and visual data fusion, the defects of insufficient accuracy and limited deep semantic understanding ability in complex three-dimensional environment perception in the prior art are solved.

The invention relates to the field of data processing, in particular to an emotion recognition method based on a large model and a related device. The method comprises the following steps: firstly, acquiring to-be-processed multi-modal data; the multi-modal data at least comprises text information, audio information and image information input by a user; the image information at least comprises a facial expression image and / or a limb action image. And inputting the multi-modal data into an emotion recognition model for emotion state analysis to obtain a predicted emotion state corresponding to the user. And finally, performing confidence evaluation based on the predicted emotional state, and generating a user emotion recognition result based on the predicted emotional state and a corresponding evaluation result. According to the method, richer emotion details are captured from the multi-modal data through the emotion recognition model, cross-modal emotion intelligent recognition is realized, the accuracy of emotion recognition is further improved by combining confidence evaluation, the emotion recognition efficiency is improved, and the application range of emotion recognition is expanded.

A holographic augmented reality ultrasound needle guide system and method includes an augmented reality display such as a headset wearable by a user. The augmented reality display is configured to depict a virtual ultrasound image. The augmented reality display is further configured to allow a user to select a desired reference point on the virtual ultrasound image. The system is configured to depict a holographic needle guide based on the selection of the desired reference point. The system is also configured to adjust a trajectory of the holographic needle guide to avoid intersecting undesired anatomical structures. The augmented reality display is further configured to stamp the holographic needle guide into a selectively locked trajectory and position.

A machine-learned architecture may predict multiple paths that an object could take in the future without regard to time at which the object may occupy positions identified by one of those paths. These time-invariant paths may be used by an autonomous vehicle to filter detected objects by relevance to an autonomous vehicle's plans, improve prediction of an object's reaction to a vehicle candidate trajectory, determine right-of-way between object(s) and the autonomous vehicle, match detected objects to lanes, and / or improve prediction of odd or out-of-turn object behavior of an object.

The invention relates to the technical field of data processing, in particular to a multi-modal data alignment method and system, and the method comprises the steps: collecting multi-modal data, and carrying out the preprocessing of the multi-modal data; the preprocessed multi-modal data are sent to an event anchor point detection module, and multi-modal emergency detection, anchor point marking and preliminary global alignment are carried out; performing time resampling and numerical value standardization on the multi-modal data after coarse alignment; mapping the resampled and standardized multi-modal data to a unified time axis; performing local fine alignment and global adjustment on the multi-modal feature sequence after unified time axis mapping; and performing confidence evaluation and anomaly correction on the fine alignment result, and outputting a final alignment sequence. According to the invention, a high-precision and multi-modal data real-time alignment method which is light in weight and has adaptive correction capability can be realized on end side equipment with limited resources.

The invention relates to the technical field of computer vision and multi-modal learning, in particular to a weak supervision target detection method guided by cross-modal pseudo labels. According to the method, a labeled source domain data set is constructed to train an image classification teacher model, and a teacher-student network structure is constructed; clustering the regional features of the target domain image, allocating pseudo tags to each cluster by optimizing the allocation cost between the source domain category and the target domain cluster, and constructing a pseudo tag pool; and training a student model on the pseudo label pool for region feature detection of the target domain image. According to the method, a cross-modal attention mechanism is introduced, so that more accurate semantic alignment between a source category label and a target domain feature is realized; the stability of label distribution is improved by a structure keeping regular term; the generalization ability of the model is further enhanced by multiple rounds of pseudo-label confidence learning. The method can be widely applied to tasks such as target detection, cross-domain transfer learning and open world recognition, and efficient and accurate weak supervision target detection is realized.

The invention belongs to the technical field of unmanned aerial vehicle intelligent navigation, and discloses a low-altitude economic unmanned aerial vehicle data processing method and system based on a large model, and the method comprises the steps: calculating the scene adaptation weight of multi-modal data through a dynamic attention mechanism based on a space-time alignment feature package, and carrying out the fusion to generate a multi-modal joint feature matrix; based on the multi-modal joint feature matrix, constructing a three-dimensional topological model of an urban airspace, predicting a dynamic obstacle trajectory in combination with a space-time diagram neural network, and generating a hierarchical navigation instruction set; the unmanned aerial vehicle executes a flight instruction according to the hierarchical navigation instruction set, and generates a flight state monitoring log by collecting data in flight of the unmanned aerial vehicle in real time; according to the method, the multi-modal joint feature matrix is generated through environment parameter driving weight distribution, and the complex scene sensing precision is remarkably improved.

The invention relates to an AI intelligent short video generation method and system based on multi-agent collaboration. The method comprises the following steps: S1, input analysis and sub-shot script generation; s2, obtaining materials; s3, sub-shot video generation; step S4: editing and synthesizing; wherein the plurality of sub-shot video clips are edited and synthesized into a complete video according to a script sequence, and style fusion processing is performed on the whole video by using a picture style unification algorithm; step S5, background music matching; and S6, outputting the slices. Through the structured sub-shot script generation, intelligent material completion, multi-shot style fusion and music synchronization technology, a professional short video of multi-scene coherent narrative can be generated, the structured sub-shot script is automatically generated, and intelligent material completion, picture style unification and music rhythm and emotion expression synchronization are realized.

The invention discloses a Transform-CNN medical image segmentation method and system based on multi-scale fusion semantic enhancement, and relates to the technical field of image segmentation, and the method comprises the steps: collecting medical image data, and constructing a medical image data set; constructing a medical image segmentation network model; training the medical image segmentation network model through the medical image data set; and inputting real-time acquired data into the trained medical image segmentation network model to obtain a medical image segmentation result. According to the invention, double encoders, LGDA modules and the like are adopted to capture multi-scale global features of kidney tumors and enhance representation, and the LGDA module is used for adapting to size and form changes of the kidney tumors; the MLCF module is used for supplementing information of the main encoder; the PSE module captures multi-scale global semantic information and integrates local context information, local features are fused through the SAM module, and kidney tumor positioning in an endoscope image is achieved.

The invention discloses a self-prompting semantic segmentation method and device based on SAM and a storage medium, and belongs to the technical field of image segmentation. The method comprises the following steps: acquiring a target image containing a target object; inputting the target image into a semantic segmentation model based on SAM to obtain a segmentation result of the target image; the segmentation result comprises the position and the type of the defect of the target object; wherein the semantic segmentation model comprises an image encoder, a target detector, a prompt encoder and a mask decoder; the target detector is used for positioning the position area of the defect of the target object on the target image according to the image features output by the image encoder, and outputting a defect detection frame; the prompt encoder is used for encoding the defect detection frame and generating position prompt characteristics; and the mask decoder is used for fusing the image features and the position prompt features and generating a segmentation result of the target image. According to the invention, end-to-end industrial defect detection can be realized, and the efficiency of industrial defect detection is improved.

The invention relates to the technical field of building appearance defect detection, in particular to a building appearance defect detection method and system based on an unmanned aerial vehicle, and the method comprises the steps: obtaining the building information of a target building, and generating a hierarchical scanning path and a three-dimensional obstacle avoidance flight path, a visible light image, an infrared thermodynamic diagram and laser radar point cloud information are collected for space-time alignment processing, and an attention mechanism neural network is used for extracting multi-scale features to generate a detection report containing defect three-dimensional coordinates, damage levels and safety risk assessment. The method achieves the purpose of efficiently and accurately detecting the building appearance defects, can adapt to complex building structures and environmental conditions, supports defect trend prediction and maintenance decision, and remarkably improves the building safety management efficiency.