1089 results about "Perception system" patented technology

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention provides a technical condition prediction method for a tunnel civil engineering structure, and relates to the technical field of traffic control monitoring systems.A biological acoustic composite sensing system comprising a distributed optical fiber acoustic sensing system and microorganism sample collection and analysis is constructed, and acoustic depth mode characteristics are mined by applying a deep learning model; the microbial dynamic biomarker is analyzed and identified by using bioinformatics; multi-modal information, a domain knowledge graph and a causal inference algorithm are fused, and a dynamic causal network model for revealing an internal driving relation in the degradation process is constructed; further, training and applying an adaptive neural network prediction model based on causal driving and fusing physical and biochemical mechanism constraints, and carrying out probabilistic prediction on future technical conditions of the tunnel; and executing anti-fact reasoning by using the causal network and the prediction model, and generating and optimizing an intervention strategy of cooperation of physical measures and biochemical measures. According to the method, the accuracy and advance of tunnel structure state prediction can be remarkably improved.

The invention relates to the technical field of target detection, in particular to a robot three-dimensional environment sensing method and device based on deep visual learning, and the method comprises the steps: collecting visual data based on a sensing system carried by a robot, carrying out the synchronous processing, extracting the spatial structure characteristics of a synchronous visual data stream and a preliminary semantic segmentation map, and carrying out the recognition of a target image; combining the spatial structure features with the preliminary semantic segmentation map to generate a geometric semantic feature map; extracting and optimizing local, regional and global features of the geometric semantic feature map, and performing three-dimensional modeling processing according to a multi-scale feature tensor to generate a three-dimensional geometric model; and carrying out fusion optimization on the structure information of the three-dimensional geometric model and the geometric semantic feature map, and carrying out verification processing based on a verification framework to obtain three-dimensional environment perception data. Through deep visual learning and visual data fusion, the defects of insufficient accuracy and limited deep semantic understanding ability in complex three-dimensional environment perception in the prior art are solved.

A sensor data evaluation device may include an artificial neural network, configured to receive and process first testing data, and output a first testing data output, wherein the first testing data represent sensor data of a first sensor detecting a sensor input; and one or more processors, configured to receive sensor deficiency information representing a deficiency of a second sensor; generate second testing data, wherein the second testing data are based on the sensor deficiency information; implement the artificial neural network to receive and process the second testing data; and output from the output layer a second testing data output; determine a difference between the first testing data output and the second testing data output; and if the difference is outside of one or more predetermined ranges or does not satisfy one or more predetermined criteria, send a signal representing a result of the determined difference.

The invention relates to the technical field of intelligent power grids, in particular to an intelligent power distribution room sensing system and method based on data fusion. The multi-source data acquisition module is used for synchronously acquiring electrical parameters, mechanical vibration signals, temperature distribution data and environment monitoring data of power equipment in a power distribution room; the edge computing gateway is connected to the multi-source data acquisition module and is configured to perform time alignment, abnormal value elimination and feature extraction on the original sensing data; the data fusion analysis server is connected to the edge computing gateway through a network and comprises a space-time alignment unit used for unifying monitoring data of different sampling frequencies to the same time reference; the self-adaptive weight fusion unit is used for dynamically adjusting fusion weight according to the reliability of the data of each sensor; the state evaluation unit is used for generating an equipment health degree score and a fault early warning signal based on a fusion result; according to the scheme, monitoring blind areas and misjudgment risks caused by data islands can be fundamentally solved.

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

The invention discloses an electromechanical equipment health assessment and early warning method based on multi-mode dynamic perception, and belongs to the field of intelligent operation and maintenance of electromechanical equipment. The problems that in the prior art, a single physical quantity cannot comprehensively reflect the equipment state and a traditional signal processing algorithm cannot adapt to the equipment degradation mode change are solved, a panoramic sensing system covering multiple physical fields such as vibration, temperature and noise is constructed through a multi-mode sensor network and a dynamic weight fusion algorithm, and the multi-physical-field multi-physical-field panoramic sensing method is applied to the multi-physical-field multi-physical-field panoramic sensing system. The problem of isolated island of traditional single-dimensional monitoring information is solved; through a physical-depth mixed feature extraction architecture, combining interpretable engineering features with abstract features extracted by a deep neural network to form a health assessment model with mechanism transparency and mode generalization ability; through deep integration of the digital twin platform and the RPA technology, the manual inspection frequency and workload are reduced, the fault recognition accuracy is promoted to increase year by year, and continuously optimized intelligent operation and maintenance ecology is formed.

Techniques for clustering sensor data are discussed herein. Sensors of a vehicle may detect data points in an environment. Clustering techniques can be used in a vehicle safety system to determine connection information between the data points. Further, the machine learned model can be trained to output motion data representing velocity data or a cluster association score. The connection information and / or motion data can be used by a vehicle computing device to detect objects in an environment and / or to control operation of a vehicle.

The technology relates to detection of a nearby occluded object based on illumination emitted from that object. Illumination by the occluded object of one or more areas in the surrounding area, for instance by headlights of an occluded vehicle, is detected by a perception system of a self-driving vehicle. The self-driving vehicle can classify the detected object to determine whether the illumination is caused by a vehicle or other road user, or from objects in the surrounding environment. Illumination data and other information can be evaluated by the self-driving vehicle, for instance to identify a type of the object, a location of the object along a roadway, to disambiguate the direction of travel of the other object, etc. As a result, the self-driving vehicle may infer the behavior of the other object and modify its own driving operations to account for the other object's presence and likely behavior.

The embodiment of the invention provides a multi-modal fusion deep learning analysis method and system. The method is applied to the technical field of multi-modal learning, and comprises the following steps: obtaining multi-modal original data, sequentially processing image, text, audio and video data, and extracting visual features of the image, semantic features of the text, frequency spectrum and time sequence features of the audio, image features and time sequence features of a video frame and time domain features of an audio sequence; and then, according to the complementary information of the multi-source features, fusion processing is carried out to form a unified multi-modal feature representation, the unified multi-modal feature representation is input to a preset deep learning analysis model, and finally a multi-modal analysis result of comprehensive expression is obtained. According to the scheme, information complementarity and robustness are enhanced through multi-modal feature fusion, the comprehensive analysis capability of the model on semantic understanding, behavior recognition and state judgment in a complex scene is remarkably improved, and a more accurate, efficient and stable decision basis is provided for a multi-modal intelligent sensing system.

The invention discloses a science and technology fast message intelligence perception system based on a large language model, and belongs to the field of intelligence perception, the system comprises an information collection module, a data standardization module and a database, the information collection module obtains multi-mode science and technology fast message data; the data standardization module is used for receiving the multi-modal data output by the information collection module and executing data cleaning, structured processing and semantic standardization operation; after standardization processing, storing the data into a database; the system further comprises an intelligence value screening module, an evaluation system module and a visualization module. And the intelligence value screening module is combined with the evaluation system to identify the intelligence value, and multi-dimensional visual presentation is realized through the visualization module. The method improves the efficiency and accuracy of information recognition and perception, and has significant advantages in the aspects of processing mass information and enhancing the information perception ability.

The invention provides a scene adaptive projection vehicle lamp system based on deep reinforcement learning and a control method, and relates to the technical field of intelligent vehicle lamps and automatic driving perception systems. Comprising a multi-modal sensing module, a feature fusion module, a strategy generation module and an execution module. The multi-mode sensing module is used for collecting environment state data and performing primary processing to form an environment data information flow; the feature fusion module is used for generating a unified environment feature vector for the environment data information flow; the strategy generation module is used for receiving the environment feature vector and generating a vehicle lamp adjustment strategy through multi-layer neural network calculation; evaluating a result obtained by executing the vehicle lamp adjustment strategy based on the vehicle lamp, and optimizing strategy parameters of the strategy network based on a PPO algorithm; the execution module is used for controlling the vehicle lamp according to the vehicle lamp adjustment strategy output by the strategy network. The intelligent level of the vehicle lamp is remarkably improved, and the system is widely applied to night driving assistance, urban interaction prompt and low-visibility driving scenes.

The invention belongs to the field of multi-mode sensing fusion, and relates to an intelligent multi-mode patrol sensing method, system and equipment and a storage medium, which are used for carrying out automatic patrol, real-time sensing and abnormity warning in a complex environment. Comprising the steps of natural language instruction analysis, task decomposition and distribution, intelligent agent scheduling and execution, multi-modal information collection and fusion, anomaly recognition and event response, result feedback and task closed loop, and through the integration of an OWL framework and a DeepSeek-R1 large model, a multi-modal data fusion technology and an NL-SLAM autonomous navigation algorithm, the multi-modal data fusion algorithm and the multi-modal data fusion technology are integrated. The problems that in the prior art, dynamic task understanding based on natural languages cannot be achieved, a sensing system lacks efficient multi-modal data fusion, so that the anomaly recognition precision is low, and complex tasks or emergencies are difficult to deal with are solved, various abnormal conditions in a complex environment are effectively dealt with, and the method is suitable for popularization and application. The stability and expansibility of task execution are improved, and the navigation success rate and the task completion rate of the system in an unstructured environment are enhanced.

The invention relates to the technical field of body-equipped intelligent robots and intelligent equipment, in particular to a self-adaptive flexible tightening body-equipped intelligent robot which comprises a body-equipped moving platform, the top of the body-equipped moving platform is provided with a body-equipped lifting mechanism, and the top of the body-equipped moving platform is provided with a body-equipped articulated arm module. And a tightening actuator and a clamping jaw actuator are arranged at the tail end of the outer part of the body articulated arm module. According to the self-adaptive flexible tightening intelligent robot and the tightening process method, sensing and self-adaptive adjustment of external environment changes are achieved through multi-module integration, and the tightening process method based on a multi-mode sensing system and an intelligent control system is constructed; vision, force sense and touch sense data are fused, real-time monitoring and dynamic adjustment of the tightening process are achieved through a machine learning algorithm, and the intelligent level and quality stability of tightening operation are effectively improved.

The invention belongs to the technical field of artificial intelligence and intelligent sensing, particularly relates to a multi-source sensor fusion sensing system based on adaptive noise suppression, and aims to solve the problems of noise interference, modal mismatch and insufficient robustness in multi-source sensor fusion in a complex dynamic environment. The system comprises a front-end preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core and a closed-loop feedback optimization mechanism. Through real-time noise modeling and dynamic weight adjustment, high-precision alignment and fusion of multi-source signals are realized, and the sensing stability and real-time performance in an extreme scene are significantly improved.

In various examples, systems and methods of the present disclosure correspond to speed limit fusion for automotive systems and applications. Systems and methods are disclosed that fuse speed limit data from both one or more map sources (e.g., nav-maps, HD maps, SD maps, etc.) and a perception system, and that employ rules to help resolve conflicts in speed limits from the perception system and the map(s). For example, if a conflict occurs, the rules may indicate whether to use the speed limit data from the perception data, the speed limit data from the map(s), and / or whether to use another type of speed limit data, such as cached speed limit data.

The invention relates to the field of industrial automation, in particular to a multi-camera vision stacking pose planning method and system for an intelligent loading and unloading robot. The method comprises the following steps: constructing a collaborative sensing system through a plurality of industrial cameras deployed in a target space, and collecting dynamic environment data of a stacking area in real time; generating a self-adaptive stacking pose based on a hierarchical pose planning mechanism: analyzing an initial pose parameter through a theme control layer and generating a preliminary placement instruction, and verifying the feasibility of the preliminary placement instruction based on environmental data of the collaborative awareness system through a visual verification layer, and a final placement coordinate is optimized by combining the accurate positioning layer with a virtual box matching algorithm. The problems that in the prior art, real-time environment sensing capacity is lacked, pose planning is disjointed with the actual environment, and dynamic change scenes cannot be processed are solved.

The invention relates to the technical field of battery fault detection, and particularly discloses a new energy automobile battery fault intelligent diagnosis method and system, and the method comprises the steps: collecting the three-dimensional time sequence data flow of the voltage, temperature and internal resistance of a battery pack through a distributed sensor array; the environment temperature, the number of charge and discharge cycles and the vehicle operation condition parameters are synchronously integrated as auxiliary diagnosis dimensions, and a multi-source heterogeneous data sensing system is constructed; the method comprises the following steps: preprocessing original data by adopting a dual-channel hybrid filtering architecture based on a multi-source heterogeneous data sensing system, and establishing an adaptive filtering parameter adjustment mechanism to generate multi-modal data; according to the method, multi-dimensional time sequence data and auxiliary diagnosis parameters of the battery pack are comprehensively collected through a constructed multi-source heterogeneous data sensing system, a rich and accurate data basis is provided for fault diagnosis, noise interference is reduced through a two-channel hybrid filtering architecture and a self-adaptive filtering parameter adjustment mechanism, and the fault diagnosis accuracy is improved. And a solid foundation is laid for subsequent feature extraction and model training.

The present invention provides an intelligent construction control method, apparatus, and system for splitting jet grouting mixing piles, relating to geotechnical engineering. The method includes receiving real-time perception information from a construction information self-perception system, which includes drilling depth, drilling rig output power, drill bit torque, drill rod axial force, and pore water pressure of the drill bit. This information is input into a trained machine-learning model for stratum information, establishing a nonlinear implicit correspondence relationship between the perception information and construction stratum conditions, and outputting real-time stratum state information. The real-time stratum state information is then input into a trained self-matching machine-learning model for optimal construction parameters, establishing a nonlinear implicit correspondence relationship between the stratum state information and optimal construction parameters, and outputting current optimal construction parameters. This invention enables real-time intelligent inversion of stratum information and dynamic self-optimization control of construction parameters.

The invention discloses a unified framework for solving automatic driving track prediction and planning consistency based on a world model. According to the method, through cooperative work of the automatic driving domain controller and the vehicle-mounted sensing system, end-to-end joint optimization of track prediction and planning in a complex traffic scene is realized, time sequence dependence and interaction dynamics among intelligent agents are accurately captured, and the prediction capability and robustness of a model are remarkably improved. The method comprises the following specific steps: firstly, constructing a generative world model, and generating potential future state representation by utilizing a behavior conditional and backtracking expansion technology; secondly, in combination with global modeling and a local convolutional network, multi-scale features are extracted, adaptive fusion is carried out, and a multi-modal prediction trajectory is generated; then, a multi-target planning model is adopted to integrate various driving indexes, and a track with the minimum loss function is generated; finally, path planning parameters are dynamically optimized through real-time environment perception and decision feedback, and the problems of prediction uncertainty and planning consistency of the automatic driving track are effectively solved.

The invention discloses a driver state sensing system based on physiological indexes and external behavior analysis, which relates to the technical field of vehicle active safety control, and comprises a sensor group integrated in a driver contact or close area and a vehicle control part and used for collecting physiological signals and behavior characteristic signals of a driver in real time, the physiological signals at least comprise heart rate and skin electric signals, and the behavior characteristic signals at least comprise eyelid movement, head posture and steering wheel operation signals. According to the driver state sensing system, the comprehensiveness and accuracy of driver state sensing are remarkably improved by integrating physiological indexes and external behavior analysis, the limitation of single signal judgment is effectively overcome through a multi-source signal fusion mechanism, the reliability of state evaluation is ensured, and the advanced signal preprocessing technology adopted by the system is high in reliability. The data anti-interference capability is enhanced, the feature extraction precision is improved, and the robustness of the model is enhanced.

A perception system may be used to generate bounding boxes for objects in a vehicle scene. The perception system may receive images and feature maps corresponding to the received images. The perception system may correlate object queries from previous time steps with object queries from the current time step.

The invention discloses an intelligent sensing system for multi-modal data fusion, and relates to the technical field of intelligent traffic. The multi-modal data fusion intelligent sensing system comprises a data acquisition and preprocessing module, a multi-modal data fusion module, an intelligent sensing module, a target detection and behavior prediction module and a system verification and optimization module, and high-precision and high-stability target sensing and tracking are realized by integrating millimeter wave radar and camera data. The multi-modal data fusion module effectively combines physical features and visual features, and improves the accuracy and robustness of perception. The intelligent sensing module adopts a volume Kalman filter and a Hungary algorithm, state estimation and target association are optimized, and continuous and accurate tracking of the target is ensured. The target detection and behavior prediction module utilizes an advanced YOLOv5 model and a DeepSORT algorithm to realize multi-target accurate detection and trajectory prediction, and the intelligent level of the system is further improved.

The invention relates to a four-way shuttle vehicle based on ontology state perception and a self-calibration positioning method of the four-way shuttle vehicle. The method comprises the following steps: establishing a body state sensing system, collecting three-axis acceleration, three-axis angular velocity, wheel set displacement and attitude angle data of a shuttle vehicle body, and synchronously forming a standardized sensing data set; performing fusion calculation on the standardized sensing data set to obtain real-time position and attitude parameters; calculating an initial value of a positioning error based on matching of the real-time position and attitude parameters with pre-stored high-precision track map data; according to the positioning error initial value, adaptive correction calculation is executed, a real-time positioning error correction amount is determined, and positioning information is corrected; parameters of a shuttle vehicle track control system are updated according to the corrected positioning information, and track navigation control is carried out; accumulated positioning error correction data are periodically analyzed, self-calibration model parameters are optimized, and positioning accuracy and stability are improved. According to the invention, the positioning reliability and path tracking capability of the four-way shuttle vehicle in a complex environment can be improved.

The invention discloses a bus and station interactive scheduling method and system based on V2X and deep reinforcement learning decision, a medium and equipment, and the method comprises the steps: collecting bus operation dynamic data, station passenger flow and environment data in real time, and combining V2X network interaction information to construct a full-dimension perception system; a deep reinforcement learning model is adopted for dynamic decision making, intelligent closed-loop control of vehicle scheduling, station service and traffic signal cooperation is achieved, the bus punctuality rate is remarkably increased, the waiting time of passengers is shortened, operation safety is guaranteed through active early warning of abnormal conditions, and intelligent upgrading of a bus system from passive response to active prevention is achieved.

The invention relates to the field of intelligent agricultural robots, in particular to a multi-robot collaborative harvesting method and system for woody oil plants based on swarm intelligence, and provides efficient harvesting of the woody oil plants by constructing a distributed sensing network and a semantic enhanced digital twinborn model, and the system adopts a three-layer decision architecture to formulate a harvesting strategy. The working efficiency is improved through self-adaptive task allocation and collaborative harvesting, multiple robots share experience through federated learning, group collaborative learning is achieved, harvesting performance is optimized, the reliability of the system is improved by 85%, single-point fault risks do not exist, the recognition accuracy is improved by 50% through a multi-modal sensing system, and the system is suitable for popularization and application. And an innovative harvesting solution is provided for the field of intelligent agricultural robots.

The invention is applicable to the field of intelligent audio adjustment, and provides an immersive audio-video follow-up adjustment method and system, and the method comprises the steps: constructing a multi-dimensional perception system, and collecting multi-source information in real time; carrying out fusion processing on the collected multi-source information based on a deep neural network model, mining a dynamic mapping relation between a user state and the video content through space-time correlation analysis, and identifying a user interaction intention and an emotional tone and a space scene attribute of the video content; according to a fusion processing result, calling a dynamic parameter adjustment engine, and generating an audio parameter adjustment scheme in real time; a user experience feedback closed loop is constructed, a visual attention area of a user is collected through eye movement tracking equipment, and personalized adjustment preference parameters are generated; according to the method and the device, the audio effect is accurately matched with the user state and the audio and video content, the naturalness and the adaptability of immersive experience are remarkably improved, universality and individual differences are considered, and the audio experience which is more suitable for scenes and needs of the user is brought to the user.

The invention discloses an airport hub construction intelligent regulation and control method and system based on BIM + 5G, and relates to the technical field of construction scheduling optimization, and the method comprises the steps: collecting on-site real-time data to construct a path construction sequence model, constructing a dynamic construction state set with consistent space and time sequence through component coding and semantic attribute mapping, and constructing a dynamic construction state set with consistent time sequence; and constructing a minimum disturbance optimization algorithm of four-dimensional disturbance cost based on the construction disturbance mapping graph, outputting procedure sequence adjustment, resource rearrangement, path decoupling and environment avoidance intervention suggestions, and tracking a construction response state in real time based on an intervention execution feedback mechanism. According to the method, unified modeling of construction plans, resource allocation and green indexes is achieved by constructing a ternary structure and a construction disturbance mapping graph, the field state is dynamically collected in combination with a 5G and edge sensing system, process abnormity and resource conflicts are accurately recognized, an efficient and executable regulation and control strategy is generated based on a multi-target disturbance optimization algorithm, and the efficiency and the reliability of the system are improved. And the intelligence, responsiveness and energy-saving level of the construction process are obviously improved.

The invention relates to the field of campus safety prevention and control, and discloses an AI campus intelligent prevention system based on intelligent perception and automatic response early warning, and the system comprises a data collection and perception module which is used for collecting multi-dimensional data in a campus in real time through a collection device; the event perception and behavior identification module and the multi-modal data fusion module are used for carrying out fusion analysis on data from different perception devices; the AI analysis and event research and judgment module, the event early warning and notification module, the emergency response and disposal module and the resource scheduling and management module are used for reasonable allocation and efficient use of resources; the event recording and data archiving module is used for carrying out detailed recording and archiving on each processed event; and a disaster emergency response and auxiliary system connection module. By constructing a video, audio and environment data three-in-one multi-source sensing system, comprehensive real-time monitoring of abnormal behaviors, sound and environment changes in a campus is realized, and the effects of improving event recognition accuracy and sensing coverage are achieved.

The invention provides a high-frequency welded pipe roller wear state monitoring method and device, and relates to the technical field of artificial intelligence and data processing, a holographic sensing system of working condition parameters and physical signals is constructed through synchronous monitoring of a multi-source sensor, and noise spectrum offset is tracked in real time based on time-frequency expression of data in different modes, so that the real-time monitoring of the wear state of a high-frequency welded pipe roller is realized. And an anti-error filtering protection channel is constructed, so that the inherent characteristics of the damage can be effectively captured. In addition, multi-physical field damage characterization information is fused from multiple complementary dimensions of transient impact (instantaneous frequency vector), spectrum complexity (spectral entropy) and amplitude distribution deflection (skewness), more comprehensive and richer state description can be provided, accurate quantitative evaluation of the roller wear level is achieved, and the capacity of real-time monitoring and accurate recognition of the roller wear state is remarkably improved.