1263 results about "Perception system" patented technology

The invention discloses an exhibition hall robot visual language navigation method based on a large model, belongs to the field of exhibition hall robots, and aims at solving the problems existing in an existing exhibition hall robot. According to the method, an exhibition hall robot multi-mode scene sensing system senses a current scene, and candidate target objects are obtained through fusion features of RGB images and laser point clouds; extracting image features of all candidate target objects in the scene; according to the exhibition hall robot multi-mode environment understanding system, an exhibition hall robot receives a voice instruction, a visual language multi-mode large model is used for understanding a target object and a user intention, then high-dimensional semantic features of a target object text are extracted from the system, and the target object is found out from candidate target objects; according to the exhibition hall robot intelligent navigation system based on the multi-modal large model, an exhibition hall robot carries out path planning and autonomous movement according to a pose sequence generated by an intention, and self behaviors are dynamically adjusted according to a real-time changing scene provided by the exhibition hall robot multi-modal scene sensing system.

Disclosed are a high-performance loosely-coupled multi-modal data fusion system for a smart driving environmental perception system and a vehicle-mounted device, comprising: a fusion detection model based on a modality-independent feature interaction strategy, which is configured for converting a LiDAR point cloud, a camera image, and a millimeter-wave radar point cloud into a unified bird's-eye view representation, and performing multi-modal fusion; and a fusion tracking model based on a motion-appearance feature cascaded coupling data association strategy, which is configured for performing subsequent trajectory tracking and matching according to multi-modal fusion feature information. A VoD data set and a K-Radar data set are selected for training, verifying, and testing the comprehensive performance of the models, and a TensorRT accelerated inference model is applied, then quantized, and deployed to a vehicle-mounted computational testing platform. The present invention is compatible with mainstream sensor deployment solutions, and achieves the efficient complementary fusion of multi-source heterogeneous sensor information, significantly improving the reliability, accuracy, and adaptability of vehicle-mounted perception systems, thereby effectively responding to extreme operating conditions such as complex traffic scenarios and inclement weather.

The invention provides a technical condition prediction method for a tunnel civil engineering structure, and relates to the technical field of traffic control monitoring systems.A biological acoustic composite sensing system comprising a distributed optical fiber acoustic sensing system and microorganism sample collection and analysis is constructed, and acoustic depth mode characteristics are mined by applying a deep learning model; the microbial dynamic biomarker is analyzed and identified by using bioinformatics; multi-modal information, a domain knowledge graph and a causal inference algorithm are fused, and a dynamic causal network model for revealing an internal driving relation in the degradation process is constructed; further, training and applying an adaptive neural network prediction model based on causal driving and fusing physical and biochemical mechanism constraints, and carrying out probabilistic prediction on future technical conditions of the tunnel; and executing anti-fact reasoning by using the causal network and the prediction model, and generating and optimizing an intervention strategy of cooperation of physical measures and biochemical measures. According to the method, the accuracy and advance of tunnel structure state prediction can be remarkably improved.

The present invention discloses a method and system for signal detection and remote early warning of wireless base stations, comprising the following steps: collecting environmental factors and signal quality data of multiple base stations in real time through a dynamic environment perception system, combining historical signal data, comprehensively evaluating the signal health status, and generating a multi-dimensional analysis report on the signal health status, using a fuzzy logic algorithm to accurately predict signal failures and attenuation trends, and dynamically adjusting the signal early warning threshold to avoid false alarms and missed alarms caused by fixed thresholds, realizing signal sharing and collaborative analysis between base stations through an intelligent collaborative detection mechanism, optimizing signal health management through cross-base station linkage, identifying potential signal failure areas in advance and optimizing response strategies, using a remote monitoring and repair platform, creating an intelligent operation and maintenance system through virtualization technology and intelligent algorithms, simulating fault repair scenarios and optimizing repair paths, and the system can adaptively learn and continuously optimize signal quality prediction and risk control capabilities.

The invention relates to the technical field of target detection, in particular to a robot three-dimensional environment sensing method and device based on deep visual learning, and the method comprises the steps: collecting visual data based on a sensing system carried by a robot, carrying out the synchronous processing, extracting the spatial structure characteristics of a synchronous visual data stream and a preliminary semantic segmentation map, and carrying out the recognition of a target image; combining the spatial structure features with the preliminary semantic segmentation map to generate a geometric semantic feature map; extracting and optimizing local, regional and global features of the geometric semantic feature map, and performing three-dimensional modeling processing according to a multi-scale feature tensor to generate a three-dimensional geometric model; and carrying out fusion optimization on the structure information of the three-dimensional geometric model and the geometric semantic feature map, and carrying out verification processing based on a verification framework to obtain three-dimensional environment perception data. Through deep visual learning and visual data fusion, the defects of insufficient accuracy and limited deep semantic understanding ability in complex three-dimensional environment perception in the prior art are solved.

A sensor data evaluation device may include an artificial neural network, configured to receive and process first testing data, and output a first testing data output, wherein the first testing data represent sensor data of a first sensor detecting a sensor input; and one or more processors, configured to receive sensor deficiency information representing a deficiency of a second sensor; generate second testing data, wherein the second testing data are based on the sensor deficiency information; implement the artificial neural network to receive and process the second testing data; and output from the output layer a second testing data output; determine a difference between the first testing data output and the second testing data output; and if the difference is outside of one or more predetermined ranges or does not satisfy one or more predetermined criteria, send a signal representing a result of the determined difference.

The invention relates to the technical field of intelligent power grids, in particular to an intelligent power distribution room sensing system and method based on data fusion. The multi-source data acquisition module is used for synchronously acquiring electrical parameters, mechanical vibration signals, temperature distribution data and environment monitoring data of power equipment in a power distribution room; the edge computing gateway is connected to the multi-source data acquisition module and is configured to perform time alignment, abnormal value elimination and feature extraction on the original sensing data; the data fusion analysis server is connected to the edge computing gateway through a network and comprises a space-time alignment unit used for unifying monitoring data of different sampling frequencies to the same time reference; the self-adaptive weight fusion unit is used for dynamically adjusting fusion weight according to the reliability of the data of each sensor; the state evaluation unit is used for generating an equipment health degree score and a fault early warning signal based on a fusion result; according to the scheme, monitoring blind areas and misjudgment risks caused by data islands can be fundamentally solved.

The embodiment of the invention provides an unhooking robot control method based on multi-source visual fusion, which is applied to the technical field of robot control and comprises the following steps: acquiring an RGB image, a depth image, an infrared image and IMU data through a multi-source sensing system mounted at the tail end of a robot; carrying out feature fusion identification by adopting a double-branch neural network, and outputting the boundary contour of the lifting hook and the three-dimensional coordinates of the optimal grabbing point; the visual coordinates are unified to a robot base coordinate system through a registration correction mechanism; a Transform prediction model is constructed based on the visual and inertial signals, and future pose changes of the lifting hook are estimated; a feedforward control track is generated to counteract swing of the lifting hook, and track correction is carried out in combination with visual servo feedback; and a joint instruction is generated through path planning and inverse kinematics solution, and the mechanical arm is driven to complete precise unhooking operation. According to the method, the recognition precision, the anti-interference capability and the operation success rate of unhooking operation in complex illumination and dynamic environments are effectively improved.

Techniques for updating data operations in a perception system are discussed herein. A vehicle may use a perception system to capture data about an environment proximate to the vehicle. The perception system may output the data about the environment to a system configured to determine positions of objects relative to the perception system over time. The positions of the objects may be used to estimate an object velocity and may be compared against machine learning model outputs in a self-supervised manner to train the machine learning model to output object velocities based on inputs from the perception system. The output of the machine learning model may include a two-dimensional velocity for objects in the environment. The two-dimensional velocity may be used for a vehicle system such that the vehicle can make environmentally aware operational decisions, which may improve reaction time(s) and / or safety outcomes of the vehicle.

The invention discloses an electromechanical equipment health assessment and early warning method based on multi-mode dynamic perception, and belongs to the field of intelligent operation and maintenance of electromechanical equipment. The problems that in the prior art, a single physical quantity cannot comprehensively reflect the equipment state and a traditional signal processing algorithm cannot adapt to the equipment degradation mode change are solved, a panoramic sensing system covering multiple physical fields such as vibration, temperature and noise is constructed through a multi-mode sensor network and a dynamic weight fusion algorithm, and the multi-physical-field multi-physical-field panoramic sensing method is applied to the multi-physical-field multi-physical-field panoramic sensing system. The problem of isolated island of traditional single-dimensional monitoring information is solved; through a physical-depth mixed feature extraction architecture, combining interpretable engineering features with abstract features extracted by a deep neural network to form a health assessment model with mechanism transparency and mode generalization ability; through deep integration of the digital twin platform and the RPA technology, the manual inspection frequency and workload are reduced, the fault recognition accuracy is promoted to increase year by year, and continuously optimized intelligent operation and maintenance ecology is formed.

Techniques for clustering sensor data are discussed herein. Sensors of a vehicle may detect data points in an environment. Clustering techniques can be used in a vehicle safety system to determine connection information between the data points. Further, the machine learned model can be trained to output motion data representing velocity data or a cluster association score. The connection information and / or motion data can be used by a vehicle computing device to detect objects in an environment and / or to control operation of a vehicle.

The technology relates to detection of a nearby occluded object based on illumination emitted from that object. Illumination by the occluded object of one or more areas in the surrounding area, for instance by headlights of an occluded vehicle, is detected by a perception system of a self-driving vehicle. The self-driving vehicle can classify the detected object to determine whether the illumination is caused by a vehicle or other road user, or from objects in the surrounding environment. Illumination data and other information can be evaluated by the self-driving vehicle, for instance to identify a type of the object, a location of the object along a roadway, to disambiguate the direction of travel of the other object, etc. As a result, the self-driving vehicle may infer the behavior of the other object and modify its own driving operations to account for the other object's presence and likely behavior.

The embodiment of the invention provides a multi-modal fusion deep learning analysis method and system. The method is applied to the technical field of multi-modal learning, and comprises the following steps: obtaining multi-modal original data, sequentially processing image, text, audio and video data, and extracting visual features of the image, semantic features of the text, frequency spectrum and time sequence features of the audio, image features and time sequence features of a video frame and time domain features of an audio sequence; and then, according to the complementary information of the multi-source features, fusion processing is carried out to form a unified multi-modal feature representation, the unified multi-modal feature representation is input to a preset deep learning analysis model, and finally a multi-modal analysis result of comprehensive expression is obtained. According to the scheme, information complementarity and robustness are enhanced through multi-modal feature fusion, the comprehensive analysis capability of the model on semantic understanding, behavior recognition and state judgment in a complex scene is remarkably improved, and a more accurate, efficient and stable decision basis is provided for a multi-modal intelligent sensing system.

The invention discloses a science and technology fast message intelligence perception system based on a large language model, and belongs to the field of intelligence perception, the system comprises an information collection module, a data standardization module and a database, the information collection module obtains multi-mode science and technology fast message data; the data standardization module is used for receiving the multi-modal data output by the information collection module and executing data cleaning, structured processing and semantic standardization operation; after standardization processing, storing the data into a database; the system further comprises an intelligence value screening module, an evaluation system module and a visualization module. And the intelligence value screening module is combined with the evaluation system to identify the intelligence value, and multi-dimensional visual presentation is realized through the visualization module. The method improves the efficiency and accuracy of information recognition and perception, and has significant advantages in the aspects of processing mass information and enhancing the information perception ability.

The invention provides a scene adaptive projection vehicle lamp system based on deep reinforcement learning and a control method, and relates to the technical field of intelligent vehicle lamps and automatic driving perception systems. Comprising a multi-modal sensing module, a feature fusion module, a strategy generation module and an execution module. The multi-mode sensing module is used for collecting environment state data and performing primary processing to form an environment data information flow; the feature fusion module is used for generating a unified environment feature vector for the environment data information flow; the strategy generation module is used for receiving the environment feature vector and generating a vehicle lamp adjustment strategy through multi-layer neural network calculation; evaluating a result obtained by executing the vehicle lamp adjustment strategy based on the vehicle lamp, and optimizing strategy parameters of the strategy network based on a PPO algorithm; the execution module is used for controlling the vehicle lamp according to the vehicle lamp adjustment strategy output by the strategy network. The intelligent level of the vehicle lamp is remarkably improved, and the system is widely applied to night driving assistance, urban interaction prompt and low-visibility driving scenes.

The described aspects and implementations enable efficient and seamless tracking of objects in vehicle environments using different sensing modalities across a wide range of distances. A perception system of a vehicle deploys an object tracking pipeline with a plurality of models that include a camera model trained to perform, using camera images, object tracking at distances exceeding a lidar sensing range, a lidar model trained to perform, using lidar images, object tracking at distances within the lidar sensing range, and a camera-lidar model trained to transfer, using the camera images and the lidar images, object tracking from the camera model to the lidar model.

The invention discloses an unmanned aerial vehicle target intelligent identification and perception system and method based on deep learning, and relates to the technical field of artificial intelligence, and the method comprises the steps: extracting the importance feature of each target region, calculating a region importance score, generating a target region monitoring plan, and obtaining continuous frame images; an improved YOLOv5 model is adopted, continuous frames of images are input, a bounding box of a suspected target wild animal is output and mapped to an actual geographic coordinate system, and actual position coordinates are obtained; key points of the t-th frame image are detected, coordinates of the key points are extracted, visibility scores and confidence coefficients are calculated, and feature descriptors are formed; performing target detection and key point detection on the (t + 1) th frame of image to obtain actual position coordinates and feature descriptors of candidate targets, performing track matching, updating the actual position coordinates and the feature descriptors of the suspected target wild animals if the actual position coordinates and the feature descriptors are matched, and defining new target animals if the actual position coordinates and the feature descriptors are not matched; counting according to the track matching result to obtain the total number of the target wild animals.

The invention belongs to the field of multi-mode sensing fusion, and relates to an intelligent multi-mode patrol sensing method, system and equipment and a storage medium, which are used for carrying out automatic patrol, real-time sensing and abnormity warning in a complex environment. Comprising the steps of natural language instruction analysis, task decomposition and distribution, intelligent agent scheduling and execution, multi-modal information collection and fusion, anomaly recognition and event response, result feedback and task closed loop, and through the integration of an OWL framework and a DeepSeek-R1 large model, a multi-modal data fusion technology and an NL-SLAM autonomous navigation algorithm, the multi-modal data fusion algorithm and the multi-modal data fusion technology are integrated. The problems that in the prior art, dynamic task understanding based on natural languages cannot be achieved, a sensing system lacks efficient multi-modal data fusion, so that the anomaly recognition precision is low, and complex tasks or emergencies are difficult to deal with are solved, various abnormal conditions in a complex environment are effectively dealt with, and the method is suitable for popularization and application. The stability and expansibility of task execution are improved, and the navigation success rate and the task completion rate of the system in an unstructured environment are enhanced.

The invention relates to a collaborative autonomous perception and navigation method in a denial environment, which comprises the following steps: constructing a self-organizing robust communication system to realize dynamic networking and data sharing among multiple unmanned aerial vehicles in the denial environment, and adopting a self-adaptive network protocol to realize automatic configuration of a communication network under network topology change or communication interference; based on a multi-sensor fusion sensing system, environment information and motion information of the unmanned aerial vehicle are processed through a data fusion algorithm; a single unmanned aerial vehicle constructs a reward function based on unmanned aerial vehicle energy consumption, an exploration area, a path overlapping degree and a path danger degree, executes autonomous perception and exploration in an unknown environment and completes mapping; the distributed multiple unmanned aerial vehicles carry out individual communication and game playing in the unmanned aerial vehicle cluster according to map information, respective positions and working states, task distribution is achieved, and path planning and action decision making are completed. According to the invention, autonomy and collaboration of the unmanned aerial vehicle in a denial environment are improved, and an effective solution is provided for navigation in a complex environment.

The invention relates to the technical field of body-equipped intelligent robots and intelligent equipment, in particular to a self-adaptive flexible tightening body-equipped intelligent robot which comprises a body-equipped moving platform, the top of the body-equipped moving platform is provided with a body-equipped lifting mechanism, and the top of the body-equipped moving platform is provided with a body-equipped articulated arm module. And a tightening actuator and a clamping jaw actuator are arranged at the tail end of the outer part of the body articulated arm module. According to the self-adaptive flexible tightening intelligent robot and the tightening process method, sensing and self-adaptive adjustment of external environment changes are achieved through multi-module integration, and the tightening process method based on a multi-mode sensing system and an intelligent control system is constructed; vision, force sense and touch sense data are fused, real-time monitoring and dynamic adjustment of the tightening process are achieved through a machine learning algorithm, and the intelligent level and quality stability of tightening operation are effectively improved.

The invention belongs to the technical field of artificial intelligence and intelligent sensing, particularly relates to a multi-source sensor fusion sensing system based on adaptive noise suppression, and aims to solve the problems of noise interference, modal mismatch and insufficient robustness in multi-source sensor fusion in a complex dynamic environment. The system comprises a front-end preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core and a closed-loop feedback optimization mechanism. Through real-time noise modeling and dynamic weight adjustment, high-precision alignment and fusion of multi-source signals are realized, and the sensing stability and real-time performance in an extreme scene are significantly improved.

An object processing system includes: an input area for receiving a plurality of objects to be processed; an output area comprising a plurality of destination containers for receiving any of the plurality of objects; a programmable motion device proximate to the input region and the output region, the programmable motion device comprising an end effector for gripping a selected object of the plurality of objects; and a perception system for detecting an unexpected occurrence of any of the plurality of objects that is not related to the end effector of the programmable motion device.

The invention belongs to the technical field of computer vision, provides a real-time semantic perception system running on equipment, performs real-time instance segmentation and semantic mapping on a working environment, and provides a perception method based on lightweight real-time instance segmentation and semantic mapping. In the first part, firstly, a color image containing color and texture information is used for object classification and bounding box regression; cutting the original depth image into a depth image with an accurate geometric edge, and inputting the depth image into the truncation network to generate a mask of the instance; a second part: (1) calculating the attitude and size of a target object in the view of the camera through principal component analysis; (2) obtaining the category of the object in the view obtained by segmentation; (3) creating a point cloud of the depth image projection of the target object in a map; and then cross-frame tracking and fusion are carried out by using data association to realize the purpose of sensing. The method is mainly applied to automatic sensing occasions.

The invention provides a credibility measurement method and system for an unmanned multi-mode perception algorithm, a data-driven virtual simulation test platform is constructed, and feature distribution is counted on the basis of actually-collected actual measurement data. Through dynamic model modeling and state transition, an end-to-end closed loop sensing process is described. Afterwards, for the problem that a traditional combined test method is low in efficiency due to the high dimension of scene parameters, a black box substitution model between the scene parameters and a perception test result is established, historical test data is evaluated by using a Bayesian optimization algorithm to generate the most antagonistic scene parameters in a next iteration test, and the efficiency of the traditional combined test method is improved. In this way, the confrontation of the test environment is continuously improved, and the test efficiency is effectively improved. And finally, a perception algorithm credibility measurement module is constructed, a perception failure scene is extracted through playback test data, a credibility evaluation index system is constructed, and comprehensive credibility evaluation of the tested perception system is realized.

In various examples, systems and methods of the present disclosure correspond to speed limit fusion for automotive systems and applications. Systems and methods are disclosed that fuse speed limit data from both one or more map sources (e.g., nav-maps, HD maps, SD maps, etc.) and a perception system, and that employ rules to help resolve conflicts in speed limits from the perception system and the map(s). For example, if a conflict occurs, the rules may indicate whether to use the speed limit data from the perception data, the speed limit data from the map(s), and / or whether to use another type of speed limit data, such as cached speed limit data.

The invention relates to the field of industrial automation, in particular to a multi-camera vision stacking pose planning method and system for an intelligent loading and unloading robot. The method comprises the following steps: constructing a collaborative sensing system through a plurality of industrial cameras deployed in a target space, and collecting dynamic environment data of a stacking area in real time; generating a self-adaptive stacking pose based on a hierarchical pose planning mechanism: analyzing an initial pose parameter through a theme control layer and generating a preliminary placement instruction, and verifying the feasibility of the preliminary placement instruction based on environmental data of the collaborative awareness system through a visual verification layer, and a final placement coordinate is optimized by combining the accurate positioning layer with a virtual box matching algorithm. The problems that in the prior art, real-time environment sensing capacity is lacked, pose planning is disjointed with the actual environment, and dynamic change scenes cannot be processed are solved.

An example method includes (a) obtaining an object detection from a perception system that describes an object in an environment of the autonomous vehicle; (b) obtaining, from a reference dataset, a label that describes a reference position of the object in the environment; (c) determining a plurality of component divergence values respectively for a plurality of divergence metrics, wherein a respective divergence value characterizes a respective difference between the object detection and the label; (d) providing the plurality of component divergence values to a machine-learned model to generate a score that indicates an aggregate divergence between the object detection and the label, wherein the machine-learned model includes a plurality of learned parameters defining an influence of the plurality of component divergence values on the score; (e) evaluating a quality of a match between the object detection and the label based on the score.

The invention relates to the technical field of battery fault detection, and particularly discloses a new energy automobile battery fault intelligent diagnosis method and system, and the method comprises the steps: collecting the three-dimensional time sequence data flow of the voltage, temperature and internal resistance of a battery pack through a distributed sensor array; the environment temperature, the number of charge and discharge cycles and the vehicle operation condition parameters are synchronously integrated as auxiliary diagnosis dimensions, and a multi-source heterogeneous data sensing system is constructed; the method comprises the following steps: preprocessing original data by adopting a dual-channel hybrid filtering architecture based on a multi-source heterogeneous data sensing system, and establishing an adaptive filtering parameter adjustment mechanism to generate multi-modal data; according to the method, multi-dimensional time sequence data and auxiliary diagnosis parameters of the battery pack are comprehensively collected through a constructed multi-source heterogeneous data sensing system, a rich and accurate data basis is provided for fault diagnosis, noise interference is reduced through a two-channel hybrid filtering architecture and a self-adaptive filtering parameter adjustment mechanism, and the fault diagnosis accuracy is improved. And a solid foundation is laid for subsequent feature extraction and model training.

The present invention provides an intelligent construction control method, apparatus, and system for splitting jet grouting mixing piles, relating to geotechnical engineering. The method includes receiving real-time perception information from a construction information self-perception system, which includes drilling depth, drilling rig output power, drill bit torque, drill rod axial force, and pore water pressure of the drill bit. This information is input into a trained machine-learning model for stratum information, establishing a nonlinear implicit correspondence relationship between the perception information and construction stratum conditions, and outputting real-time stratum state information. The real-time stratum state information is then input into a trained self-matching machine-learning model for optimal construction parameters, establishing a nonlinear implicit correspondence relationship between the stratum state information and optimal construction parameters, and outputting current optimal construction parameters. This invention enables real-time intelligent inversion of stratum information and dynamic self-optimization control of construction parameters.

The invention discloses a unified framework for solving automatic driving track prediction and planning consistency based on a world model. According to the method, through cooperative work of the automatic driving domain controller and the vehicle-mounted sensing system, end-to-end joint optimization of track prediction and planning in a complex traffic scene is realized, time sequence dependence and interaction dynamics among intelligent agents are accurately captured, and the prediction capability and robustness of a model are remarkably improved. The method comprises the following specific steps: firstly, constructing a generative world model, and generating potential future state representation by utilizing a behavior conditional and backtracking expansion technology; secondly, in combination with global modeling and a local convolutional network, multi-scale features are extracted, adaptive fusion is carried out, and a multi-modal prediction trajectory is generated; then, a multi-target planning model is adopted to integrate various driving indexes, and a track with the minimum loss function is generated; finally, path planning parameters are dynamically optimized through real-time environment perception and decision feedback, and the problems of prediction uncertainty and planning consistency of the automatic driving track are effectively solved.