571 results about "Perception system" patented technology

The invention belongs to the technical field of artificial intelligence and intelligent sensing, particularly relates to a multi-source sensor fusion sensing system based on adaptive noise suppression, and aims to solve the problems of noise interference, modal mismatch and insufficient robustness in multi-source sensor fusion in a complex dynamic environment. The system comprises a front-end preprocessing module, an adaptive noise suppression engine, a multi-modal feature alignment unit, a credibility-driven fusion reasoning core and a closed-loop feedback optimization mechanism. Through real-time noise modeling and dynamic weight adjustment, high-precision alignment and fusion of multi-source signals are realized, and the sensing stability and real-time performance in an extreme scene are significantly improved.

The invention discloses a unified framework for solving automatic driving track prediction and planning consistency based on a world model. According to the method, through cooperative work of the automatic driving domain controller and the vehicle-mounted sensing system, end-to-end joint optimization of track prediction and planning in a complex traffic scene is realized, time sequence dependence and interaction dynamics among intelligent agents are accurately captured, and the prediction capability and robustness of a model are remarkably improved. The method comprises the following specific steps: firstly, constructing a generative world model, and generating potential future state representation by utilizing a behavior conditional and backtracking expansion technology; secondly, in combination with global modeling and a local convolutional network, multi-scale features are extracted, adaptive fusion is carried out, and a multi-modal prediction trajectory is generated; then, a multi-target planning model is adopted to integrate various driving indexes, and a track with the minimum loss function is generated; finally, path planning parameters are dynamically optimized through real-time environment perception and decision feedback, and the problems of prediction uncertainty and planning consistency of the automatic driving track are effectively solved.

The invention provides an unmanned aerial vehicle full-time perception image reconstruction method based on multi-mode cooperative reinforcement learning and degeneration decoupling. A frequency perception feature modulation model, a dual-mode dual-domain transformation module and a dynamic bidirectional guide mechanism are included. According to the system, firstly, feature information of different frequency bands is adaptively separated and modulated through a frequency sensing feature modulation model, and decoupling and compensation of composite unknown degradation are achieved; realizing cross-domain interaction and information fusion of visible light and infrared characteristics in a spatial domain and a channel domain by using a bimodal dual-domain transformation module; and finally, realizing collaborative enhancement of cross-modal degradation perception through a bidirectional dynamic guide mechanism, and generating an unmanned aerial vehicle visible light reconstruction image and an infrared super-resolution image with higher structural consistency and texture fidelity. According to the method, deep fusion and degeneration decoupling of multi-modal information can be realized in a complex degeneration environment, and the imaging quality and the environmental adaptability of an unmanned aerial vehicle full-time sensing system are remarkably improved.

The invention relates to the technical field of agricultural facility management, and discloses an AI-based agricultural scene omnibearing perception system and an implementation method, and the system comprises an order analysis module which is used for receiving order data and generating decision variables based on the order data; the collaborative optimization module is used for obtaining an optimization scheme based on a multi-objective function; the resource arrangement module is used for calculating a resource configuration scheme based on multi-target scheduling optimization; the sensing correction module is used for updating model parameters of the prediction model; the settlement loop module is used for performing delivery settlement based on the actual execution data; according to the invention, through adoption of a collaborative architecture, full-chain intelligent decision-making from market order analysis to agricultural product delivery is realized, through full-chain optimization and accurate decision-making, the net income of agricultural production is improved, and the production cost is reduced; through multi-objective function construction and a robust optimization algorithm, an optimal decision scheme can be found under complex constraint conditions, and the overall benefit of agricultural production is significantly improved.

The invention relates to an electrochromic window multi-scene control method, system and device based on indoor human body thermal comfort and a medium. The method comprises the steps that a perception data set is constructed; based on the sensing data set, performing multi-parameter coupling calculation of solar incidence azimuth deviation constraint and dynamic projection overlapping to obtain a direct radiation component, performing window space coupling analysis and double space compensation calculation to obtain a scattered radiation component, and performing fusion to obtain an individual thermal radiation influence value; determining a scene category based on the perception data set, and establishing a thermal radiation evaluation benchmark by adopting a mode matched with the scene category; and if the thermal radiation evaluation benchmark exceeds a preset threshold value, the required transmissivity of the window faces is obtained through a full-transmissivity radiation distribution method, the corresponding window faces are adjusted to the target transmission gradient according to the required transmissivity, and coupling control over the wind direction and / or power of the air conditioning system is selectively triggered. According to the invention, by constructing a multi-dimensional dynamic sensing system and an intelligent control framework, collaborative optimization of human body thermal comfort and building energy efficiency is realized.

The invention discloses a numerical control machine tool dynamic error cooperative compensation control method adaptive to high-speed machining, and relates to the technical field of numerical control machine tool error control. According to the method, a multi-source dynamic error sensing system comprising a grating displacement sensor, a six-dimensional force sensor and the like is constructed to acquire data; after wavelet threshold denoising and Kalman filtering preprocessing, inputting a three-layer LSTM error coupling prediction model combined with an attention mechanism, embedding a servo motor load characteristic curve in the model, and outputting three types of error compensation amounts; through servo-level compensation and machining-level compensation, the position of a feed shaft, the rotating speed of a main shaft, the cutting feed rate and the behavior of a micro-displacement actuator are corrected, and machining errors caused by deflection and vibration conduction of the main shaft are counteracted. And iteratively updating model parameters by using a gradient descent algorithm. According to the method, through multi-source error synchronous sensing, error coupling modeling and hierarchical cooperative compensation, dynamic error cooperative control more adaptive to a high-speed processing scene is realized, and the method has a wide application value.

The invention provides an AI human shape recognition perception system and method based on binocular vision, and the method comprises the steps: carrying out the real-time collection through a binocular camera when a doorbell key is triggered, and carrying out the preprocessing of an original image collected in real time; performing coarse parallax estimation on the real-time image rectification to obtain a full-field coarse depth map, determining a human shape candidate region list by using the full-field coarse depth map and combining the heat source region of interest, and performing fine parallax estimation on the human shape candidate region list to obtain a fine depth patch; converting the corresponding fine depth patch into a three-dimensional point cloud set according to the pose information, and performing scale prior screening based on the corresponding three-dimensional point cloud set to obtain a plurality of human shape candidate reserved areas; and performing fusion identification according to the extracted multi-modal features to obtain a human shape identification result. According to the technical scheme provided by the invention, layered parallax estimation and three-dimensional scale prior screening can be carried out on the real-time image to realize high-reliability human shape recognition under the condition of low power consumption, so that the recognition reliability of a sensing system is improved.

The invention discloses an unmanned aerial vehicle motion planning method and system for guiding visual heat conduction based on depth information, and the method comprises the steps: constructing a motion planning model, and inputting a depth image, an unmanned aerial vehicle attitude and an expected speed into the motion planning model, and real-time high-speed obstacle avoidance of the quad-rotor unmanned aerial vehicle in an unknown complex environment is realized. According to the motion planning model, a visual heat conduction module with a global receptive field is adopted as a sensing system trunk, a depth information guide heat conduction operator module is introduced, depth information is coded into an energy weight, and heat conduction calculation is performed after spatial information and scene representation are guided to be coupled, so that the model can focus on a close-range obstacle. The generated depth information guide scene representation is then input to a decision module to generate an action instruction.

The invention discloses a B-type sleeve welding control system and method, and belongs to the technical field of welding automation, and the B-type sleeve welding control system comprises a welding trolley, a welding gun, a visual perception unit and a control unit. By constructing a composite sensing system combining laser vision, molten pool recognition and welding gun sounding calibration, continuous and effective positioning in a narrow space in the sleeve under the condition that multiple layers of welding beads cover layer by layer is achieved. A welding gun is used for conducting welding wire penetration detection in a set coordinate system, space point positions are recorded in real time when the end of a welding wire makes contact with a sleeve and a base metal entity, self-adaptive correction of a welding bead starting point and a swing center is achieved, the positioning precision in the welding process of multiple layers of welding beads is greatly improved, and reliable reference is provided for follow-up path planning of the welding beads. A multi-sensor fusion algorithm is adopted, parameters such as the swing amplitude, the swing frequency, the welding height and the walking speed in the welding process are adjusted in real time, and the forming quality consistency of multiple layers of welding seams can be guaranteed.

The invention discloses a real-time collaborative cross-scene visual assistance and environment perception system for visually impaired people based on head-mounted equipment. According to the invention, through fusion of a multi-mode perception technology and a neural feedback mechanism, all-around environmental cognition support is provided for visually impaired people. Key targets such as curbs, steps and traffic signals in a complex scene can be accurately recognized, and a safe navigation scheme is generated in real time in combination with a dynamic path planning and obstacle avoidance algorithm. Through personalized feedback modes such as voice and vibration, the system can adjust the interaction rhythm according to the behavior habit and cognitive state of the user, ensure efficient and natural information transmission, effectively reduce the cognitive load of the user, help the visually impaired people to travel autonomously in different environments, improve the independent living ability, and improve the user experience. Real-time response and calculation efficiency are balanced through a collaborative architecture, so that a high-performance auxiliary function is not limited by heavy hardware equipment any more. The universal design reduces the use threshold, and is helpful for more visually impaired people to enjoy the convenience brought by science and technology.

A multi-arm robotic harvesting apparatus is provided. In another aspect, a robotic harvesting apparatus and method automatically optically locate a fruit in a tree, move and align an arm to the fruit, apply a vacuum pressure to temporarily pull the fruit against an end of the arm, rotate the arm to pick the fruit off of the tree, retract the arm and attached fruit, release the vacuum pressure to drop the fruit onto a receiving surface, and simultaneously operate another arm relative to another fruit on the same tree, while avoiding a collision between the arms. A further aspect of a mobile robotic harvesting apparatus and method applies a vacuum pressure to multiple robotically and automatically movable, fruit picking arms from a single vacuum pump, and the apparatus includes a shared optical perception system and programmable controller.

The invention provides an equipment vibration measurement device based on machine vision and a laser radar. The equipment vibration measurement device comprises a collaborative sensing system, a visual vibration inversion module, a multi-modal feature fusion and fault knowledge base module and an edge end lightweight intelligent diagnosis module. The collaborative sensing system is used for collecting a video stream of a target area and providing an absolute displacement reference; the visual vibration inversion module is used for processing the video stream to generate a vibration spectrogram; the multi-modal feature fusion and fault knowledge base module is used for constructing a multi-dimensional feature vector and a feature knowledge base; and the edge end lightweight intelligent diagnosis module is used for identifying a fault type and outputting a diagnosis result and confidence. According to the invention, non-contact, high-precision and full-automatic vibration state sensing and fault diagnosis are realized, the core target is to completely get rid of the dependence on a contact sensor, the limitation of a pure vision scheme in a complex industrial environment is overcome, and finally millisecond-level real-time fault identification and early warning are realized on the edge side with limited resources.

A high-performance loosely coupled multi-modal data fusion system for an intelligent driving environment perception system, and on-board equipment are disclosed in the present disclosure. The data fusion system includes a fusion detection model based on a modal-specific feature interaction strategy, configured to convert LiDAR point clouds, camera images, and millimeter-wave radar point clouds into unified bird's-eye view (BEV) features and perform multi-modal fusion; and a fusion tracking model based on a cascade coupling data association strategy of motion-appearance features, configured to perform subsequent trajectory tracking and matching based on feature information of the multi-modal fusion. A VoD dataset and a K-Radar dataset are selected for training, validating, and testing comprehensive performance of the models. An inference model is accelerated by applying TensorRT, to be quantified and deployed on an on-board computing test platform.

The invention relates to a multi-modal data fusion method, system and device and a medium. The method comprises the following steps: acquiring synchronous RGB (Red, Green and Blue) images and laser radar point cloud data; geometric alignment is carried out on the point cloud data, and a mapping relation between the point cloud data and image pixels is established; respectively extracting two-dimensional visual features of the image and three-dimensional geometric features of the point cloud based on the mapping relation, and re-projecting the three-dimensional features to a two-dimensional space aligned with the visual features; and finally, dynamic weighted fusion is carried out on the two types of features through an adaptive attention mechanism, and a multi-modal fusion feature map is generated. By adopting the method, the fusion weight can be automatically adjusted according to the environment change, and the robustness and accuracy of a sensing system in a complex scene are effectively improved.

The invention provides an unmanned aerial vehicle riverway inspection method based on artificial intelligence, and belongs to the technical field of unmanned aerial vehicle riverway inspection. A riverway environment multi-scale feature sensing system is established to respectively construct a fine-grained feature matrix for recording surface texture changes and a coarse-grained environment matrix for storing overall form information; different scale feature matrixes are mapped to corresponding high-dimensional feature spaces by adopting a radial basis kernel function and a polynomial kernel function to form a fine-grained feature mapping matrix and a coarse-grained environment mapping matrix, result quality is evaluated through an inspection result judgment function, and a recollection instruction matrix is generated to guide supplementary data collection when confidence is insufficient. And finally, trend analysis is performed in combination with a historical data fusion module to provide parameter optimization guidance for a subsequent inspection task, and the technical problem that multi-scale feature information is difficult to adaptively process in the unmanned aerial vehicle river channel inspection process is solved.

The invention relates to an intelligent automobile environment sensing system, in particular to an intelligent automobile robust aerial view environment sensing system based on depth mapping and KAN-Mama fusion. Comprising an image modal semantic coding and hierarchical fusion module, a laser radar point cloud sparse coding and BEV feature construction module, a local self-attention depth feature enhancement module, a CMSM gating interaction module, a KAN high-order nonlinear fusion module, a sensor adaptive disturbance modeling module and a multi-task perception decoding module. The excessive dependence of the existing BEV environment sensing system on the ideal calibration condition of the sensor is avoided, the sensing performance can still be ensured when the sensor has a slight alignment error, and meanwhile, the defect of the self-adaptive modeling capability of the local area of the existing BEV environment sensing system is overcome; the non-ideal calibration robustness and the local area modeling robustness of the intelligent automobile BEV environment sensing system are remarkably improved, and a basic interaction architecture with high information coupling degree and strong feature expression is constructed for the intelligent automobile sensing system.

The invention discloses an unmanned sweeping robot navigation method based on multi-sensor fusion, and belongs to the technical field of sweeping robots. According to the method, an environment sensing system is constructed through multi-source sensor fusion, and efficient path planning and obstacle avoidance decision making in a dynamic environment are realized in combination with a quantum heuristic behavior decision engine; a traditional decision problem is converted into quantum state superposition operation through a quantum state probability amplitude model, and dynamic coupling of environment perception and a task target is achieved through the joint effect of a complex phase angle and a weight coefficient; and the dynamic collapse equation improves the adaptive capacity of behavior selection. The sweeping efficiency, safety and environment adaptability of the sweeping robot in a complex dynamic environment are remarkably improved, and the sweeping robot is suitable for various scenes such as families and offices.

The invention discloses a train control system, method, equipment and medium based on active sensing and autonomous localization, and the system comprises a multi-source autonomous sensing system which collects and outputs the information of the environment in front of a train by fusing the sensing data of a plurality of sensors; the vehicle-mounted standby system is in communication connection with the multi-source autonomous sensing system, accesses a pre-stored digital track map and is used for dynamically generating route topology and movement authorization and generating a train control instruction according to the train front environment information, the train real-time position and the line data of the digital track map; a dual-mode communication switching unit is arranged in the vehicle-mounted standby system and used for monitoring the communication state of the main train control system, and when the main train control system breaks down, the train control right is automatically switched to the standby system. Compared with the prior art, after the main train control system loses efficacy, the train can be made to be independent of a central control system and a fixed electronic map, the real-time state of a line is automatically adapted, and the safe driving instruction is generated.

A perception system may be used to generate bounding boxes for objects in a vehicle scene. The perception system may receive images and feature maps corresponding to the received images. The perception system may use radar and vision based images to generate one or more bounding boxes for objects in the vehicle scene.

The invention discloses an inspection agent collaborative perception system based on semantic driving, and relates to the technical field of intelligent inspection, and the system comprises a prototype mapping module which collects inspection target category information and inspection target monitoring indexes, and generates an inspection task semantic prototype set and an inspection task semantic mapping table through semantic conversion; the semantic map module is used for collecting inspection image data and inspection video data through an inspection agent, performing pixel-level visual feature matching based on an inspection task semantic prototype set, and generating a semantic confidence map and a semantic request map; the coupling mutual sending module is used for executing sparse selection and directional mutual sending of semantic supply and demand coupling under the common constraint of the semantic confidence graph and the semantic request graph, and generating a multi-source sparse semantic feature queue; and the fusion remarking module is used for executing position-level semantic attention fusion and measurable sensitivity recalibration on the multi-source sparse semantic feature queue to generate a fusion probability graph and a fusion instance table.

A low-altitude unmanned aerial vehicle cooperative sensing system based on 5G-A communication and sensing integration comprises a plurality of 5G-A communication and sensing integration base stations deployed on the ground, a plurality of low-altitude flight unmanned aerial vehicle nodes and a cooperative sensing dispatching center located on the core network side, and the 5G-A communication and sensing integration base stations are provided with millimeter wave large-scale MIMO antenna arrays integrating communication and radar sensing functions. The unmanned aerial vehicle node is used for transmitting communication signals and sensing detection beams to the air at the same time, the unmanned aerial vehicle node carries a communication sensing fusion terminal supporting a 5G-A air interface protocol, and the terminal comprises a radio frequency receiving and transmitting module, a beam forming processor and a local sensing data caching unit. According to the invention, the millimeter wave large-scale MIMO antenna array of the 5G-A communication and sensing integrated base station is utilized and is divided into the communication sub-array and the sensing sub-array which are independent and controllable, so that the extreme multiplexing of a frequency spectrum and hardware resources is realized, and the communication and the sensing can run in parallel on the same frequency band and the same equipment without interfering with each other.

The invention discloses a comprehensive traffic low-altitude global intelligent sensing method based on autonomous evolution, and belongs to the crossing field of intelligent traffic and computer vision. The method comprises the following steps: collecting multi-source sensing data and extracting a structured causal variable; a dynamic causal graph is constructed, and attribution analysis of perception errors is realized through intervention learning; realizing cross-scene model migration and few-sample self-adaption based on the knowledge graph; carrying out adaptive reasoning by adopting an environment-aware dynamic graph neural network; and realizing a causal-driven autonomous evolution closed loop based on multi-source feedback. According to the method, the crossing of the perception system from'perception-optimization 'to'understanding-evolution' in a low-altitude complex traffic environment is realized, the perception precision, the scene adaptability and the system interpretability are remarkably improved, and the manual operation and maintenance cost is reduced.

The invention discloses an anti-swing predictive control method for an offshore crane, and belongs to the technical field of ocean engineering equipment control. The problem that in the prior art, due to control lag and an inaccurate model, the load swing restraining effect of an offshore crane is poor is solved. According to the scheme, the method is characterized in that the state of a crane and prediction information of future waves are obtained in real time through a state sensor set and a multi-source environment sensing system; mixing the prediction model to predict a crane system state sequence under different control instructions in a future time domain in a rolling manner; based on the prediction sequence, solving a reference control track aiming at suppressing swing and reducing structural fatigue in upper-layer optimization, and solving and outputting an instant control instruction meeting the constraint of an execution mechanism in lower-layer rapid optimization; meanwhile, system health management is independently executed, and a control mode is dynamically adjusted according to evaluation. The method is mainly used for precise anti-swing control of the offshore crane under the complex sea condition, load swing can be effectively inhibited in advance, and operation precision and equipment safety are improved.

There is provided a computer-implemented method for verifying the robustness of a neural network classifier with respect to one or more parameterised transformations applied to an input, the classifier comprising one or more convolutional layers, the method comprising: encoding each layer of the classifier as one or more algebraic classifier constraints; encoding each transformation as one or more algebraic transformation constraints; encoding a change in an output classifier label from the classifier as an algebraic output constraint; determining whether a solution exists which satisfies the classifier constraints, transformation constraints and output constraints, and determining the classifier as robust to the local transformations if no such solution exists. A perception system and a computer readable medium are also provided.

A perception system implemented as a base neural network is trained on training data elements describing the evolution of an environment during a period of time, and having multimodal data formats: (1) a consecutive sequence of RGB images, (2) a consecutive sequence of radar range-azimuth heatmaps, and (3) a set of Doppler spectrograms. The base neural network may later be used in a specific perception application after training. For example, the pretrained neural network model or a subset of its layers may be used in another neural net (a “task-specific network”) which is trained to perform a task on at least a received radar data set captured from a real-world environment.

The invention discloses a financial flow situation awareness system and method based on a multi-modal large model, and relates to the technical field of supply chain financial risk control. Aiming at the problems of data isolation, high processing delay and risk perception lagging in the existing storage pledge financing scene, the method comprises the following steps: obtaining a storage digital planar graph, dividing logic sub-regions, collecting and processing multi-source data of each sub-region in parallel, and extracting pledge stock time sequence data and warehouse receipt information by utilizing a visual and text model respectively; constructing a sub-region association map by taking the cargo identifier as a node and fusing the fund flow; and fusing all the sub-maps to construct a global association map, performing cross-regional consistency verification and anomaly recognition, generating a dynamic risk judgment result, and outputting hierarchical situation awareness information. According to the invention, accurate association and panoramic risk perception of the "object-bill-money" state are realized, and the real-time performance and accuracy of risk identification and the expandability of the system are improved.

The invention discloses an intensive culture precise feeding control method based on fuzzy-MPC control, and relates to the technical field of aquaculture intelligent control, and the method comprises the steps: employing a sensing system, collecting multi-source sensing data including fish activity images, healthy growth states and water quality environment data in real time, and forming a comprehensive state data set; and based on the acquired multi-source sensing data, constructing an MTL-LSTM-SAT multi-task learning model. According to the method, the multi-task learning model fusing the LSTM and the soft attention mechanism is constructed, the long-term dependency relationship and dynamic association among the water quality, the feeding and the fish state can be captured, the attention mechanism enables the model to be automatically focused on a key time step, the perception ability of the water quality fluctuation and the feeding influence stage is enhanced, and the accuracy of the water quality fluctuation and feeding influence stage is improved. Meanwhile, double-task collaborative prediction of the growth speed and the disease proportion is achieved through the shared feature layer, a prospective state sequence of the future 7 days is output, and a quantitative and reliable prediction basis is provided for feeding decision making.

In various examples, probabilistic-based techniques may be used to predict intended paths of machines through an environment. For instance, various input data from perception systems, localization systems, mapping systems, and / or other sources of data may be used to determine occupant intent and compute scores associated with road segments in an environment. The scores may indicate a probability that certain road segments are part of the occupant's intended path for the machine, and the scores may be aggregated for each of the road segments across multiple instances of receiving an analyzing the input data. In some instances, a highest scoring road segment(s) may be selected as part of a predicted path of the machine. For instance, at a junction(s) where multiple road segments meet, the highest scoring road segment(s) may be selected as the predicted path.

In various examples, machine learning models that benefit from temporal context while being computationally efficient to train and use are described herein. For instance, the disclosed systems and methods may apply a temporal series of images to a model and use intermediate features output from one or more backbone layers of the model as training data. In some examples, one or more recursive layers and / or one or more head layers of the model—or another model—may be trained using the training data by applying the intermediate features to the recursive layer(s). The recursive layer(s) may output a state representative of a temporal combination of the intermediate features, and the state may be applied to the head layer(s) to make one or more predictions. During inference, the recursive layer(s) may, in some examples, continuously update the state based on previous states of the recursive layer(s).

The invention relates to an intelligent automobile aerial view angle environment sensing system, in particular to a high-robustness intelligent automobile aerial view angle multi-source space-time fusion environment sensing system. Comprising an image feature extraction module, a point cloud feature extraction module, a cross-modal interaction fusion module, a diffusion generation module, a time cross attention fusion module and an environment perception decoding module. Complementary enhancement of camera images and laser radar point cloud BEV features is realized through a bidirectional cross attention mechanism, a semantic-geometric bidirectional correlation weight matrix is constructed, and dynamic distribution of cross-modal feature weights is realized; gaussian noise is injected into the fused BEV feature space through a diffusion generation technology to simulate multi-source noise interference in a real environment, and a multi-scale denoising network guide system based on a U-Net architecture is adopted to separate real environment semantics from noisy features; and space-time alignment between the historical BEV features and the current BEV features is realized by constructing a multi-head time cross attention mechanism.