93 results about "Cooperative perception" patented technology

The invention relates to the technical field of industrial safety, and discloses an electrolytic aluminum short circuit port operation safety early warning system based on multi-parameter collaborative awareness and intelligent diagnosis, and the system comprises a parameter collaborative awareness module, a dynamic diagnosis module, an early warning decision module, and an execution feedback module. By constructing a multi-dimensional parameter collaborative sensing mechanism, fusing temperature field distribution, current balance degree and insulation state multi-source data in real time and dynamically capturing early abnormal symptoms of a short circuit port, the hysteresis problem of traditional single-parameter threshold monitoring is solved, conversion from passive response to active defense is achieved, and the comprehensiveness and timeliness of operation state monitoring are improved; and meanwhile, based on a historical fault database and a real-time evolution model, a health index is generated and a fault path is predicted, so that maintenance personnel can pre-judge a development trend and a time window of potential risks in advance, and sudden equipment accidents are avoided.

The invention belongs to the technical field of ship collision avoidance, discloses a ship collision avoidance early warning method and system based on vision and AIS data collaborative perception, and firstly provides an enhanced YOLO (E-YOLO) target detector which can detect a ship in a complex environment in a high-precision and real-time manner and perform ship tracking and visual trajectory extraction in combination with a DeepSORT algorithm. Secondly, a homography coordinate conversion method based on multiple frames of static images is provided, conversion between a pixel coordinate system and a world coordinate system can be realized without accurate camera calibration, and the fusion precision of visual data and AIS data is improved. AIS and visual information are matched through a K-D tree algorithm, AIS data are projected to visual coordinates, and accurate matching of heterogeneous data is achieved. The system is provided with ship illegal behavior judgment and collision early warning rules, behaviors such as overspeed and abnormal AIS closing can be recognized, and early warning is triggered. Experiments prove that the method provided by the invention is strong in adaptability and high in robustness, and can effectively improve the ship monitoring and traffic management level of the bridge water area.

Cooperative perception systems comprise a plurality of imaging sensors that are connected to provide output images to one of one or more machine learning (ML) systems, each ML system is trained to process the output images to yield variational hypotheses. Each of the variational hypotheses comprises one or more objects and, for each of the objects, values for each of a plurality of regressed parameters and variation data indicating uncertainty of the value for each of the plurality of regressed parameters. A processor receives and fuses the hypotheses using the variation data to yield a refined hypothesis. The refined hypothesis may provide an input to a control system for a vehicle, robot or other apparatus.

The invention relates to an Internet of Vehicles multi-agent cooperative sensing method and system. The method comprises the following steps: constructing a collaborative sensing network, wherein the collaborative sensing network comprises a self-agent and a plurality of collaborative agents; acquiring and processing sensing data through the collaborative sensing network; performing feature extraction to obtain intermediate features; self-adaptive sparsification is carried out to obtain sparse features, and the sparse features are compressed and transmitted to a self-agent; performing time sequence feature enhancement on the features of all the agents at the self-agent end; fusing the features to obtain fused features; and constructing an Internet of Vehicles perception model, and realizing perception by the detection model according to the fused features. According to the method, the calculation complexity of traditional global attention is reduced from the square level to the linear level through an adaptive sparsification mechanism, the calculation overhead is remarkably reduced while the multi-agent feature interaction precision is kept, and the method is more suitable for real-time operation on the vehicle-mounted edge equipment with limited resources.

The invention provides a multi-vehicle cooperative sensing method based on context sensing, and the method comprises the steps: obtaining the sensing data of a vehicle and a cooperative vehicle, including the position and speed of the vehicle, and the context information of historical frame data; performing feature extraction on the acquired data by using a backbone network to obtain own vehicle features and cooperative features, and sharing feature information through compression and sparsification; the motion state information and the sparse feature mapping graph of the vehicle are utilized, and a graph attention mechanism is combined to dynamically select a cooperative vehicle capable of providing significant perception performance improvement in the current scene; fusing and aligning the sensing features of the historical frame and the current frame from the local of the vehicle by adopting a feature fusion algorithm based on Transform to form a time-space fusion feature of the vehicle; and carrying out multi-scale attention fusion on the self-vehicle space-time fusion feature and the cooperative vehicle feature, and carrying out a target detection task based on the fused global perception feature.

The invention discloses a time delay constrained Internet of Vehicles multi-mode cooperative sensing method and system. The method comprises the following steps: collecting point cloud data and image data, and carrying out feature extraction, fusion and space projection; constructing a two-dimensional convolution neural network, and extracting a position-level importance probability graph and a mask used for screening key information; constructing a benefit estimation neural network and a constraint evaluation neural network, and predicting expected values of system benefits and constraint conditions; meanwhile, an overall strategy selection module is established; the motorcade carries out target identification and calculates rewards based on information interaction so as to obtain a perception experience sample; and updating network parameters based on the perception experience sample to obtain a final overall strategy selection module. According to the method, the transmitting power, the transmission channel and the sensing data area of the vehicle are optimized based on reinforcement learning, a strategy correction mechanism is introduced, a risk strategy of high time delay or high packet loss is dynamically corrected, and target identification deviation and decision errors caused by sensing information delay and wrong transmission are avoided.

The invention discloses a cross-view multi-target collaborative perception and global tracking method. The method comprises the following steps: receiving original video streams of different views in parallel; carrying out feature extraction and content perception feature recombination processing; establishing a homography transformation matrix representing the mapping relation between the visual angles; performing forward geometric mapping by using the homography transformation matrix, and performing reverse mapping, secondary confirmation and state updating; a joint cost model based on space-time-appearance fusion is constructed, the potential association degree of unassociated residual targets is quantified, and missing targets are recalled; performing duplicate removal and optimization on the targets in the association set; and training and optimizing the network. According to the embodiment of the invention, the method achieves the capturing of a pixel-level target, and improves the texture and edge expression capability of a tiny target in a deep feature map. Efficient fusion of multi-dimensional spatio-temporal information among visual angles is realized; higher robustness and tracking continuity are achieved, and the integrity and fineness of a final trajectory result are improved; and the recall is ensured, and the generation of redundant tracks is avoided at the same time.

The invention relates to a multi-vehicle cooperative sensing method based on an attention mechanism, and belongs to the technical field of mobile communication. A multi-vehicle collaborative perception scene lacks selective attention to perception information of different values, so that perception characteristics of each vehicle are difficult to effectively utilize. At the same time, in a high mobility scene, it is difficult to align sensing information time delay and asynchronous features, resulting in low dynamic target sensing precision. In order to solve the problem, the method comprises the following steps: firstly, filtering redundant sensing information of cooperative vehicles, and constructing a screening model based on aerial view feature intersection-to-union ratio so as to select high-quality cooperative vehicles; spatial calibration is completed in combination with sparse consensus foreground features and geometric consistency verification, and feature fusion is realized through an attention mechanism; and finally, finishing time calibration of the dynamic target by adopting a time alignment strategy based on distillation learning. The method is suitable for multi-vehicle cooperative perception in an Internet of Vehicles scene, the focus on high-quality perception information is improved, and meanwhile, the real-time perception precision and robustness are enhanced.

A method, apparatus and system for cooperatively classifying objects in an autonomous vehicle environment are provided. A controller receives data quality indicators from multiple devices. Each data quality indicator reflects quality of an instance of object sensing data held by a device. Each device determines its data quality indicators based on its own sensor data and indications of object bounding regions. The controller makes decisions based at least in part on the data quality indicators, such as decisions on which instances of object sensing data are to be used for an object classification task, which device is to perform the object classification task, any downsampling of sensor data to be performed prior to object classification, and allocations of resources for use in object classification. The decision may be made by solving an optimization problem.

The invention discloses an inspection agent collaborative perception system based on semantic driving, and relates to the technical field of intelligent inspection, and the system comprises a prototype mapping module which collects inspection target category information and inspection target monitoring indexes, and generates an inspection task semantic prototype set and an inspection task semantic mapping table through semantic conversion; the semantic map module is used for collecting inspection image data and inspection video data through an inspection agent, performing pixel-level visual feature matching based on an inspection task semantic prototype set, and generating a semantic confidence map and a semantic request map; the coupling mutual sending module is used for executing sparse selection and directional mutual sending of semantic supply and demand coupling under the common constraint of the semantic confidence graph and the semantic request graph, and generating a multi-source sparse semantic feature queue; and the fusion remarking module is used for executing position-level semantic attention fusion and measurable sensitivity recalibration on the multi-source sparse semantic feature queue to generate a fusion probability graph and a fusion instance table.

The invention provides an intelligent network connection equipment cooperative sensing control method and system based on edge computing, and relates to the technical field of edge computing and intelligent Internet of Things, and the method comprises the steps: obtaining multi-equipment sensing data, extracting space-time marking features, constructing a cooperative sensing dependency graph, and carrying out consistency verification, redundancy suppression and blind area reasoning completion on edge computing nodes. And generating a global collaborative perception result, constructing and solving a distributed collaborative optimization model, and issuing a control instruction. According to the invention, efficient cooperative sensing and distributed control among multiple devices are realized, the sensing precision and the control efficiency are improved, and the communication overhead is reduced.

The invention discloses a power transmission line multi-mode cooperative sensing method, a mechanical intrusion early warning method and system and a storage medium, and the sensing method comprises the steps: constructing a reference coordinate system, and projecting multi-source data into the reference coordinate system; calling a tension-length coupling correction operator, and carrying out static equilibrium iteration solution on the design span to obtain a three-dimensional conductor sag physical prior curve; inputting the conductor sag physical prior curve and a historical curve in a sliding window of a preset period into a space-time continuous network, and outputting a conductor continuous sag curve in a span range; the method comprises the following steps: respectively cutting interest areas in a visible light image frame and a 4D three-coordinate radio frequency radar frame by taking plane projection of a conductor continuous sag curve as guidance, respectively extracting conductor optical anchor points and conductor radar candidate points, and generating a conductor fine three-dimensional point array in a span range through a medium radian conservation progressive registration operator; according to the invention, centimeter-level precision sensing can be maintained under extreme high-low temperature and complex load conditions.

The embodiment of the invention relates to the technical field of visual feedback, and provides a self-adaptive visual feedback system and method for ensuring material forming quality, the system comprises a front-end framework and a rear-end framework which work cooperatively, the front-end framework comprises a sensing module, a feature extraction module, an error compensation module and an execution control module, the back-end architecture comprises a decision module, a twin simulation module and an evolution module; the system further comprises a quality analysis module. According to the embodiment of the invention, environmental interference is overcome through active vision and event camera collaborative perception, long-term precision is maintained through online calibration compensation, quality quantitative evaluation is realized by combining three-dimensional shape real-time reconstruction and a deep learning method, a digital twin technology is introduced to carry out strategy security verification, and continuous evolution of the system is realized by means of a federated learning architecture. The method is not only suitable for the welding process, but also can effectively meet the high-quality control requirements of various material forming processes such as spraying, cladding and surfacing.

Systems and methods for cooperative perception are described. In some examples, the system can comprise a first subsystem comprising a first sensor, a first communication device, and a processor, which can cause the system to: detect, by the first sensor, first point cloud data, apply a data preprocessing process to the first point cloud data to generate first preprocessed sensor data, apply a feature encoding process to the first preprocessed sensor data to generate first feature data; apply an adaptive feature filtering process to the first feature data to select a first subset of features from the first feature data; apply a cooperative feature aggregation process to fuse the first subset of features with other subsets of features, to generate a fused feature map; and apply an object perception model to the fused feature map to generate object perception data.

The present invention discloses a multi-agent collaborative perception method under complex noise based on lidar, which includes the following steps: Step S1, perform coordinate transformation and preprocessing on the lidar point cloud of the collaborative agents; Step S2, use the method of deep learning to extract the feature maps of the lidar point clouds of all collaborative agents, and then compress the feature maps of the lidar point clouds of the collaborative agents and transmit them to the central vehicle; Step S3, the central vehicle decompresses the received feature maps of the lidar point clouds of the collaborative agents, and at the same time uses the feature maps of the lidar point clouds of a small number of previously received collaborative agents to perform corresponding deep learning processing on the transmission time delay and relative pose error; Step S4, the central vehicle uses the attention mechanism to fuse the feature maps of the lidar point clouds of the collaborative agents into the feature map of the central vehicle to obtain the final feature map; Step S5, send the final feature map into the detection head to obtain the detection result; solves the problem of the performance degradation of the multi-agent perception system under the transmission time delay and relative pose error.

The invention belongs to the technical field of rain and fog image processing, and particularly relates to a rain and fog image restoration method and system based on space-frequency collaborative awareness and electronic equipment, and the method comprises the steps: obtaining a rain and fog image, and respectively extracting the spatial domain features and frequency domain features of the rain and fog image; fusing the spatial domain features and the frequency domain features based on an adaptive channel attention mechanism to obtain dual-domain fusion features; the dual-domain fusion features with recovered resolution and the spatial domain features with the same size are added through residual connection to obtain a feature map, convolution processing is performed on the feature map to obtain a clear image, jump connection of traditional U-net is replaced with a residual layer structure in residual connection design, the gradient disappearance problem is relieved through residual connection, and the model training efficiency is improved. According to the design, cross-level fusion of shallow-layer edge information and deep-layer semantic features is achieved, and compared with original jump connection, recovery of local details (such as object contours) and removal of rain and fog textures are better facilitated.

The invention discloses a multi-agent cooperative sensing system based on an evidence neural network, and relates to a multi-agent cooperative sensing implementation method and implementation system. The invention aims to solve the problems that an intelligent agent cannot deal with noise, delay and sensor errors when processing a dynamic environment, and the sensing precision and the real-time performance cannot be ensured. The method comprises the following steps: step 1, extracting point cloud features by using a convolutional neural network; 2, obtaining a sparse graph from the point cloud features through a sparse graph generation module, and transmitting the sparse graph to other agents in a broadcast form; step 3, the adjacent intelligent agent determines whether to transmit the features to the own intelligent agent or not through a feature selection module; step 4, the self intelligent agent receives the characteristics transmitted by the adjacent intelligent agent and fuses the characteristics; and step 5, the fusion features pass through a detection module and an evidence regression module to obtain perception information and uncertainty estimation, and the perception information is optimized according to the uncertainty estimation module to obtain a reliable detection result. The invention relates to the field of multi-agent collaborative awareness.

The application discloses an air-ground cooperative perception method and device based on reinforcement learning and a readable medium, and relates to the field of cooperative perception, which comprises the following steps: inputting first image data collected by a vehicle and second image data collected by an aircraft into a feature extraction module in a trained perception task classification and identification model, obtaining a first feature map and a second feature map of a current time slot, and transforming to obtain an aligned second feature map of the current time slot; constructing a state of the current time slot and inputting the state into a trained policy network to obtain a policy of the current time slot, and combining the state to segment the aligned second feature map to obtain a segmented second feature map of the current time slot, and performing fusion on the segmented second feature map of the current time slot and the first feature map of the current time slot through a feature fusion module to obtain a fusion feature map of the current time slot; and inputting the fusion feature map of the current time slot into a second feature decoding module to obtain a target class probability and a target spatial position of the current time slot. The application solves the problem of low perception accuracy.

The invention discloses an information data transmission method based on a computer network, relates to the technical field of computer networks and data communication, and can intelligently maintain fairness and stability of network transmission. Through cooperative perception and decision-making of intelligent agents at a sending end and a receiving end, the method does not independently pursue speed maximization of a single data stream any more, but considers own behaviors in a global view of a whole network environment; according to the core algorithm, network flow with friendly form and smooth fluctuation is actively generated by balancing multi-dimensional targets such as throughput, time delay stability and rate smoothness, so that impact on other service flows in a shared network link is remarkably reduced, congestion caused by instant accumulation of queues is avoided, and the service quality of the shared network is improved. Fair bandwidth sharing among different data streams is promoted fundamentally, and negative intervention of a QoS strategy of an operator is effectively inhibited.

The invention discloses an ensemble synchronous light guide system, belongs to the technical field of intelligent musical instruments, music education and stage performance, and is particularly suitable for realizing accurate synchronization and cooperative training during ensemble of various musical instruments through light guide. The system adopts a terminal (terminal and intelligent musical instrument hardware)-central control (central control layer and tablet App synchronization and cooperation)-cloud (cloud, data storage and adaptive algorithm model) architecture, combines a Bluetooth Mesh + local area network dual-protocol synchronization mechanism, realizes high-precision synchronization of less than or equal to 5ms, provides a sound analysis algorithm (the accuracy rate is greater than or equal to 99%), a global light prompt rule and a collaborative analysis module, and realizes high-precision synchronization of less than or equal to 5ms. The problems that traditional ensemble training lacks visual synchronous guidance, cooperative perception is weak, feedback is not timely and the like are solved, high-precision, high-reliability, intelligent and personalized ensemble synchronous light guidance is achieved, and therefore ensemble efficiency and teaching quality are improved.

A communication system, apparatus, method, and one or more non-transitory computer-readable storage devices that implement integration of perception and communication employ the following steps: transmitting a radio frequency (RF) signal. The RF signal includes one or more communication symbols for communication and one or more perceived signal components for perceiving an object, the RF signal including timing information of the one or more perceived signal components.

The application belongs to the technical field of edge computing, and particularly relates to an edge perception device based on a human-computer interaction network, which comprises a multi-modal cooperative perception module, a lightweight intelligent processing module, a human-computer interaction adaptation module, a local data storage and privacy protection module, a real-time communication module and a power supply and resource scheduling module; the multi-modal cooperative perception module is used for synchronously collecting multi-source heterogeneous data, and the data is transmitted to the intelligent processing module, so that comprehensive capture of human, machine and environment data is realized; the lightweight intelligent processing module is used for completing data analysis, feature extraction and decision output locally, only uploading key data; the local storage module is used for encrypting and protecting the data, so that privacy leakage and bandwidth waste are avoided.

The invention provides an intraoperative gauze counting and tracking method based on visual perception, and relates to the technical field of visual pattern recognition. A collaborative perception framework integrating multispectral physical fingerprint recognition, visual space-time tracking and adaptive entangled state filtering is constructed; the core problem of target identity confirmation and persistent tracking in the prior art is solved. High-robustness fluorescent fingerprints are introduced to serve as identity anchor points, deep coupling and mutual correction of physical identities and spatial-temporal trajectories are achieved through an adaptive fusion algorithm, and finally absolute identity confirmation and high-precision and high-robustness continuous tracking of each target are achieved. According to the method, the recognition accuracy in a seriously polluted and shielded environment is improved, error accumulation in a long-term tracking process is effectively inhibited, and the self-adaptive capability and reliability of the whole system in a dynamic complex scene are enhanced.

The application provides a task-driven defogging perception method and system, and belongs to the technical field of cooperative perception of Internet of Vehicles. The method comprises the following steps: collecting a first foggy image stream and a second foggy image stream; obtaining a first processed image by intercepting and processing the first foggy image stream; obtaining a second processed image by intercepting and processing the second foggy image stream; respectively performing geometric alignment on the first processed image and the second processed image to obtain a first aligned image and a second aligned image; performing defogging and image enhancement processing on the first aligned image and extracting a first feature; performing defogging and image enhancement processing on the second aligned image and extracting a second feature; respectively repairing the first feature and the second feature by using a diffusion model; fusing the repaired first feature and the second feature, and determining a final perception result based on the fused feature. The application improves the robustness of automatic driving and balances the low power consumption of AI intelligent glasses and the high performance computing demand of a vehicle-mounted system.

The present invention relates to a method for allocating collaborative perception resources in an Internet of Vehicles (IoV) based on ISAC, and belongs to the field of mobile communication technology. The method comprises: constructing a multi-layer collaborative perception area of ​​interest scheme based on relative position; confirming a common perceiver based on the relative position of the perception area of ​​interest of the current vehicle and the road test unit and other IoV vehicles; constructing an ISAC time allocation scheme based on a time-division dynamic frame structure to obtain the total perception mutual information SMI of the radar sensing duration of the common perceiver, the average communication rate of the communication duration, and the execution delay of each processing mode; establishing a joint problem of perception task allocation, offloading, and computing resource allocation with the goal of minimizing the delay in completing the collaborative perception task; and using a multi-agent deep deterministic policy gradient algorithm in a hybrid action space to solve the optimization problem and obtain a resource allocation scheme. The present invention can optimize the resource allocation strategy for collaborative perception of the IoV and reduce the execution delay of collaborative perception.

The present invention discloses a V2X collaborative perception method and system based on a lightweight domain adapter. The present invention places a lightweight domain adapter network in the feature map input branch of each collaborative vehicle of the first V2X collaborative perception network carried by the ego vehicle to form a second V2X collaborative perception network, and uses the trained second V2X collaborative perception network to realize collaborative perception of the ego vehicle and the collaborative vehicle. By introducing a domain adapter network, the present invention successfully realizes collaborative 3D target detection in the case of heterogeneous and heterogeneous feature maps of the ego vehicle and the collaborative vehicle. Without the need to retrain the entire basic collaborative detection network, the present invention effectively reduces the domain differences between heterogeneous and heterogeneous feature maps by inserting domain adapters corresponding to different feature maps, greatly improving the detection performance and scene adaptability of V2X-based collaborative 3D target detection, and has high practical value for improving the collaborative perception performance of autonomous driving.

System, methods, and other embodiments described herein relate to cooperative perception. In one embodiment, a method includes computing, at a first timestep, a base relative pose between an ego vehicle and a remote vehicle based upon respective point clouds of the ego vehicle and the remote vehicle. The method includes computing, at a second timestep, a relative pose between the ego vehicle and the remote vehicle based upon the base relative pose, a first temporal relative pose of the ego vehicle, and a second temporal relative pose received from the remote vehicle. The method includes generating a combined point cloud based upon a first point cloud of the ego vehicle, a second point cloud received from the remote vehicle, and the relative pose.

The invention relates to an automatic driving cooperative sensing method and system based on 3D Gaussian sputtering. The system is composed of a sender vehicle and a receiver vehicle. The sender vehicle extracts 2D features from the multi-view image, discretizes the space into a voxel grid, and converts the 2D features into 3D features through projection; performing feature decoding on the 3D voxel features, and performing optimization adjustment on a preset 3D Gaussian in each voxel grid; the sender screens the 3D gausses containing the semantic information and transmits the 3D gausses to the receiver; and the receiver vehicle aligns and aggregates the received 3D gauss from the sender with the own three-dimensional coordinate system, performs Gaussian rasterization processing on the aggregated 3D gauss, and maps the Gaussian representation to the view angle of the aerial view through the 3D Gaussian splashing process to obtain a BEV semantic segmentation result. According to the method, Gaussian representation can be promoted to be used for cooperative sensing to remarkably improve the sensing precision, and the optimal sensing precision is realized on the premise of ensuring the real-time performance.

The invention provides a cooperative perception delay alignment method based on space-time common inductance calibration, and the method comprises the steps: receiving delay perception data with a timestamp from at least one cooperative vehicle, and employing historical perception data and a historical alignment result, and through a time alignment model containing a historical alignment attention mechanism, carrying out the time alignment of at least one cooperative vehicle; aligning the delay sensing data to the current moment, and calculating the motion state curvature of the target vehicle; based on the aligned data and the motion state curvature, quantifying spatial inconsistency caused by delay and motion state difference, and generating an inconsistency factor; constructing and adjusting an information matrix of the spatial common inductance calibration network based on the inconsistent factors; and by taking low-delay observation data as a fixed constraint, optimizing the spatial common inductance calibration network through a graph optimization algorithm to realize spatial consistency calibration of multi-view perception data. According to the method, time-space dual-domain collaborative optimization is realized.

A system for real-time cooperation between multiple vehicles using an edge-fog composite perception network, comprising: an edge layer with a multitude of vehicles, each vehicle having one or more sensors and an onboard processing unit configured to generate local perception data and abstract the local perception data into a spatiotemporal perception vector; a fog plane with one or more fog nodes that are communicatively coupled to the multitude of vehicles, each fog node being configured to receive spatiotemporal perception vectors from multiple vehicles and perform federated fusion to generate a local dynamic environment model; and a cloud layer with at least one cloud server configured to coordinate a federated learning process by aggregating model updates received from the fog nodes to generate an updated global perception model, the updated global perception model is distributed to the vehicles via the fog nodes to enable cooperative perception beyond the detection range of a single vehicle.