50 results about "Cooperative perception" patented technology

The invention relates to an Internet of Vehicles multi-agent cooperative sensing method and system. The method comprises the following steps: constructing a collaborative sensing network, wherein the collaborative sensing network comprises a self-agent and a plurality of collaborative agents; acquiring and processing sensing data through the collaborative sensing network; performing feature extraction to obtain intermediate features; self-adaptive sparsification is carried out to obtain sparse features, and the sparse features are compressed and transmitted to a self-agent; performing time sequence feature enhancement on the features of all the agents at the self-agent end; fusing the features to obtain fused features; and constructing an Internet of Vehicles perception model, and realizing perception by the detection model according to the fused features. According to the method, the calculation complexity of traditional global attention is reduced from the square level to the linear level through an adaptive sparsification mechanism, the calculation overhead is remarkably reduced while the multi-agent feature interaction precision is kept, and the method is more suitable for real-time operation on the vehicle-mounted edge equipment with limited resources.

The invention provides a multi-vehicle cooperative sensing method based on context sensing, and the method comprises the steps: obtaining the sensing data of a vehicle and a cooperative vehicle, including the position and speed of the vehicle, and the context information of historical frame data; performing feature extraction on the acquired data by using a backbone network to obtain own vehicle features and cooperative features, and sharing feature information through compression and sparsification; the motion state information and the sparse feature mapping graph of the vehicle are utilized, and a graph attention mechanism is combined to dynamically select a cooperative vehicle capable of providing significant perception performance improvement in the current scene; fusing and aligning the sensing features of the historical frame and the current frame from the local of the vehicle by adopting a feature fusion algorithm based on Transform to form a time-space fusion feature of the vehicle; and carrying out multi-scale attention fusion on the self-vehicle space-time fusion feature and the cooperative vehicle feature, and carrying out a target detection task based on the fused global perception feature.

The invention discloses a time delay constrained Internet of Vehicles multi-mode cooperative sensing method and system. The method comprises the following steps: collecting point cloud data and image data, and carrying out feature extraction, fusion and space projection; constructing a two-dimensional convolution neural network, and extracting a position-level importance probability graph and a mask used for screening key information; constructing a benefit estimation neural network and a constraint evaluation neural network, and predicting expected values of system benefits and constraint conditions; meanwhile, an overall strategy selection module is established; the motorcade carries out target identification and calculates rewards based on information interaction so as to obtain a perception experience sample; and updating network parameters based on the perception experience sample to obtain a final overall strategy selection module. According to the method, the transmitting power, the transmission channel and the sensing data area of the vehicle are optimized based on reinforcement learning, a strategy correction mechanism is introduced, a risk strategy of high time delay or high packet loss is dynamically corrected, and target identification deviation and decision errors caused by sensing information delay and wrong transmission are avoided.

The invention discloses a cross-view multi-target collaborative perception and global tracking method. The method comprises the following steps: receiving original video streams of different views in parallel; carrying out feature extraction and content perception feature recombination processing; establishing a homography transformation matrix representing the mapping relation between the visual angles; performing forward geometric mapping by using the homography transformation matrix, and performing reverse mapping, secondary confirmation and state updating; a joint cost model based on space-time-appearance fusion is constructed, the potential association degree of unassociated residual targets is quantified, and missing targets are recalled; performing duplicate removal and optimization on the targets in the association set; and training and optimizing the network. According to the embodiment of the invention, the method achieves the capturing of a pixel-level target, and improves the texture and edge expression capability of a tiny target in a deep feature map. Efficient fusion of multi-dimensional spatio-temporal information among visual angles is realized; higher robustness and tracking continuity are achieved, and the integrity and fineness of a final trajectory result are improved; and the recall is ensured, and the generation of redundant tracks is avoided at the same time.

The invention discloses an inspection agent collaborative perception system based on semantic driving, and relates to the technical field of intelligent inspection, and the system comprises a prototype mapping module which collects inspection target category information and inspection target monitoring indexes, and generates an inspection task semantic prototype set and an inspection task semantic mapping table through semantic conversion; the semantic map module is used for collecting inspection image data and inspection video data through an inspection agent, performing pixel-level visual feature matching based on an inspection task semantic prototype set, and generating a semantic confidence map and a semantic request map; the coupling mutual sending module is used for executing sparse selection and directional mutual sending of semantic supply and demand coupling under the common constraint of the semantic confidence graph and the semantic request graph, and generating a multi-source sparse semantic feature queue; and the fusion remarking module is used for executing position-level semantic attention fusion and measurable sensitivity recalibration on the multi-source sparse semantic feature queue to generate a fusion probability graph and a fusion instance table.

The invention provides an intelligent network connection equipment cooperative sensing control method and system based on edge computing, and relates to the technical field of edge computing and intelligent Internet of Things, and the method comprises the steps: obtaining multi-equipment sensing data, extracting space-time marking features, constructing a cooperative sensing dependency graph, and carrying out consistency verification, redundancy suppression and blind area reasoning completion on edge computing nodes. And generating a global collaborative perception result, constructing and solving a distributed collaborative optimization model, and issuing a control instruction. According to the invention, efficient cooperative sensing and distributed control among multiple devices are realized, the sensing precision and the control efficiency are improved, and the communication overhead is reduced.

The embodiment of the invention relates to the technical field of visual feedback, and provides a self-adaptive visual feedback system and method for ensuring material forming quality, the system comprises a front-end framework and a rear-end framework which work cooperatively, the front-end framework comprises a sensing module, a feature extraction module, an error compensation module and an execution control module, the back-end architecture comprises a decision module, a twin simulation module and an evolution module; the system further comprises a quality analysis module. According to the embodiment of the invention, environmental interference is overcome through active vision and event camera collaborative perception, long-term precision is maintained through online calibration compensation, quality quantitative evaluation is realized by combining three-dimensional shape real-time reconstruction and a deep learning method, a digital twin technology is introduced to carry out strategy security verification, and continuous evolution of the system is realized by means of a federated learning architecture. The method is not only suitable for the welding process, but also can effectively meet the high-quality control requirements of various material forming processes such as spraying, cladding and surfacing.

The application discloses an air-ground cooperative perception method and device based on reinforcement learning and a readable medium, and relates to the field of cooperative perception, which comprises the following steps: inputting first image data collected by a vehicle and second image data collected by an aircraft into a feature extraction module in a trained perception task classification and identification model, obtaining a first feature map and a second feature map of a current time slot, and transforming to obtain an aligned second feature map of the current time slot; constructing a state of the current time slot and inputting the state into a trained policy network to obtain a policy of the current time slot, and combining the state to segment the aligned second feature map to obtain a segmented second feature map of the current time slot, and performing fusion on the segmented second feature map of the current time slot and the first feature map of the current time slot through a feature fusion module to obtain a fusion feature map of the current time slot; and inputting the fusion feature map of the current time slot into a second feature decoding module to obtain a target class probability and a target spatial position of the current time slot. The application solves the problem of low perception accuracy.

The invention discloses an ensemble synchronous light guide system, belongs to the technical field of intelligent musical instruments, music education and stage performance, and is particularly suitable for realizing accurate synchronization and cooperative training during ensemble of various musical instruments through light guide. The system adopts a terminal (terminal and intelligent musical instrument hardware)-central control (central control layer and tablet App synchronization and cooperation)-cloud (cloud, data storage and adaptive algorithm model) architecture, combines a Bluetooth Mesh + local area network dual-protocol synchronization mechanism, realizes high-precision synchronization of less than or equal to 5ms, provides a sound analysis algorithm (the accuracy rate is greater than or equal to 99%), a global light prompt rule and a collaborative analysis module, and realizes high-precision synchronization of less than or equal to 5ms. The problems that traditional ensemble training lacks visual synchronous guidance, cooperative perception is weak, feedback is not timely and the like are solved, high-precision, high-reliability, intelligent and personalized ensemble synchronous light guidance is achieved, and therefore ensemble efficiency and teaching quality are improved.

The application belongs to the technical field of edge computing, and particularly relates to an edge perception device based on a human-computer interaction network, which comprises a multi-modal cooperative perception module, a lightweight intelligent processing module, a human-computer interaction adaptation module, a local data storage and privacy protection module, a real-time communication module and a power supply and resource scheduling module; the multi-modal cooperative perception module is used for synchronously collecting multi-source heterogeneous data, and the data is transmitted to the intelligent processing module, so that comprehensive capture of human, machine and environment data is realized; the lightweight intelligent processing module is used for completing data analysis, feature extraction and decision output locally, only uploading key data; the local storage module is used for encrypting and protecting the data, so that privacy leakage and bandwidth waste are avoided.

The invention provides an intraoperative gauze counting and tracking method based on visual perception, and relates to the technical field of visual pattern recognition. A collaborative perception framework integrating multispectral physical fingerprint recognition, visual space-time tracking and adaptive entangled state filtering is constructed; the core problem of target identity confirmation and persistent tracking in the prior art is solved. High-robustness fluorescent fingerprints are introduced to serve as identity anchor points, deep coupling and mutual correction of physical identities and spatial-temporal trajectories are achieved through an adaptive fusion algorithm, and finally absolute identity confirmation and high-precision and high-robustness continuous tracking of each target are achieved. According to the method, the recognition accuracy in a seriously polluted and shielded environment is improved, error accumulation in a long-term tracking process is effectively inhibited, and the self-adaptive capability and reliability of the whole system in a dynamic complex scene are enhanced.

The application provides a task-driven defogging perception method and system, and belongs to the technical field of cooperative perception of Internet of Vehicles. The method comprises the following steps: collecting a first foggy image stream and a second foggy image stream; obtaining a first processed image by intercepting and processing the first foggy image stream; obtaining a second processed image by intercepting and processing the second foggy image stream; respectively performing geometric alignment on the first processed image and the second processed image to obtain a first aligned image and a second aligned image; performing defogging and image enhancement processing on the first aligned image and extracting a first feature; performing defogging and image enhancement processing on the second aligned image and extracting a second feature; respectively repairing the first feature and the second feature by using a diffusion model; fusing the repaired first feature and the second feature, and determining a final perception result based on the fused feature. The application improves the robustness of automatic driving and balances the low power consumption of AI intelligent glasses and the high performance computing demand of a vehicle-mounted system.

The invention provides a cooperative perception delay alignment method based on space-time common inductance calibration, and the method comprises the steps: receiving delay perception data with a timestamp from at least one cooperative vehicle, and employing historical perception data and a historical alignment result, and through a time alignment model containing a historical alignment attention mechanism, carrying out the time alignment of at least one cooperative vehicle; aligning the delay sensing data to the current moment, and calculating the motion state curvature of the target vehicle; based on the aligned data and the motion state curvature, quantifying spatial inconsistency caused by delay and motion state difference, and generating an inconsistency factor; constructing and adjusting an information matrix of the spatial common inductance calibration network based on the inconsistent factors; and by taking low-delay observation data as a fixed constraint, optimizing the spatial common inductance calibration network through a graph optimization algorithm to realize spatial consistency calibration of multi-view perception data. According to the method, time-space dual-domain collaborative optimization is realized.

A system for real-time cooperation between multiple vehicles using an edge-fog composite perception network, comprising: an edge layer with a multitude of vehicles, each vehicle having one or more sensors and an onboard processing unit configured to generate local perception data and abstract the local perception data into a spatiotemporal perception vector; a fog plane with one or more fog nodes that are communicatively coupled to the multitude of vehicles, each fog node being configured to receive spatiotemporal perception vectors from multiple vehicles and perform federated fusion to generate a local dynamic environment model; and a cloud layer with at least one cloud server configured to coordinate a federated learning process by aggregating model updates received from the fog nodes to generate an updated global perception model, the updated global perception model is distributed to the vehicles via the fog nodes to enable cooperative perception beyond the detection range of a single vehicle.

The application belongs to the technical field of vehicle networking cooperative perception, and relates to a vehicle networking cooperative perception parameter efficient domain self-adaptation method, device and medium; feature extraction is performed on environment data collected by each vehicle-mounted node in a target domain to obtain a space-time feature vector of each vehicle-mounted node in the target domain; based on Euclidean distances between the space-time feature vectors of each vehicle-mounted node, an optimal transmission distance between each two vehicle-mounted nodes is obtained; taking minimization of a maximum value of the optimal transmission distance between each vehicle-mounted node in the target domain and a selected target vehicle-mounted node set as an objective, a vehicle-mounted node selection function is constructed and solved to obtain the target vehicle-mounted node set; and environment data collected by each vehicle-mounted node in the target vehicle-mounted node set is used to perform parameter efficient fine-tuning on a vehicle networking cooperative perception model carried on each vehicle-mounted node.

This invention relates to a multi-vehicle cooperative perception method, comprising the following steps: Step 1, acquiring multi-view images captured by the driver vehicle and cooperative vehicles respectively; Step 2, constructing a cooperative perception model, which includes a voxel generator, a compression fusion module, and an occupancy prediction head connected in sequence; the voxel generator converts multi-view images into voxel features based on a probabilistic kernel truncated voxel generation mechanism; the compression fusion module fuses voxel features among cooperative vehicles based on a cardinality conditional attention mechanism, selectively aggregating key occupancy perception cues; Step 3, simultaneously inputting the acquired multi-view images of the driver vehicle and cooperative vehicles into the cooperative perception model, ultimately obtaining the perception results of the environment where the driver vehicle and cooperative vehicles are located. This method improves query accuracy while reducing computational overhead.

The invention relates to the technical field of low-altitude aircraft supervision, and provides a low-altitude cooperative sensing method, system and device, a medium and a program product, and the method comprises the steps: obtaining multivariate flight sensing data; performing fusion processing based on the same type of data according to the multivariate flight perception data to obtain multiple groups of similar track data; according to the multiple groups of similar track data, based on cross fusion processing, obtaining regional sensing data; the cross fusion processing is used for fusing different types of similar track data corresponding to the same track. The multi-element flight sensing data is utilized to sense the low-altitude aircraft from multiple dimensions, different types of sensing modes make up and cooperate with each other, the more comprehensive sensing effect is achieved, and the sensing coverage range is expanded. According to the method, perception data of the same type are fused, perception flight paths of various types are determined, and flight paths of different types are fused, so that the flight path of the low-altitude aircraft is determined more accurately, the perception accuracy is improved, and the effect of comprehensively and accurately perceiving the aircraft is achieved.

The application discloses a kind of based on collaborative perception's steam turbine generator rotor winding interturn short circuit early warning method, it is related to generator technical field, the early warning method first utilizes the real-time working condition data collected by steam turbine generator DCS system, using Pearson correlation coefficient obtains the correlation degree between excitation flow and vibration signal, then adopts the method of collaborative gain transformation and fuses "current-vibration" correlation coefficient, and combines the residual value between excitation current forecast value and measured value, further calculates collaborative gain residual, finally it is compared with the collaborative gain residual threshold value set, to judge steam turbine generator rotor winding interturn insulation condition.Compared with the diagnosis mode based on single variable in the prior art, the "current-vibration" collaborative perception fusion diagnosis mode provided by the application can realize early online early warning of steam turbine generator rotor winding interturn short circuit fault, with higher accuracy and sensitivity, can effectively guarantee the safe and stable operation of unit.

The application relates to the technical field of low-delay transmission, in particular to a low-delay transmission method for multi-source collaborative perception information based on intelligent traffic networking, which comprises the following steps: at a source end, generating a sparse semantic vector containing T non-zero components from an original local feature vector from a sensor; at an edge node, fusing sparse semantic vectors of semantic packets received from multiple source ends; and at a scheduler, calculating a priority index for a to-be-sent semantic packet and sequentially sending the semantic packet in descending order of the priority index. The application aims to reduce the information transmission delay of an end-to-end decision task and enhance the timeliness and robustness of a system.

The application discloses an interconnected automatic driving decision-making method based on cooperative perception and adaptive information fusion, and mainly solves the problem that the existing automatic driving decision-making is less applicable under the condition of complex road structure and traffic light information. The method considers a multi-lane traffic environment under a world coordinate system, wherein a mixed traffic flow composed of interconnected automatic driving vehicles and human-driven vehicles is formed. Each CAV can obtain surrounding multi-modal environment features (such as lane information, HDV vehicle information and red-green traffic light information) through a vehicle-mounted sensor and an offline high-precision map. With the help of vehicle-to-vehicle communication, the CAVs can share their information and make decisions within a specified time step t. The goal of the method is to generate speed decisions and steering angle decisions for the CAVs. With such action decisions, the automatic driving vehicles can safely and effectively travel along a specific route, while maximizing the comfort of passengers and minimizing the impact on surrounding HDVs.

The invention provides a distributed cooperative sensing response adaptive navigation method, which comprises the following steps: acquiring distributed sensing data at a field end and an end side, and converting the distributed sensing data into a unified real-time global dynamic map; based on the global dynamic map, identifying a current operation scene, distributing tasks for an end side and planning a conflict-free global path; and converting the global path into an end-side executable local navigation track, responding to a dynamic obstacle in real time, and correcting the local navigation track based on a state feedback mechanism. According to the technical scheme provided by the invention, the technical problems of poor coordination of single navigation blind areas and poor robustness of centralized adaptation of the intelligent mobile robot in the prior art can be effectively solved.

The application belongs to the technical field of multi-robot cooperative perception and autonomous navigation, and particularly relates to a multi-robot cooperative semantic SLAM and dynamic exploration method, which comprises the following steps: acquiring environment data; inputting the environment data into an improved YOLOv8 model to obtain object semantic information, wherein the improved YOLOv8 model is obtained by improving the YOLOv8 model; constructing a three-dimensional ellipsoid model based on the object semantic information; matching and aligning multi-view data based on the object semantic information and the three-dimensional ellipsoid model, and performing global pose joint optimization combined with a nonlinear optimization algorithm to obtain a high-precision map; and performing dynamic exploration based on the high-precision map and in combination with a multi-robot exploration task. The application fuses a cooperative SLAM framework of geometric and semantic information, and designs a layered exploration strategy, thereby combining a semantic-enhanced perception capability and a task allocation mechanism to improve the mapping precision and significantly improve the overall exploration efficiency.

This invention discloses a frequency-space collaborative sensing method and system for target detection in sonar images. The method includes the following steps: acquiring sonar images, preprocessing them using contrast-limited adaptive equalization, and dividing them into training and validation sets; constructing an initial target detection network and training it based on the training and validation sets to obtain the target detection network; acquiring the sonar image to be detected and inputting it into the target detection network for target detection to obtain the detection result. The contrast-limited adaptive equalization grayscale transformation preprocessing of this invention enhances the contrast of sonar images, making target boundaries clearer, textures more stable, and background stripes and speckles more distinguishable, effectively improving the model's detection accuracy.

The invention discloses a traffic participant behavior recognition collaborative perception method and control equipment, and the method comprises the steps: firstly, taking an extracted multi-modal feature as a diffusion target, and providing a basis for the subsequent feature optimization; secondly, the features are enhanced by injecting Gaussian noise, and the diversity and robustness of the features are improved; and finally, partial mask processing is performed on the features by adopting a progressive sensor discarding training (PSDT) method, a sensor fault scene is simulated, and the adaptability of the model to incomplete data is further enhanced. The processed features are input into a DuAT (check) module, the module adopts a global self-attention module (GLSA) as an encoder block, the features are denoised and refined, and the relevance between the global features and the local features is fully captured. Finally, optimized feature representation is transmitted to a specific task head, and high-precision three-dimensional target detection and aerial view map segmentation are realized.

The invention discloses a vehicle infrastructure cooperative sensing method and device, computer equipment and a medium, and the method comprises the steps: building personalized neural network elements through a differentiable architecture search, and obtaining a discretization architecture of a feature maintaining unit and a dimension reduction unit; determining the network depth based on the discretization architecture and the local data of the vehicle-mounted node; constructing a local personalized model based on the discretization architecture and the network depth, and establishing an operation alignment hierarchical aggregation mapping table M based on the unit sequence and the model parameter of each vehicle-mounted node; and finally, performing model aggregation of operation alignment driving based on the aggregation mapping table M, updating model parameters of each vehicle-mounted node to obtain a vehicle-road collaborative perception model, and performing vehicle-road collaborative perception by adopting the vehicle-road collaborative perception model. According to the invention, the cooperative sensing precision is improved.

A train control system based on train-ground-air active cooperative sensing comprises a far-end sensing computing system used for monitoring far-end environment sensing information of a rail-mounted area in real time and sending the far-end environment sensing information to a central service system; the central service system is used for sending second environment sensing information of an area within a second distance in front of the train head of the train to the train-mounted multi-information fusion system of the train based on the far-end environment sensing information; the vehicle-mounted multi-information fusion system is arranged on the train and is used for acquiring first environment sensing information of an area within a first distance in front of the train head and behind the train tail of the train; the server is also used for fusing the first environment sensing information and the second environment sensing information, generating a fused sensing result and sending the fused sensing result to the vehicle-mounted train control auxiliary system; the second distance is greater than the first distance; and the vehicle-mounted train control auxiliary system is used for generating a train control strategy according to the fusion sensing result so as to assist in controlling the train to run. Through the scheme provided by the embodiment, the active safety guarantee level of train operation can be effectively improved.

The invention discloses a global track identity management method and device for roadside multi-sensor collaborative awareness, and relates to the technical field of target tracking, and the method comprises the steps: monitoring a target through a thunder-vision all-in-one machine, and generating a track corresponding to the target; when it is detected that two tracks monitored by different Leiyu all-in-one machines belong to the same target and need to be associated, a root node query function is adopted to check whether the two tracks have association conflicts, if the association conflicts exist, the association conflicts are dynamically adjusted, looked up and eliminated, and if the association conflicts do not exist, the association conflicts do not exist; if yes, associating the two tracks belonging to the same target; and for the two tracks of which the association conflict is eliminated, searching and searching sets, determining timestamps of root identities corresponding to the two tracks, taking the root identity corresponding to the timestamp with a small numerical value as the identity of the associated track, merging the tracks corresponding to the timestamp with a large numerical value into the track corresponding to the timestamp with a small numerical value, and carrying out association. The operation complexity can be remarkably reduced, and the system operation efficiency can be improved.

This invention discloses a vehicle-road dual-channel cooperative driving method for a vision-language-action model. In a vehicle-to-everything (V2X) cooperative perception scenario, the vehicle extracts an image frame at time t and divides it into several patches. The vehicle and cooperative attention weights for each patch are simultaneously calculated and fused. The fused attention weights are then divided into discrete semantic importance levels according to a quantization threshold. Corresponding control parameters are matched from a set of compression operators to perform feature compression processing. Next, a reinforcement learning agent outputs instructions for primary and backup channel allocation. The primary channel is used to send compressed features for all patches, while the backup channel performs redundant transmission for patches with key semantic meanings. The receiving end statistically analyzes various indicators and provides feedback. The vehicle continues to extract an image frame at time t+1 and repeats the above steps. After obtaining compressed features, an optimized reinforcement learning agent is used to transmit them through the primary and backup channels. This invention significantly improves transmission stability, real-time performance, and the efficient utilization of communication resources.

This invention discloses a multi-target cooperative perception method for connected autonomous vehicles, relating to the field of autonomous driving technology. It includes: acquiring environmental data to construct local perception features and compressing them to obtain compressed features; constructing a path loss and channel gain model and calculating communication transmission delay using orthogonal frequency division multiple access (OFDM); establishing end-to-end perception delay and energy consumption models to obtain the perception delay and energy consumption corresponding to each role; constructing a multi-target optimization model; asynchronously generating action policies through an Actor network with a self-attention mechanism and evaluating the global state value through a Critic network with a graph attention mechanism to output the action policies; decompressing and fusing the compressed features for target detection; calculating multi-target reward values ​​and updating the parameters of the Actor network and Critic network respectively. This invention achieves adaptive cooperation by combining asynchronous architecture and multi-target optimization with attention reinforcement learning, balancing accuracy and energy consumption, and improving perception efficiency and robustness.

The invention discloses an air-ground cooperative sensing method and device based on reinforcement learning and a readable medium, and relates to the field of cooperative sensing, and the method comprises the steps: inputting first image data collected by a vehicle and second image data collected by an aircraft into a feature extraction module in a trained sensing task classification recognition model, obtaining a first feature map and a second feature map of the current time slot, and converting to obtain an aligned second feature map of the current time slot; constructing the state of the current time slot and inputting the state into the trained strategy network to obtain the strategy of the current time slot, segmenting the aligned second feature map in combination with the state to obtain the segmented second feature map of the current time slot, and fusing the segmented second feature map with the first feature map of the current time slot through a feature fusion module to obtain a first feature map of the current time slot; and the fusion feature map of the current time slot is obtained, and the fusion feature map of the current time slot is input to a second feature decoding module to obtain the target category probability and the target spatial position of the current time slot. The problem of low sensing precision is solved.