85 results about "Perceptual information" patented technology

The invention provides a multi-agent collaborative visual completion method and system based on semantic communication, and the method comprises the steps: a first agent collects an occluded overall original image through a visual sensor, generates an occluded region mask through a lightweight semantic segmentation network, extracts mask semantic information, and transmits the mask semantic information to a second agent; the second intelligent agent matches the global semantic feature of the second intelligent agent with the received mask semantic information, locates a sheltered area, then cuts the sheltered area to obtain a semantic feature vector of the area, and returns the semantic feature vector to the first intelligent agent; and after the first intelligent agent receives the image, performing high-fidelity reconstruction on the occlusion area by using a conditional diffusion model to obtain a complemented image with consistent semantics, and performing weighted fusion and local color matching and splicing on the complemented image and the original image to obtain a global perception result. According to the scheme, perceptual information sharing and occlusion area reconstruction with low bit rate, high robustness and high real-time performance can be realized under limited communication resources.

The invention discloses a time delay constrained Internet of Vehicles multi-mode cooperative sensing method and system. The method comprises the following steps: collecting point cloud data and image data, and carrying out feature extraction, fusion and space projection; constructing a two-dimensional convolution neural network, and extracting a position-level importance probability graph and a mask used for screening key information; constructing a benefit estimation neural network and a constraint evaluation neural network, and predicting expected values of system benefits and constraint conditions; meanwhile, an overall strategy selection module is established; the motorcade carries out target identification and calculates rewards based on information interaction so as to obtain a perception experience sample; and updating network parameters based on the perception experience sample to obtain a final overall strategy selection module. According to the method, the transmitting power, the transmission channel and the sensing data area of the vehicle are optimized based on reinforcement learning, a strategy correction mechanism is introduced, a risk strategy of high time delay or high packet loss is dynamically corrected, and target identification deviation and decision errors caused by sensing information delay and wrong transmission are avoided.

The invention relates to the technical field of vehicles, in particular to a multi-mode out-of-vehicle interaction method and related equipment based on environment self-adaptation, and the method comprises the steps: obtaining perception information in real time through an environment perception module, the perception information comprising environment conditions, interaction object information and scene semantic understanding information; based on the perception information, utilizing an adaptive decision engine to dynamically analyze an optimized interaction mode and parameters, and generating interaction content; outputting interaction information according to the interaction content generated by the self-adaptive decision engine through a multi-modal output module; the interaction strategy is dynamically adjusted by sensing the environment state in real time, and the safety and the interaction efficiency can be improved.

The invention discloses a space robot compliance control method and device based on multi-dimensional visual constraints, and belongs to the technical field of space robots and autonomous control. The method comprises the following steps: collecting and fusing data through a heterogeneous visual sensor group, and carrying out uncertainty quantification to generate a perception confidence index; constructing and dynamically adjusting a visual constraint function set based on the index, solving a motion feasible region according to the visual constraint function set, and generating an expected motion track; inputting the track into an impedance controller, and adjusting the rigidity and damping parameters in real time according to the sensing confidence so as to output an instruction; the instruction is verified through digital twin simulation, and if the instruction does not pass the verification, a fault-tolerant mechanism is triggered for optimization; and finally, updating the system parameters on line according to the deviation between the actual execution data and the simulation data. According to the method, the technical problems of poor operation safety, low control precision and insufficient autonomy of the space robot in a complex environment caused by single sensing information dimension, lack of uncertainty quantization and fixed control strategy are solved.

The invention discloses a traffic subject collaborative cognition method based on semantic representation in the technical field of traffic transportation. The method comprises the following steps: S1, collecting original data including state information of a traffic scene and a traffic subject, integrating the original data into perception information through a data fusion technology, and converting the perception information into structured semantic representation information; s2, establishing a multi-mode collaborative interaction architecture for different traffic scenes, dynamically selecting the collaborative interaction architecture according to a real-time traffic state, and performing ordered information interaction according to tasks; and S3, the edge traffic subjects autonomously generate decisions, and the central traffic subject performs overall optimization to realize multi-traffic subject collaborative cognition and behavior consistency. According to the method, semantic mutual understanding among different traffic subjects is realized, and an information barrier is broken; the method realizes the balance between local decision and global optimization, improves the decision efficiency, and has good expandability and adaptability.

The invention provides a multi-mode sensing inspection method based on terminal equipment. The multi-mode sensing inspection method comprises the following steps: collecting video data, audio data, geographic position data and IMU (Inertial Measurement Unit) data; video frames are extracted based on language trigger thinning, pose trigger thinning and visual trigger thinning; and finally, identifying hidden dangers based on the extracted video frames. According to the method, significant thinning processing of the video frames is realized through various kinds of perception information, the input load of the large model is greatly reduced on the premise that key semantic information is not missed, the problems of frame input limitation and low analysis efficiency of the multi-mode large model in analysis of the long-time industrial inspection video are solved, and the method is suitable for large-scale industrial inspection. Therefore, the efficiency and accuracy of large model analysis are improved. And the frame extraction process does not depend on a fixed frame rate, but is based on semantic driving of human behaviors and equipment states, so that the extracted key frame is more effective.

The present invention relates to an embodied intelligence agent based on physical intelligence and active perception, and a control method therefor. The method controls the embodied intelligence agent on the basis of a multimodal embodiment large model, and the multimodal embodiment large model comprises a data processing module, a perception and planning module, a physical intelligence module, an information transmission module, an environment interaction module, and a motion control execution module. The method comprises: the perception and planning module receives data processed by the data processing module and, according to the data, outputs perception information and a task planning instruction; the physical intelligence module receives the perception information and the task planning instruction, and outputs a motion action planning instruction; the environment interaction module collects environment information in real time and compares same with the perception information; and the motion control module receives the motion action planning instruction and incorporates the comparison result to perform action correction so as to implement control of the embodied intelligence agent. Compared with the prior art, the present invention has the advantages of high generalisation performance and high task execution accuracy.

A controller may obtain perception information from a perception sensor system of the machine that includes a plurality of sensors, wherein the perception information includes respective perception data captured by the plurality of sensors. The controller may determine validity information associated with the perception information. The controller may select, based on the validity information, a portion of the perception information. The controller may determine, based on the portion of the perception information, a current distance to a target.

The invention provides an AI intelligent agent system, and the system comprises a perception processing module which is used for receiving external perception information and processing the external perception information through a corresponding model, and obtaining corresponding perception data; the cache middleware module is used for storing process data, and the process data comprises the perception data and / or context data; the agent scheduling module is used for acquiring a user instruction, acquiring the stored process data from the cache middleware module, filling the instruction and the process data into a cue word template, and delivering the cue word template to the LLM after fine adjustment; the LLM after fine tuning is used for reasoning according to the content of the cue word template to generate a reasoning result. The invention further provides a corresponding working method of the AI agent system. According to the method, the interpretability, the maintainability, the higher generalization ability and the higher response speed of modular design can be achieved at the same time.

The embodiment of the invention provides a sensing method and a related device. The method comprises the following steps: acquiring first sensing information of a plurality of communication devices for a first area; the multiple pieces of first perception information are input into a perception model, second perception information is obtained, and the perception model is used for predicting new perception information according to the input perception information; and optimizing the perception model according to the second perception information and third perception information of the first area, the third perception information being a reference value of the perception information of the first area. According to the embodiment of the invention, the accuracy of sensing information can be improved, so that the sensing performance of a communication network is improved.

A control unit (114) can receive perceptual information from a perception sensor system (112) of the machine (100), which comprises a plurality of sensors (126). The perceptual information includes respective perception data acquired by the plurality of sensors (126). The control unit (114) can determine validity information associated with the perceptual information. Based on the validity information, the control unit (114) can select a portion of the perceptual information. Based on this portion of the perceptual information, the control unit (114) can determine a current distance to a target.

The application discloses a kind of memory hub methods and systems for multi-agent cognitive decision-making, comprising: obtaining the perception information, behavior result or decision context generated in the running process of cognitive system;The perception information is structured modeling, and memory unit with time attribute, access attribute and evolution attribute is generated;The memory unit is stored and managed in life cycle;Before or in the process of cognitive decision generation, according to the current decision context, select relevant historical experience from the memory unit to participate in decision formation;Based on the use effect of memory unit in the decision process, update the hierarchical state or call strategy of the memory unit, and construct the continuously evolving cognitive decision process based on historical experience feedback. Unified management and monitoring can be carried out on memory access behavior, permission configuration, life cycle state and abnormal conditions, and parameter configuration and state viewing can be supported during system operation.

The invention provides a multi-scene state machine fused four-wheel-foot robot motion control method and system, and relates to the technical field of intelligent control, and the method comprises the steps: obtaining the perception information of a robot body; according to the perception information, constructing an observation space vector by fusing the current control instruction; inputting the observation space vector into a deep reinforcement learning control network based on an actor-commentator architecture, and generating an action instruction; training a deep reinforcement learning control network through a near-end strategy optimization algorithm according to the action instruction and a preset multi-dimensional reward function, and optimizing a motion control strategy; based on the optimized motion control strategy, constructing a finite-state machine framework, and defining a plurality of motion states; and according to the current topographic features and task requirements, the four-wheel-foot robot is controlled to be gradually switched among all the motion states through a preset state transition condition.

The invention discloses a sensing information sending method and device, a sensing information receiving method and device, equipment and a storage medium, and relates to the technical field of communication. The method comprises: a first node sending perception information within a valid interval (410). By constraining the effective interval of the sensing information, the sensing node sends the sensing information in the effective interval, but the sensing information not in the effective interval does not need to be sent, the situation that the whole amount of sensing information is fed back all the time is avoided, the processing complexity and energy consumption of the sensing node can be reduced, the feedback quantity can be reduced, and the sensing efficiency is improved. Transmission resources are saved, and the transmission efficiency of sensing information is improved.

This invention discloses an intelligent puncture control method, system, surgical aid, and medium based on ultrasound images, relating to the field of medical device technology. The method includes: constructing a three-dimensional environment model based on real-time acquired target perception information; determining a target control path based on the three-dimensional environment model; controlling the surgical aid to perform coordinated movement according to the target control path, thereby driving an actuator operably connected to the surgical aid to move along the target control path; monitoring the perception information during the movement of the actuator, and triggering a protective response when preset risk conditions are met. This addresses the problem of intraoperative anatomical deviations caused by reliance on preoperative static imaging in traditional surgery; by continuously monitoring perception information during the movement of the actuator and triggering a protective response through preset risk conditions, dynamic safety control of the surgical process is achieved, effectively avoiding damage to risk structures by the actuator.

The embodiment of the invention relates to an interaction method and device, equipment and a storage medium. The method comprises the steps that whether the position of a virtual object in a virtual scene meets a preset condition or not is determined; in response to the fact that the position meets the preset condition, perception information associated with the position is obtained, the perception information describes an interaction environment associated with the position and / or an interaction event associated with the position, and the type of the perception information is determined based on the met preset condition; and driving the virtual object based on the perception information. In this way, according to the embodiment of the invention, the interaction efficiency in the virtual scene can be improved.

The invention provides a target detection method, device and system, and relates to the field of wireless communication. In the method, a perception management device can obtain first position information used for indicating the position of a first target at a second moment at a first moment, and obtain first perception information. Wherein the second moment is later than the first moment, and the first sensing information indicates the information of the first target sensed at the second moment according to the first position information. The first sensing information is obtained by sensing the first target according to the position indicated by the second moment and the first position information, so that the pertinence is better, and the obtained first sensing information is more accurate. Subsequently, the perception management device can determine the position information of the first target at the second moment according to the first perception information. The first sensing information is relatively accurate, so that the position information obtained according to the first sensing information is also relatively accurate. Therefore, the target detection method, device and system provided by the invention can improve the positioning precision.

The invention discloses a communication method and device, which can reduce the feedback overhead of sensing information based on the sparsity of wavelet coefficients. The method comprises the following steps: a device for obtaining the sensing information performs M-level discrete wavelet transform on the sensing information to obtain a sensing information codebook, and sends the sensing information codebook to a network, and the network can recover the sensing information based on the sensing information codebook. Wherein the perceptual information can comprise perceptual data of each position point in the perceptual space. The perceptual information codebook includes a highest decomposition level (i.e., Mth level) scale coefficient codebook and all decomposition levels (i.e., M) wavelet coefficient codebooks. Each wavelet coefficient book in the M wavelet coefficient books is in one-to-one correspondence with each level of discrete wavelet transform, the jth wavelet coefficient book comprises a part of wavelet coefficients in wavelet coefficients obtained after the jth level of discrete wavelet transform, j = 1, 2,..., M, and M is a positive integer greater than or equal to 1.

The invention provides an image processing method and device, equipment, a storage medium and a program product, and relates to application of an artificial intelligence technology in the field of image processing. The method comprises the following steps: determining an original multi-view image of a driving scene and a target object to be added into the driving scene; processing the original multi-view image based on the target object to generate multi-view perception information containing the target object; wherein the multi-view perceptual information comprises at least one of the following items consistent with the original multi-view image in size: a depth image, a normal image, an edge image, an object cutting image and an object mask image; and inputting a pre-trained model by taking the multi-view perceptual information as a guide condition to generate an image, and obtaining a synthetic multi-view image containing the target object. According to the method, the multi-view perception information containing the target object can be generated according to the original multi-view image and the to-be-added target object, and the multi-view perception information is taken as a guide condition to synthesize the multi-view image, so that the multi-view three-dimensional rationality of a generation result is ensured.

This invention discloses an integrated wheeled and legged robot with omnidirectional mobility, belonging to the field of mobile robot technology. It includes several integrated wheel sets, allowing free switching between wheeled and legged locomotion modes; a control module that integrates sensory information, environmental data, and robot status, and makes decisions and plans actions based on this information; a power supply module, composed of batteries, providing the necessary electrical energy for the entire integrated wheeled and legged robot; an intelligent sensing system, installed on the side of the vehicle body, enabling efficient perception and data acquisition of environmental information to support the robot's flexible movement, obstacle crossing, and path planning capabilities under different terrain and obstacle conditions; and a task execution device, installed on the vehicle body, for performing task operations. This invention can efficiently switch between wheeled and legged modes, reducing the robot's mechanical complexity, improving control accuracy, and exhibiting better stability, adaptability, and obstacle crossing capabilities.

The application discloses a method for self-adaptive region low-light enhancement and related equipment, and the method comprises the following steps: acquiring a low-illumination image, inputting the low-illumination image into a perception guide map module to obtain guide image information; obtaining a guide map based on the guide image information, inputting the guide map into a multi-scale dilation convolution attention mechanism module to obtain a first image after detail repair; performing a global average pooling operation on the first image to obtain a second image after enhancement, and processing the second image to obtain a low-light enhancement image. The purpose of the application is to improve the visual effect under low-light conditions, to make the neural network adaptively enhance the low-light image in a region-dependent manner by proposing a self-adaptive learning guide map, and to further assimilate the perception information of the adaptive guide map through a multi-scale receptive field module, so that the model learns more abundant and more real texture details.

This invention provides a method and system for understanding long-form traffic videos that combines generation and cognition. Through latent semantic modeling and a perceptual reflection loop mechanism, it achieves continuous cognition of traffic scenes over long time scales, avoiding reliance on only local fragment information and improving the overall understanding of long-form traffic videos. By introducing traffic scene generation and future state prediction, the system can simultaneously consider past and potential states, improving its ability to judge complex traffic problems and enhancing its cognitive and reasoning abilities regarding future traffic states. By unifying the representation of historical perception information and future generated information, information fragmentation is reduced, improving the consistency and stability of cognitive reasoning. This solution integrates historical perception and future generation of traffic scenes into a perceptual reflection loop, achieving a forward-looking and holistic cognitive understanding of long-form traffic videos, providing a sound theoretical foundation for intelligent traffic monitoring and event analysis systems.

This invention discloses a time-delay-constrained multimodal cooperative perception method and system for vehicle-to-everything (V2X) networks, comprising: acquiring point cloud data and image data and performing feature extraction, fusion, and spatial projection; constructing a two-dimensional convolutional neural network to extract a position-level importance probability map and a mask for filtering key information; constructing a benefit estimation neural network and a constraint evaluation neural network to predict the expected values ​​of system benefits and constraint conditions; simultaneously establishing an overall strategy selection module; the vehicle fleet performing target recognition and calculating rewards based on information interaction to obtain perception experience samples; updating network parameters based on perception experience samples to obtain the final overall strategy selection module. This invention optimizes vehicle transmission power, transmission channels, and perception data areas based on reinforcement learning, and introduces a strategy correction mechanism to dynamically correct risky strategies with high latency or high packet loss, avoiding target recognition deviations and decision-making errors caused by perception information delays and erroneous transmission.

The invention provides a perception information acquisition method, device and equipment, a computer readable storage medium and a computer program product, and relates to the technical field of Internet of Things, the method is applied to first equipment, and the first equipment is electronic equipment with a perception function in all electronic equipment located in an environment space. The method comprises the following steps: determining electronic equipment which is connected to the same local area network with first equipment and has a sensing function from an environment space, and obtaining a target equipment set; associating the first device with at least one electronic device in the target device set to obtain a sensing network topology; and sensing the environment space based on the sensing network topology to obtain a sensing result. The physical limitation that traditional single link CSI sensing is limited by an elliptical sensing area with the AP and the STA as focuses is broken through, sensing blind areas can be reduced or even eliminated, high-precision cooperative sensing can be carried out on all local space areas in the environment space, and the sensing range of the whole environment space is effectively expanded.

Embodiments of the invention relate to a first communication device (100) for operating in different transmission modes. When operating in a first transmission mode (TM1), the first communication device (100) transmits a data signal to a second communication device (300). When operating in a second transmission mode (TM2), the first communication device (100) sends joint data and a perceived signal to a second communication device (300). Accordingly, the second communication device (300) may be operated in a first reception mode (RM1) to receive the data signal from the first communication device (100), and may be operated in a second reception mode (RM2) to receive the joint data and a perceived signal from the first communication device (100). Therefore, data and perception information can be transmitted and / or received flexibly and efficiently. The invention further relates to a corresponding method and to a computer program.

This application provides a control method for autonomously planning safe grasping and flipping of a robotic arm, comprising: activating the multimodal perception system of the robotic arm to collect basic perception information of the package to be processed, wherein the robotic arm includes a robotic hand and a robotic arm; performing multi-dimensional state analysis based on the basic perception information, wherein the multi-dimensional state analysis includes one or more of the following: gripping stability judgment, cooperative operation adaptability judgment, energy consumption efficiency judgment, and safety risk judgment; and adaptively adjusting based on the results of the multi-dimensional state analysis, controlling the robotic arm to perform the package flipping operation step by step in the order of positioning, axial alignment, flipping, and placement. This application, through multimodal perception and multi-dimensional adaptive control, enables the bionic dexterous arm of the robotic arm to autonomously adjust its gripping strategy and motion trajectory according to the collected basic information, achieving high-success-rate grasping, low-damage flipping, and safe stacking, suitable for irregularly shaped, fragile, and high-speed logistics scenarios.

The embodiment of the invention provides a sensing method. The sensing method comprises the steps that first sensing measurement configuration from network equipment is received, the first sensing measurement configuration is used for indicating a sensing measurement period, and N times of sensing tasks are executed in the sensing measurement period. And executing N times of sensing tasks in the sensing measurement period, and obtaining N pieces of sensing information corresponding to the N times of sensing tasks. And if the N pieces of perception information meet the first condition, first indication information is sent to the network equipment, and the first indication information is used for the network equipment to send the second perception measurement configuration. And receiving a second sensing measurement configuration, and executing P times of sensing tasks in the sensing measurement period according to the second sensing measurement configuration.

The application discloses a water surface environment perception method based on Bayesian-fuzzy fusion, which comprises the following steps: (1) collecting multi-modal environment perception information, performing time alignment and space alignment on the collected modal environment perception information, and identifying obstacles in the current environment based on the multi-modal environment perception information; (2) predicting the probability P that a target obstacle in the current environment belongs to a real obstacle based on a Bayesian inference model and a fuzzy inference model, and identifying the target obstacle with a large probability P as an obstacle. Through the hierarchical coupling of Bayesian inference and fuzzy discrimination, the probability uncertainty and fuzzy uncertainty problems in the complex water environment are simultaneously solved, so that the accuracy and stability of the environment perception result are improved, and the perception robustness in the complex scene is also improved.