84 results about "Object perception" patented technology

The invention discloses a three-dimensional dynamic scene graph construction method based on 3D Gauss, and belongs to the field of computer body intelligence. The implementation method comprises the following steps of: realizing object perception and semantic feature extraction of open vocabularies by utilizing a visual basic model; a 3D Gaussian scene with high fidelity and continuous object semantics is constructed through multi-view multi-dimension optimization of 3D Gaussian representation; constructing a multi-level three-dimensional scene graph, extracting spatial levels and semantic relationships among objects by using 3D spatial positions and semantic tags of instance objects existing in a semantic Gaussian graph, and constructing a multi-level spatial semantic topology to accurately represent an environment layout; according to the method for realizing local updating for the Gaussian scene graph based on the environmental structural similarity, environmental change detection is carried out through real-time RGB-D observation and the structural similarity between high-quality rendering views of the Gaussian scene graph, and corresponding local updating is carried out by using rapid training and differentiable rendering of 3D Gaussian representation. And the capability of adapting to a complex dynamic environment of the 3D Gaussian scene graph is improved.

The invention belongs to the technical field of computer vision. The invention provides a switch cabinet state identification method based on small target perception. According to the embodiment of the invention, the SPD non-stepping convolution is introduced into the backbone network of the YOLOv8 to replace the stepping convolution in the original backbone network, so that the feature extraction capability of the model is improved, and the loss of small target features is reduced. An SPPF module in a backbone network of YOLOv8 is improved into a multi-scale feature extraction module MSFEAC, so that multi-scale perception of a target can be realized, a receptive field is expanded, and global information of a small target of a switch cabinet indicator lamp is better reserved. And a multi-scale feature extraction module and a small target detection layer are added, so that the network can pay attention to more detail information, thereby improving the recognition effect of a small target, namely the state of a transformer substation drainage wire bolt and each indicator lamp of a switch cabinet, and being more suitable for an intelligent transformer substation application scene.

Systems, methods, and apparatuses are provided for animated representation of at least one object perception and of a driving intention of an assistance system for at least partially automated maneuvering of a vehicle. An object is perceived within an environment of the vehicle using the assistance system. The perceived object influences a control strategy of the at least partially automated maneuvering of the assistance system. The perceived object is represented. The vehicle is represented by an avatar. A first animation is displayed of an interaction between the avatar and the perceived object. The avatar is animated to mimic a first action based on the perceived object. A second animation is displayed representing the driving intention of the assistance system. The avatar is animated to mimic a second action based on a plan of the control strategy influenced by the perceived object.

The invention provides an image text description generation method, electronic equipment and a readable storage medium. According to the method, the object perception prototype learning module and the global context feature extraction module are introduced, so that fine-grained information and global semantic understanding in the image are effectively balanced. The visual backbone network module can extract multi-scale and multi-level image features and perform fusion, thereby enhancing the expression ability of the image features. The object perception prototype learning module further extracts an object prototype from the fusion features to ensure that the model can accurately capture key objects and attributes thereof in the image, and the global context feature extraction module ensures that the overall context of the image is fully understood. On the basis, the encoding and decoding module combines the global context and the object prototype to generate the text description, so that the semantic splitting phenomenon in the traditional method is avoided, and the detail information in the image is effectively reserved, thereby improving the accuracy and integrity of the image description.

The invention relates to the technical field of component detection, and provides a component detection method and system for executing cross-category object perception and control, and the method comprises the steps: S1, extracting backbone features of an object point cloud point by point; s2, component semantic classification is carried out, and a vector from each point to a component point cloud center to which the point belongs is predicted; s3, clustering point clouds of the centers of the parts; s4, normalizing the clustered point cloud component instances into a sphere with a unit radius, and performing feature extraction; s5, predicting the confidence coefficient by using the global features of the point cloud component instances, and correcting the point cloud component instances of which the confidence coefficient is below a preset threshold value by using a segmentation cutting basic model; s6, reasoning confidence coefficients of the point cloud component instances again, and deleting the point cloud component instances with relatively large overlap by adopting non-maximum suppression; and S7, using the point-by-point features of the point cloud component instance to predict the primary NOCS coordinates, and then performing secondary attitude estimation on the secondary NOCS coordinates. And the detection quality of the cross-object class part and the accuracy of attitude estimation are improved.

Embodiments of the present disclosure provide a method, device, and medium for training a large scale object foundation model. The method comprises obtaining a training dataset comprising a plurality of subsets for a plurality of object perception tasks, wherein a sample in the training dataset comprises an image with an object, a prompt indicating the object, and labeled object perception information of the image. The method further comprises generating, by the image encoder, an image feature based on the image. The method further comprises generating, by the text encoder or the visual prompt encoder, a prompt embedding based on the prompt. The method further comprises generating, by the object decoder, object perception information of the object based on the image feature and the prompt embedding. In addition, the method further comprises training the object processing model based on the generated object perception information and the labeled object perception information.

The invention discloses a dexterous hand environment article sensing and modeling algorithm based on body sensing. The algorithm comprises the steps that a mechanical arm drives a dexterous hand to approach an object; tactile-spatial data acquisition: a three-axis force sensor is arranged at the tail end of each finger of the dexterous hand, and the system is in contact with an object through multiple fingers of the dexterous hand to acquire multi-dimensional information of the appearance, surface characteristics and local topology of the object; active exploration sampling, multi-finger movement of the mechanical arm and the dexterous hand, and active penetration of unknown object surface and enhanced sampling of local features through the dexterous hand by the system; the method comprises the following steps: modeling an object attribute, analyzing a mechanical response sequence obtained by tactile-spatial data acquisition and active exploration sampling, modeling a target object in three dimensions of elasticity, hardness and local geometric morphology, and realizing accurate identification and modeling of an unknown object appearance form and an object attribute in an environment. The distribution uniformity of the tactile point cloud and the model integrity are improved, the hardware dependency is reduced, and the environmental adaptability is enhanced.

A method for (3D) object detection includes: receiving an input image with respect to a 3D space, an input point cloud with respect to the 3D space, and an input language with respect to a target object in the 3D space; using an encoding model to generate candidate image features of partial areas of the input image, a point cloud feature of the input point cloud, and a linguistic feature of the input language; selecting a target image feature corresponding to the linguistic feature from among the candidate image features based on similarity scores of similarities between the candidate image features and the linguistic feature; generating a decoding output by executing a multi-modal decoding model based on the target image feature and the point cloud feature; and detecting a 3D bounding box corresponding to the target object by executing an object detection model based on the decoding output.

A method for monitoring a performance of an object perception system of an automated driving system of a vehicle and related aspects is disclosed. The object perception system is configured to ingest sensor data samples generated by vehicle-mounted sensors and to output object perception data indicative of detected objects in a surrounding environment of the vehicle and one or more attributes of the detected objects. The method includes outputting reference data indicative of detected objects in the surrounding environment of the vehicle and of one or more attributes of the detected objects. The method further includes comparing the object perception data with the reference data and assigning confidence values to the object perception data based on the comparison. The method further includes controlling the vehicle, the object perception system, and / or downstream ADS functions configured to ingest the object perception data, based on the assigned one or more confidence values.

The invention discloses a sparse view angle three-dimensional Gaussian language field construction method and system for humanoid robot grabbing. Comprising the following steps: acquiring an RGB image set and a user natural language instruction under a sparse view angle; reconstructing a three-dimensional point cloud through stereo matching, and initializing a three-dimensional Gaussian primitive field after noise elimination and measurement alignment; two-dimensional semantic features are embedded into the field, joint optimization is carried out through a double-path semantic supervision module, and a three-dimensional Gaussian language field with consistent semantics is constructed; wherein the dual-path semantic supervision comprises an object perception path and a global context path, the local semantic consistency and the overall semantic relationship are constrained respectively, and the final semantic representation of each Gaussian primitive is obtained through weighted fusion; and finally, candidate grabbing postures are generated, semantic reordering is carried out in combination with a language instruction, and the optimal grabbing posture is screened through geometric-semantic joint scoring. Under the sparse image set input condition, the execution accuracy of a robot grabbing task under a complex instruction can be improved.

The invention provides an object sensing method and system based on proximity vision and tactile deformation, which are applied to the technical field of tactile sensing, and the method comprises the steps: obtaining a binocular image collected by a visual tactile sensor after a target object is in contact with a preset visual tactile sensor; performing visual feature extraction based on the binocular image to obtain a binocular feature point two-dimensional coordinate of the target object; extracting a tactile mark based on the binocular image to obtain a two-dimensional coordinate of a tactile mark point of the target object; through a three-dimensional reconstruction model based on ray tracing, based on the two-dimensional coordinates of the binocular feature points, determining approximate three-dimensional visual perception of the target object in the non-contact area; and determining contact three-dimensional deformation perception of the target object in the contact area through a three-dimensional reconstruction model based on ray tracing and based on the two-dimensional coordinates of the binocular feature points. According to the invention, the perception capability of the visual tactile sensor on the object state and the interaction degree can be improved.

The invention discloses a neural implicit vision SLAM (Simultaneous Localization and Mapping) method based on dynamic perception. The method aims at solving the core technical problems that an existing visual SLAM method is insufficient in robustness, poor in global consistency, large in calculation overhead and the like in challenging environments such as dynamic scenes, weak texture areas and violent illumination changes. According to the method, the feature processing capability of deep learning, efficient dynamic object perception, advanced neural implicit mapping and a global optimization mechanism are integrated, so that more accurate camera pose estimation and higher-quality static environment map construction are realized. In the tracking module, a six-step workflow based on mask guidance is adopted, dynamic objects are filtered from the source, frame-level pre-screening is carried out, and the robustness and the calculation efficiency of the system are remarkably improved. In a dynamic local mapping module, a pixel-level fusion method based on transmission probability and inverse variance weight is innovatively adopted, texture blurring and geometric distortion at the boundary of a plurality of sub-maps are effectively inhibited, and the visual quality of a global map is improved. Besides, by introducing a loop candidate frame reordering strategy based on pose uncertainty weighting in loop detection, visual similarity and geometric credibility can be combined, the false detection rate is effectively reduced, and global consistency and long-term precision of the map are ensured.

The invention discloses an LVM guided infrared and visible light image fusion method based on object perception and context contrast learning, and belongs to the technical field of image fusion processing. According to the method, the feature interaction fusion network is constructed, the feature interaction fusion block is designed, and global space information is reserved while an object specific to a modal is highlighted. And an SAM guide mask generator is provided, and a complete detection frame is obtained by using a text-driven target detection model and is used as a frame prompt in the SAM. For comparative learning in a multi-modal task with content information, a context space is constructed to calculate the distance between samples. In order to provide more cohesive samples for a context space, a feature interaction fusion network is designed, key features are strategically and preferentially considered, meanwhile, complex inter-modal relationships are captured, and context-related information is highlighted through a feature interaction fusion block. According to the method, the scene information can be reserved while the integrity of the highlighted object in the fused image is consolidated.

The invention provides an object-oriented hierarchical memory network device. The object-oriented hierarchical memory network device comprises a low-level object sensing module, a space-time memory construction module and a long-term space-time memory modeling module, the low-level object sensing module is used for extracting object features of an input video through a segmentation basic model; the space-time memory construction module is used for extracting object interaction and scene information by analyzing an inter-frame space and time relationship so as to splice object features and input original video frames to generate a space representation of target perception; and the long-term space-time memory modeling module is used for storing historical information through a memory bank, capturing a global dependency relationship of a video level, and splicing a plurality of adjacent video frames output by the space-time memory construction module on a channel dimension to form local multi-frame space-time representation. And the purpose of enhancing the capturing capability of the model on fine-grained space-time dynamic in the long video is achieved.

The invention discloses a visual language large model perception enhancement method and system based on visual prompt, and relates to the field of visual language large models. The problem that how to deploy a small-scale large-language model under the condition that resources are limited in the prior art becomes an urgent problem to be solved is solved. The method comprises the following steps of: segmenting an original image by adopting a segmentation component to generate a mask and an object segmentation list; respectively processing an original image and an image mask generated by a semantic segmentation device by using a visual encoder, so as to extract multi-level visual features which highlight the position and boundary of an object; layer normalization and MLP layer processing are carried out to form visual features; taking the generated mask and the segmentation result list of the object as a text instruction, and inputting the extracted multi-level visual features which highlight the position and the boundary of the object and the visual features into a visual language large model to perform autoregressive semantic generation; the method is also suitable for the technical field of improving the object perception and question-answering ability of the visual language large model without adding extra training parameters.

The application discloses a threat object detection method based on reinforcement learning, and aims to improve the threat object detection accuracy in the field of automatic driving. The technical scheme is to construct a threat object reasoning and detection system based on reinforcement learning, which is composed of a strategy model, a reward value calculation module, a group advantage calculation module and a strategy model updating module. The strategy model in the system is trained to obtain a trained threat object reasoning and detection system with threat reasoning and threat object perception capabilities. Finally, the trained threat object detection system is used for threat object detection to obtain the boundary box and threat level of the threat object. The application enables the multi-modal large language model to realize fine-grained object-level threat reasoning based on the context information of the driving scene, significantly improves the understanding ability of the automatic driving system to threat semantics and the perception ability of the threat object, and improves the threat object detection accuracy in the field of automatic driving.

The invention discloses a sorting system and method based on machine vision, and the sorting system based on machine vision comprises an image collection module, a data processing module and a sorting execution module. The image acquisition module is used for synchronously acquiring a hyperspectral image and three-dimensional point cloud data of a target area. And the data processing module is in communication connection with the image acquisition module and comprises a multi-modal feature fusion unit and a dynamic sorting decision unit. According to the invention, through multi-modal information fusion, chemical components and geometric space information of an object can be obtained at the same time, and more comprehensive object perception is realized; a fusion network based on an attention mechanism is adopted, different modal features are weighted in a self-adaptive mode, and the recognition precision is improved; physical constraints and task targets are comprehensively considered, an optimal grabbing point and a movement track are generated, and the sorting success rate is increased; the method has a self-learning capability, and can continuously optimize the system performance from practical experience.

Systems and methods for cooperative perception are described. In some examples, the system can comprise a first subsystem comprising a first sensor, a first communication device, and a processor, which can cause the system to: detect, by the first sensor, first point cloud data, apply a data preprocessing process to the first point cloud data to generate first preprocessed sensor data, apply a feature encoding process to the first preprocessed sensor data to generate first feature data; apply an adaptive feature filtering process to the first feature data to select a first subset of features from the first feature data; apply a cooperative feature aggregation process to fuse the first subset of features with other subsets of features, to generate a fused feature map; and apply an object perception model to the fused feature map to generate object perception data.

The invention discloses a space object sensing method based on a privacy protection camera and a radar, and the method comprises the steps: obtaining the three-dimensional space characteristics of a current space through the radar, judging whether a human body exists in the current space or not through the three-dimensional space characteristics, and starting the privacy camera to collect a global image if the human body exists in the current space; obtaining two-dimensional space features of the current space through the global image, and fusing the two-dimensional space features and the three-dimensional space features of the current space to obtain fused space features of the current space; inputting the fusion feature of the current space into a pre-trained object recognition model to obtain an object type of the current space; interference source labels and universal furniture labels are marked according to the object types of the current space, the current space objects with the interference source labels are filtered, and universal furniture distribution of the current space is obtained; and according to the current space general furniture distribution, radar detection parameters are adjusted according to detection areas, and detection thresholds of different detection areas are obtained.

The invention discloses a self-supervised learning method of a point cloud combining partial-overall perception, and belongs to the technical field of artificial intelligence. Aiming at the problem that the invariant perception of point cloud features is generally enhanced by expanding point cloud data and the influence of part of information on object perception is ignored in the existing method, the method comprises the following steps: preprocessing the point cloud data, extracting enhanced point cloud and part of point cloud feature vectors, and then enhancing the similarity condition of the point cloud using comparison loss evaluation feature vectors; a part of feature vectors of the point cloud are obtained through a point cloud feature extractor network and a point cloud feature projection head network, the part of feature vectors are put into an implicit field to learn implicit representation of the part, and the feature vectors of the enhanced point cloud are integrated to obtain an integrated overall feature vector and an integrated partial feature vector; and comparing the difference between the two feature vectors, and performing back propagation to optimize the network according to the difference condition. According to the method, the relation between local and overall point cloud is fully combined, and richer point cloud information representation is obtained.

The embodiment of the invention provides a scene recognition method and device based on contrast invariant risk minimization, a storage medium and electronic equipment, and belongs to the technical field of scene recognition. The method comprises the steps of: dividing a sample into a plurality of environments; obtaining image features and object perception features; extracting verb invariant features and role specific features; respectively setting verb classifiers for different environments, and introducing invariant risk minimization constraint and contrast learning loss; identifying and positioning each entity in the scene, and generating a corresponding target category and a bounding box result; and through training, carrying out joint optimization on verb classification loss, role recognition loss and auxiliary loss. The method can effectively improve the accuracy of the model on action recognition, noun recognition and position positioning, and has better cross-domain generalization performance.

A method is provided for determining a pitch angle of a perception system of a vehicle with respect to a road surface on which the vehicle is located. Via the perception system, road perception information is detected from the road surface, and object perception information is detected from objects being different from the road surface. A primary pitch angle is determined by using the road perception information, and a relative pitch angle is determined by using the object perception information. The primary pitch angle and the relative pitch angle are combined in order to determine a final pitch angle of the perception system.

The invention provides a target object sensing method, a communication device and a program product, relates to the technical field of communication, and can at least solve the problem of low target sensing accuracy in related technologies. The method comprises the following steps: acquiring perception information of a target object perceived by a plurality of perception devices at different positions; and selecting at least two pieces of target perception information from the multiple pieces of perception information, and confirming the behavior or position of the target object according to the at least two pieces of target perception information. The target sensing accuracy can be improved.

The invention provides an object sensing method and device based on binaural auditory signal expression and a hearing aid, and belongs to the technical field of hearing aids. According to the object sensing method based on binaural auditory signal expression, radar sensing, object analysis, target identification, spatial sound effect simulation, hearing aid broadcasting and other technologies are integrated; visual perception of the target object to the surrounding environment in a vision-limited or hearing-limited scene is achieved, the target object can know the dynamic state of the surrounding environment in real time without depending on visual information, the auxiliary ability of the wearable device to a user in a complex scene is enhanced, and compared with traditional visual display or voice prompt, the feedback mode of the hearing signal has the advantage that the user experience is improved. The method is more suitable for the requirements of vision-limited users, hearing-limited users or in specific complex scenes, and the use experience of the target object can be effectively improved.

The embodiment of the invention provides a target detection method and device based on vision and radar, a vehicle and a medium. The method comprises the steps that first target object data and second target object data of a vehicle driving environment at the current moment are acquired, and the first target object data and the second target object data are fused to generate third target object data; and finally, determining fourth target object data according to the third target object data and historical third target object data generated at the previous moment. Wherein the first target object data is acquired through a camera, and the second target object data is acquired through a radar. The method is used for achieving the technical effect of improving the sensing precision and robustness of the target object.

A communication system, apparatus, method, and one or more non-transitory computer-readable storage devices for cooperative perception in communication perception integration employ the steps of: determining a type of a first communication node for object perception; and notifying the first communication node of the determined type. The step of determining the type of the first communication node comprises the following steps of: determining the type of the first communication node as a sensing transceiving node in a sensing transceiving set so as to receive or send a sensing signal for sensing an object, or determining the type of the first communication node as a sensing activation node in a sensing activation set so as to receive or send a sensing signal for sensing an object; wherein the sensing activation set comprises the sensing transceiving set.

The application discloses a sparse-view three-dimensional Gaussian language field construction method and system for humanoid robot grasping. The method comprises the following steps: acquiring an RGB image set under sparse-view and a natural language instruction of a user; reconstructing a three-dimensional point cloud through stereo matching, initializing a three-dimensional Gaussian primitive field after noise elimination and metric alignment; embedding two-dimensional semantic features in the field, jointly optimizing through a double-path semantic supervision module, and constructing a semantic-consistent three-dimensional Gaussian language field; wherein the double-path semantic supervision comprises an object perception path and a global context path, which respectively constrain local semantic consistency and overall semantic relationship, and the final semantic representation of each Gaussian primitive is obtained through weighted fusion; finally, candidate grasping postures are generated, semantic reordering is performed in combination with the language instruction, and the optimal grasping posture is selected through geometric-semantic joint scoring. The method can improve the execution accuracy of the robot grasping task under complex instructions under the condition of sparse image set input.

An object processing system within a trailer for a tractor trailer is disclosed. The object processing system includes an input area of the trailer at which objects to be processed may be presented, a perception system for providing perception data regarding objects to be processed, and a primary transport system for providing transport of each object in one of at least two primary transport directions within the trailer based on the perception data.

The invention discloses a robust three-dimensional Gaussian sputtering new view angle synthesis method in the presence of interferents. The method comprises the following steps: acquiring scene images of a plurality of view angles and corresponding camera parameters; based on the scene image and the camera parameters, carrying out three-dimensional scene modeling by utilizing three-dimensional Gaussian sputtering, and carrying out iterative optimization to construct a three-dimensional Gaussian model; and using the three-dimensional Gaussian model to render and generate a new view angle image. Interferents in a scene can be automatically identified and ignored, and manual labeling is not needed; by learning a neural decision boundary and introducing object perception, the accuracy of interferent identification is improved; a high-quality new view angle synthesis result is obtained in a scene with interferents, and meanwhile, good background reconstruction quality is kept; the method has good generalization and can process different types and numbers of interferents.

This application provides a vehicle control method and apparatus, storage medium, and electronic device, relating to the fields of smart mines, autonomous driving, and unmanned vehicles. The method includes: fusing the perception results of a first target object acquired by a lidar and the perception results of a second target object acquired by a millimeter-wave radar to obtain a target object perception queue; and fusing the target object perception queue with a first target object tracking queue using a target fusion strategy to obtain a second target object tracking queue. The target fusion strategy includes: for each perception element, if a matching first tracking element exists, determining whether its source includes both lidar and millimeter-wave radar; if both are included, determining the target sensor speed information based on the respective speed information of the lidar and millimeter-wave radar, and then updating the first tracking element to obtain the second tracking element. This application ensures the reliability of target object detection and improves the safety and efficiency of autonomous driving in mines.