51 results about "Object perception" patented technology

The invention provides an image text description generation method, electronic equipment and a readable storage medium. According to the method, the object perception prototype learning module and the global context feature extraction module are introduced, so that fine-grained information and global semantic understanding in the image are effectively balanced. The visual backbone network module can extract multi-scale and multi-level image features and perform fusion, thereby enhancing the expression ability of the image features. The object perception prototype learning module further extracts an object prototype from the fusion features to ensure that the model can accurately capture key objects and attributes thereof in the image, and the global context feature extraction module ensures that the overall context of the image is fully understood. On the basis, the encoding and decoding module combines the global context and the object prototype to generate the text description, so that the semantic splitting phenomenon in the traditional method is avoided, and the detail information in the image is effectively reserved, thereby improving the accuracy and integrity of the image description.

Embodiments of the present disclosure provide a method, device, and medium for training a large scale object foundation model. The method comprises obtaining a training dataset comprising a plurality of subsets for a plurality of object perception tasks, wherein a sample in the training dataset comprises an image with an object, a prompt indicating the object, and labeled object perception information of the image. The method further comprises generating, by the image encoder, an image feature based on the image. The method further comprises generating, by the text encoder or the visual prompt encoder, a prompt embedding based on the prompt. The method further comprises generating, by the object decoder, object perception information of the object based on the image feature and the prompt embedding. In addition, the method further comprises training the object processing model based on the generated object perception information and the labeled object perception information.

The invention discloses a dexterous hand environment article sensing and modeling algorithm based on body sensing. The algorithm comprises the steps that a mechanical arm drives a dexterous hand to approach an object; tactile-spatial data acquisition: a three-axis force sensor is arranged at the tail end of each finger of the dexterous hand, and the system is in contact with an object through multiple fingers of the dexterous hand to acquire multi-dimensional information of the appearance, surface characteristics and local topology of the object; active exploration sampling, multi-finger movement of the mechanical arm and the dexterous hand, and active penetration of unknown object surface and enhanced sampling of local features through the dexterous hand by the system; the method comprises the following steps: modeling an object attribute, analyzing a mechanical response sequence obtained by tactile-spatial data acquisition and active exploration sampling, modeling a target object in three dimensions of elasticity, hardness and local geometric morphology, and realizing accurate identification and modeling of an unknown object appearance form and an object attribute in an environment. The distribution uniformity of the tactile point cloud and the model integrity are improved, the hardware dependency is reduced, and the environmental adaptability is enhanced.

A method for monitoring a performance of an object perception system of an automated driving system of a vehicle and related aspects is disclosed. The object perception system is configured to ingest sensor data samples generated by vehicle-mounted sensors and to output object perception data indicative of detected objects in a surrounding environment of the vehicle and one or more attributes of the detected objects. The method includes outputting reference data indicative of detected objects in the surrounding environment of the vehicle and of one or more attributes of the detected objects. The method further includes comparing the object perception data with the reference data and assigning confidence values to the object perception data based on the comparison. The method further includes controlling the vehicle, the object perception system, and / or downstream ADS functions configured to ingest the object perception data, based on the assigned one or more confidence values.

The invention discloses a sparse view angle three-dimensional Gaussian language field construction method and system for humanoid robot grabbing. Comprising the following steps: acquiring an RGB image set and a user natural language instruction under a sparse view angle; reconstructing a three-dimensional point cloud through stereo matching, and initializing a three-dimensional Gaussian primitive field after noise elimination and measurement alignment; two-dimensional semantic features are embedded into the field, joint optimization is carried out through a double-path semantic supervision module, and a three-dimensional Gaussian language field with consistent semantics is constructed; wherein the dual-path semantic supervision comprises an object perception path and a global context path, the local semantic consistency and the overall semantic relationship are constrained respectively, and the final semantic representation of each Gaussian primitive is obtained through weighted fusion; and finally, candidate grabbing postures are generated, semantic reordering is carried out in combination with a language instruction, and the optimal grabbing posture is screened through geometric-semantic joint scoring. Under the sparse image set input condition, the execution accuracy of a robot grabbing task under a complex instruction can be improved.

The invention provides an object sensing method and system based on proximity vision and tactile deformation, which are applied to the technical field of tactile sensing, and the method comprises the steps: obtaining a binocular image collected by a visual tactile sensor after a target object is in contact with a preset visual tactile sensor; performing visual feature extraction based on the binocular image to obtain a binocular feature point two-dimensional coordinate of the target object; extracting a tactile mark based on the binocular image to obtain a two-dimensional coordinate of a tactile mark point of the target object; through a three-dimensional reconstruction model based on ray tracing, based on the two-dimensional coordinates of the binocular feature points, determining approximate three-dimensional visual perception of the target object in the non-contact area; and determining contact three-dimensional deformation perception of the target object in the contact area through a three-dimensional reconstruction model based on ray tracing and based on the two-dimensional coordinates of the binocular feature points. According to the invention, the perception capability of the visual tactile sensor on the object state and the interaction degree can be improved.

The invention discloses a neural implicit vision SLAM (Simultaneous Localization and Mapping) method based on dynamic perception. The method aims at solving the core technical problems that an existing visual SLAM method is insufficient in robustness, poor in global consistency, large in calculation overhead and the like in challenging environments such as dynamic scenes, weak texture areas and violent illumination changes. According to the method, the feature processing capability of deep learning, efficient dynamic object perception, advanced neural implicit mapping and a global optimization mechanism are integrated, so that more accurate camera pose estimation and higher-quality static environment map construction are realized. In the tracking module, a six-step workflow based on mask guidance is adopted, dynamic objects are filtered from the source, frame-level pre-screening is carried out, and the robustness and the calculation efficiency of the system are remarkably improved. In a dynamic local mapping module, a pixel-level fusion method based on transmission probability and inverse variance weight is innovatively adopted, texture blurring and geometric distortion at the boundary of a plurality of sub-maps are effectively inhibited, and the visual quality of a global map is improved. Besides, by introducing a loop candidate frame reordering strategy based on pose uncertainty weighting in loop detection, visual similarity and geometric credibility can be combined, the false detection rate is effectively reduced, and global consistency and long-term precision of the map are ensured.

The invention provides an object-oriented hierarchical memory network device. The object-oriented hierarchical memory network device comprises a low-level object sensing module, a space-time memory construction module and a long-term space-time memory modeling module, the low-level object sensing module is used for extracting object features of an input video through a segmentation basic model; the space-time memory construction module is used for extracting object interaction and scene information by analyzing an inter-frame space and time relationship so as to splice object features and input original video frames to generate a space representation of target perception; and the long-term space-time memory modeling module is used for storing historical information through a memory bank, capturing a global dependency relationship of a video level, and splicing a plurality of adjacent video frames output by the space-time memory construction module on a channel dimension to form local multi-frame space-time representation. And the purpose of enhancing the capturing capability of the model on fine-grained space-time dynamic in the long video is achieved.

The application discloses a threat object detection method based on reinforcement learning, and aims to improve the threat object detection accuracy in the field of automatic driving. The technical scheme is to construct a threat object reasoning and detection system based on reinforcement learning, which is composed of a strategy model, a reward value calculation module, a group advantage calculation module and a strategy model updating module. The strategy model in the system is trained to obtain a trained threat object reasoning and detection system with threat reasoning and threat object perception capabilities. Finally, the trained threat object detection system is used for threat object detection to obtain the boundary box and threat level of the threat object. The application enables the multi-modal large language model to realize fine-grained object-level threat reasoning based on the context information of the driving scene, significantly improves the understanding ability of the automatic driving system to threat semantics and the perception ability of the threat object, and improves the threat object detection accuracy in the field of automatic driving.

The invention discloses a sorting system and method based on machine vision, and the sorting system based on machine vision comprises an image collection module, a data processing module and a sorting execution module. The image acquisition module is used for synchronously acquiring a hyperspectral image and three-dimensional point cloud data of a target area. And the data processing module is in communication connection with the image acquisition module and comprises a multi-modal feature fusion unit and a dynamic sorting decision unit. According to the invention, through multi-modal information fusion, chemical components and geometric space information of an object can be obtained at the same time, and more comprehensive object perception is realized; a fusion network based on an attention mechanism is adopted, different modal features are weighted in a self-adaptive mode, and the recognition precision is improved; physical constraints and task targets are comprehensively considered, an optimal grabbing point and a movement track are generated, and the sorting success rate is increased; the method has a self-learning capability, and can continuously optimize the system performance from practical experience.

Systems and methods for cooperative perception are described. In some examples, the system can comprise a first subsystem comprising a first sensor, a first communication device, and a processor, which can cause the system to: detect, by the first sensor, first point cloud data, apply a data preprocessing process to the first point cloud data to generate first preprocessed sensor data, apply a feature encoding process to the first preprocessed sensor data to generate first feature data; apply an adaptive feature filtering process to the first feature data to select a first subset of features from the first feature data; apply a cooperative feature aggregation process to fuse the first subset of features with other subsets of features, to generate a fused feature map; and apply an object perception model to the fused feature map to generate object perception data.

The invention provides a target object sensing method, a communication device and a program product, relates to the technical field of communication, and can at least solve the problem of low target sensing accuracy in related technologies. The method comprises the following steps: acquiring perception information of a target object perceived by a plurality of perception devices at different positions; and selecting at least two pieces of target perception information from the multiple pieces of perception information, and confirming the behavior or position of the target object according to the at least two pieces of target perception information. The target sensing accuracy can be improved.

The embodiment of the invention provides a target detection method and device based on vision and radar, a vehicle and a medium. The method comprises the steps that first target object data and second target object data of a vehicle driving environment at the current moment are acquired, and the first target object data and the second target object data are fused to generate third target object data; and finally, determining fourth target object data according to the third target object data and historical third target object data generated at the previous moment. Wherein the first target object data is acquired through a camera, and the second target object data is acquired through a radar. The method is used for achieving the technical effect of improving the sensing precision and robustness of the target object.

A communication system, apparatus, method, and one or more non-transitory computer-readable storage devices for cooperative perception in communication perception integration employ the steps of: determining a type of a first communication node for object perception; and notifying the first communication node of the determined type. The step of determining the type of the first communication node comprises the following steps of: determining the type of the first communication node as a sensing transceiving node in a sensing transceiving set so as to receive or send a sensing signal for sensing an object, or determining the type of the first communication node as a sensing activation node in a sensing activation set so as to receive or send a sensing signal for sensing an object; wherein the sensing activation set comprises the sensing transceiving set.

The application discloses a sparse-view three-dimensional Gaussian language field construction method and system for humanoid robot grasping. The method comprises the following steps: acquiring an RGB image set under sparse-view and a natural language instruction of a user; reconstructing a three-dimensional point cloud through stereo matching, initializing a three-dimensional Gaussian primitive field after noise elimination and metric alignment; embedding two-dimensional semantic features in the field, jointly optimizing through a double-path semantic supervision module, and constructing a semantic-consistent three-dimensional Gaussian language field; wherein the double-path semantic supervision comprises an object perception path and a global context path, which respectively constrain local semantic consistency and overall semantic relationship, and the final semantic representation of each Gaussian primitive is obtained through weighted fusion; finally, candidate grasping postures are generated, semantic reordering is performed in combination with the language instruction, and the optimal grasping posture is selected through geometric-semantic joint scoring. The method can improve the execution accuracy of the robot grasping task under complex instructions under the condition of sparse image set input.

An object processing system within a trailer for a tractor trailer is disclosed. The object processing system includes an input area of the trailer at which objects to be processed may be presented, a perception system for providing perception data regarding objects to be processed, and a primary transport system for providing transport of each object in one of at least two primary transport directions within the trailer based on the perception data.

This application provides a vehicle control method and apparatus, storage medium, and electronic device, relating to the fields of smart mines, autonomous driving, and unmanned vehicles. The method includes: fusing the perception results of a first target object acquired by a lidar and the perception results of a second target object acquired by a millimeter-wave radar to obtain a target object perception queue; and fusing the target object perception queue with a first target object tracking queue using a target fusion strategy to obtain a second target object tracking queue. The target fusion strategy includes: for each perception element, if a matching first tracking element exists, determining whether its source includes both lidar and millimeter-wave radar; if both are included, determining the target sensor speed information based on the respective speed information of the lidar and millimeter-wave radar, and then updating the first tracking element to obtain the second tracking element. This application ensures the reliability of target object detection and improves the safety and efficiency of autonomous driving in mines.

The application discloses a tunnel target object sensing method based on 4D millimeter wave radar and camera fusion, comprising the following steps: judging whether a vehicle enters a tunnel or exits the tunnel; when judging that the vehicle enters the tunnel, tracking the target object until the vehicle exits the tunnel; collecting image data of the target object in a 2D space through a camera, converting the pixel depth information of the image data into a 3D space detection result and outputting; collecting point cloud information through a 4D millimeter wave radar, denoising and clustering the point cloud information to obtain a 3D target detection result and output; and integrating the detection result and the 3D target detection result to generate target object information. The application realizes redundant design of a sensing end, improves environmental sensing capability, and in the case that the misrecognition rate is low, can stably identify stationary vehicles or obstacles in the tunnel, improves the robustness and safety of an autonomous vehicle, and ensures passenger safety.

The invention provides a two-dimensional code attached video identification analysis method based on object perception, and the method comprises a mobile terminal which is provided with a two-dimensional code scanning verification module. The method further comprises the following steps: S1, real-time video acquisition and preprocessing; s2, feature extraction; s3, feature fusion and classification; s4, outputting a result and performing anti-cheating judgment; the method overcomes the defects that an existing two-dimensional code inspection technology cannot identify the physical attribute of a carrier, the response speed is low, and cheating is easily caused by stored and pre-stored pictures / videos, and the like, and the mobile terminal is forced to shoot a short video containing a target two-dimensional code in real time (generally 2-3 seconds). And the dynamic visual features (total time consumption is less than or equal to 5 seconds) of the two-dimensional code area and the surrounding background in the video frame sequence are quickly analyzed, and whether the two-dimensional code is attached to the surface of a three-dimensional object or only exists on a two-dimensional plane medium (picture or screen) is judged, so that cheating behaviors implemented by using the pre-stored two-dimensional code picture / video are prevented from the source.

The application discloses a dynamic object perception system and method based on multi-source sensor information fusion, relates to the field of dynamic object perception, and through parallel processing of multi-source heterogeneous data such as RGB-D images, 3D point clouds and millimeter wave radars, and introduction of a cross-modal inconsistency measurement mechanism, differences and conflicts between different sensor features are actively analyzed and quantified to evaluate the perception confidence of each sensing mode under the current environment in real time. Based on this, an adaptive attention mechanism is driven to dynamically generate the fusion weight of each mode, and finally, intelligent weighted fusion of each mode feature vector is realized, and a high-robustness fusion target state is output. In this way, low-quality or conflicting sensor information can be intelligently suppressed, and the contribution of high-quality information can be enhanced, so that more robust and accurate dynamic object perception can be realized in a complex and changeable environment.

The invention discloses a threat object detection method based on reinforcement learning, and aims to improve the detection accuracy of threat objects in the field of automatic driving. According to the technical scheme, the threat object reasoning detection system based on reinforcement learning is constructed and composed of a strategy model, a reward value calculation module, a group advantage calculation module and a strategy model updating module. And training a strategy model in the system to obtain a trained threat object reasoning detection system with threat reasoning and threat object sensing capabilities. And finally, performing threat object detection by adopting the trained threat object detection system to obtain a bounding box and a threat level of the threat object. According to the method, the multi-mode large language model can realize fine-grained object-level threat reasoning based on the context information of the driving scene, the ability of an automatic driving system to understand threat semantics and the ability of the automatic driving system to perceive threat objects are remarkably improved, and the detection accuracy of the threat objects in the field of automatic driving is improved.

In various examples, certain objects typically found in predictable locations within an environment can be detected more reliably by utilizing known information about the locations and / or the objects themselves. For instance, the disclosed systems and methods can determine the locations of target areas within a coordinate system associated with a target region in an environment and use the target areas to detect and track specific objects that might otherwise be difficult to detect. For example, a target region might be a parking space for a machine, and the coordinate system might specify target areas corresponding to wheel stops, curbs, ground barriers, or other objects commonly associated with parking spaces.The systems can scan various points, which represent sensor feedback, to determine whether a target object is located in a target area, and in some cases also to track the target object.

A communication method and a related device, in the method, after a first communication device sends first information for requesting position information of a target object, the first communication device may receive second information, and determine the position information of the target object based on the second information. In this way, different communication devices can determine the position of the target object in a mutual cooperation mode so as to realize perception of the target object. In some implementations, a first communication device does not have an angle measurement capability or the angle measurement capability of the first communication device satisfies a first condition such that the first communication device is able to implement object perception through cooperation of other communication devices in the event that there is a lack of some or all of the angle measurement capabilities. The flexibility and robustness when the communication equipment executes the sensing task can be enhanced.

The application provides an object perception method and device based on binaural hearing signal expression and a hearing aid, and belongs to the technical field of hearing aids. The object perception method based on binaural hearing signal expression integrates radar perception, object analysis, target identification, spatial sound effect simulation and hearing aid broadcasting and the like, realizes intuitive perception of a target object to a surrounding environment in a vision-limited or hearing-limited scene, and enables the target object to understand the dynamics of the surrounding environment in real time without relying on visual information, thereby enhancing the auxiliary ability of a wearable device to a user in a complex scene. Compared with traditional visual display or voice prompt, the feedback mode of the hearing signal is more suitable for the needs of vision-limited users, hearing-limited users or in a specific complex scene, and can effectively improve the use experience of the target object.

The invention provides a method for generating a data message based on a detected target flyer, electronic equipment and a non-instantaneous computer readable storage medium, the method is applied to sensing equipment, and the method comprises the following steps: acquiring parameter data of the target flyer; a unified data model is constructed according to the parameter data, and the data model comprises flying object target dynamic data, flying object target supplementary description data and sensing equipment supplementary description data; and generating a sensing target data message based on the data model. According to the embodiment of the invention, the flying object target dynamic data, the flying object target supplementary description data and the sensing equipment supplementary description data are constructed in the data model, so that the capability of sensing the equipment for the target flying object is improved, meanwhile, the dimensions of identifying and sensing the non-cooperative target are increased, and the recognition efficiency of the non-cooperative target is improved. And the seamless fusion capability of the sensing equipment and the service system is promoted.

Disclosed in embodiments of this disclosure are a method for determining a perception result, a medium, and a device. The method includes: determining a first image captured by a wide-angle camera and a second image captured by a narrow-angle camera, where a field of view (FOV) of the narrow-angle camera is smaller than a FOV of the wide-angle camera; determining, based on the first image, the second image, and perception task models corresponding to distance ranges, first perception results corresponding to distance ranges; and determining an object perception result based on the first perception results corresponding to distance ranges.

The invention provides a cross-domain object existence perception model training method, an object perception method and equipment, which can be applied to the technical field of artificial intelligence. The method comprises the following steps: respectively carrying out multi-dimensional phase error correction on channel state sample information of a plurality of different domains to obtain a plurality of target phase sample data; performing feature extraction on the target phase sample data by using an initial feature encoder of the initial existence perception model to obtain a plurality of channel scattering sample features; based on the respective distribution characteristics of the plurality of channel scattering sample characteristics, determining the distribution difference between every two channel scattering sample characteristics of the plurality of different domains; determining a comparison loss based on the similarity between the channel scattering sample features and the category label of each channel scattering sample feature; and training the initial existence perception model by taking minimization of distribution difference and comparison loss as a joint optimization target to obtain an existence perception model.

The application relates to the technical field of computer vision, and discloses an occluded pedestrian re-identification method based on occlusion perception and feature restoration, which comprises the following steps: obtaining a test data set containing a retrieval pedestrian image, inputting the retrieval pedestrian image into a feature extraction model, and obtaining N local features of the retrieval pedestrian image; inputting the N local features into a visibility perception model to obtain local visibility scores of the N local features; obtaining feature distances between the retrieval pedestrian image and searched pedestrian images in a gallery according to the local visibility scores; obtaining visible local features of K searched pedestrian images with the smallest feature distances, and completing the occluded local features of the retrieval pedestrian image; and re-searching similar pedestrian images according to the local features and the local visibility scores of the completed retrieval pedestrian image. The application realizes accurate occlusion object perception of occluded pedestrian images under different occlusion scenes and more robust pedestrian re-identification effects.

The invention relates to performing object perception using location-based knowledge for autonomous systems and applications. In various examples, certain objects that are common in predictable locations of an environment may be perceived more reliably by utilizing known information associated with locations and / or objects themselves. For example, the disclosed systems and methods may determine a location of a target region in a coordinate system associated with the target region in an environment, and use the target region to detect and track certain objects that may otherwise be difficult to perceive. As an example, the target area may be a parking space for the machine, and the coordinate system may indicate a target zone corresponding to a wheel stop, a curb, a ground lock, or other objects typically associated with the parking space. The system may sample a plurality of points representing the sensor return to determine whether the target object is located in the target region, and in some instances track the target object.

The present disclosure relates to an object perception method and an object perception apparatus. An object perception method may include detecting candidate objects by using at least one sensor; determining objects from a vehicle among the candidate objects; determining preceding objects among the vehicle objects from the vehicle; and determining the first closest preceding object and the second closest preceding object among the preceding objects.