117 results about "Depth perception" patented technology

The invention provides a phytoplankton chromatography sequence identification method and a phytoplankton chromatography sequence model building method, and belongs to the technical field of image enhancement identification. The method comprises the following steps: firstly, acquiring microscopic chromatography sequence data of phytoplankton, performing view field extraction and serialization recombination, and constructing a three-dimensional data set; then, constructing a three-dimensional recognition model containing physical perception and a sequence aggregation mechanism, extracting single-frame semantic features by the model by adopting a parameter-shared twin network, and introducing a physical definition prior module to calculate a space-frequency domain quality score of a slice; secondly, designing a deep perception sequence aggregation module, and adaptively aggregating key features of a high signal-to-noise ratio by taking definition scores as gating signals and combining spatial context information between slices; and finally, training and optimizing the model based on the image-level weak supervision label to obtain an optimal model. According to the method, the problems of information truncation and out-of-focus noise interference caused by extremely shallow depth of field of high-power microscopic imaging are solved, and full-depth-of-field stereoscopic perception can be realized under the condition that frame-by-frame fine labeling is not needed.

The invention relates to the technical field of robot control, in particular to a robot autonomous navigation method and device, a robot and a computer readable storage medium, and the method comprises the following steps: obtaining real-time body state information, a current position, a target position and depth perception information of an external environment of the robot; encoding the depth perception information, and splicing the encoded depth perception information with the real-time body state information to obtain unified features representing the external environment and the state of the robot; inputting the unified feature and the target position into a preset navigation decision model to generate a preliminary movement speed of the robot; the linear distance between the current position and the target position is calculated in real time, the initial movement speed is adjusted based on the linear distance, and a stable expected speed parameter is obtained; and according to the stable expected speed parameter and the real-time body state information, generating an execution instruction for driving the robot to move and controlling execution, and navigating to a target position to realize autonomous navigation of the robot in a map-free scene.

The invention belongs to the technical field of computer vision, and particularly discloses a monocular 3D object detection method for realizing depth enhancement based on a visual basic model, electronic equipment and a readable storage medium, and the method comprises the steps: S1, building a data set: employing a monocular camera to collect a pavement scene, and obtaining an RGB image in the pavement scene; s2, image preprocessing: preprocessing the RGB image for subsequent feature extraction and depth estimation; s3, performing feature extraction by adopting a dual-backbone network: performing visual semantic feature extraction on the preprocessed RGB image by using DINOv2; performing depth feature extraction on the preprocessed RGB image by using a DPT head; s4, generation of depth perception query points: inputting the visual semantic features and the depth features into a DETR network to generate the depth perception query points; and S5, target detection output: using an MLP-based detection head to obtain information of the category, the size, the center point position, the depth, the 3D size and the direction of the object.

The invention discloses a pseudo 3D display method and system based on eye movement tracking and affine transformation, particularly relates to the technical field of display and man-machine interaction, and comprises the steps of 3D model preprocessing, real-time eye movement tracking, view angle parameter calculation, dynamic projection generation, affine transformation correction, display image adjustment and dynamic pseudo 3D display effect realization. The pseudo 3D effect that the visual angle is dynamically adjusted along with the eye position of the user can be achieved through a common 2D screen, special 3D display equipment is not needed, pixel-level perspective correction is achieved, the depth perception of a pseudo 3D image observed by the user is closer to a real three-dimensional space, the visual immersion and interaction naturalness are effectively improved, and the user experience is improved. The core problems that traditional pseudo 3D depth perception is fuzzy and distortion is easily generated due to view angle deviation are solved, the user immersion and interaction naturalness are remarkably improved, and the method can be widely applied to scenes such as virtual social contact, vehicle-mounted HMI, telemedicine and education simulation.

The invention discloses a visual training method, system and device for myopia prevention and control and correction based on a naked-eye 3D display, and a storage medium, relates to the technical field of three-dimensional image generation and display control, and comprises the field of visual training of myopia prevention and control constructed on a naked-eye 3D display interface, a first visual training area and a second sensing and integrating area, in the first visual training area, through 3D interlaced pictures and videos, fine or dynamic stereoscopic vision is stimulated, front and back intersections of sight lines are adjusted, and split vision training is carried out; in the second perception and integration area, through perception, spatial positioning, binocular coordination and deep perception training, the spatial ability and response ability of eyes are stimulated; in combination with a three-dimensional display mechanism, displaying a naked eye three-dimensional image on a display, acquiring eyeball position information by adopting a human eye tracking technology, and dynamically adjusting a three-dimensional image display area; according to the method, the visual content structured organization and three-dimensional generation cooperative control is realized, and the three-dimensional display interaction matching precision is improved.

The application discloses a semantic segmentation method and device based on depth information position coding guidance, and the method comprises the following steps: acquiring an image to be processed; inputting the image to be processed into a pre-trained spatial depth perception auxiliary network to extract multi-scale features, and obtaining corresponding depth feature maps according to the multi-scale features; generating multi-scale feature vectors according to the multi-scale features and the corresponding depth feature maps; processing the multi-scale feature vectors by using a multi-scale attention mechanism and a feedforward neural network to obtain final features; performing fusion processing on the final features, and performing semantic segmentation on the processed fusion features to obtain a semantic segmentation map; in this way, the depth information is embedded in the features, the calculation cost is reduced, and the image segmentation performance is improved.

The application discloses a kind of dynamic ring vision splicing method and system based on image overlap area feature perception, method includes based on the depth information and the feature information dynamic planning splicing path;According to the splicing path of planning, image splicing is executed, including the multi-band image fusion based on depth perception.The present application extracts local and global features of the overlapping area and calculates pixel-level depth information, constructs a geometric transformation model optimized by feature-depth combination, so that panoramic stitching can accurately identify the outline of close-range objects, and dynamically plan the optimal stitching line to bypass prominent objects;By searching the distance splicing template based on the real distance and automatically unifying the focal length of each camera, the field of view range is kept consistent;Through the multi-scale depth estimation network guided by features and joint reprojection error optimization, high-precision registration can still be maintained in scenes with few feature points or low overlap areas.

A vision test can be performed based on real-time audio instructions in a virtual environment. An electronic device, such as a head-mounted display, can execute a visual assessment application, including generating a user interface corresponding to a three-dimensional virtual environment. The electronic device can display a plurality of visual stimuli in the user interface, and each visual stimulus can be displayed in duplication with respect to a respective target depth. The electronic device can receive one or more user responses, and each user response can indicate whether a user perceives a corresponding visual stimulus in duplication at the respective target depth. Based on the one or more user responses, the electronic device can determine a depth perception profile of the user, and the depth perception profile can include a plurality of depth perception levels corresponding to a plurality of target depths.

This invention discloses a method for monitoring regional passenger flow density based on video image recognition, belonging to the field of video passenger flow density monitoring technology. The method includes structuring a video stream to segment independent moving entities and calculating their trajectory overlap density distribution, identifying high-frequency interactive nodes and low-frequency silent regions. Based on this, a spatial pressure field model is constructed, and its gradient characteristics are used to dynamically correct the estimated entity motion velocity. The corrected velocity and density distribution are integrated to form a dynamic density field, and temporal slicing analysis is performed to extract field strength fluctuation patterns. Historical data of this pattern drives the adaptive updating of the warning threshold. This method achieves deep perception of dynamic passenger flow behavior, improving the accuracy of density monitoring and the system's adaptive warning capability.

The disclosure provides a target depth estimation model training method and a target depth estimation method. The target depth estimation model training method comprises: obtaining a plurality of groups of training samples, each group of training samples comprising a sample image and an image label, the image label comprising a labeled depth of a target of interest in the sample image; obtaining depth prior information corresponding to the sample image, the depth prior information comprising a prior depth of a pixel in the sample image; and training a target depth prediction model using the sample image, the depth prior information and the image label. The target depth prediction model training method provided by the present scheme has more effective information of training samples than the model training method of the prior art, and thus the target depth prediction model obtained by training is more accurate and has more stable depth perception ability.

The present disclosure relates to a medical system (2) for imaging, having a digital microscope (4) having a focus (8) that can be adjusted along an optical axis (6) of the microscope (4), a camera (10) that is separate from the microscope (4) and designed to determine a distance (14) from the camera (10) to an object surface (16) by means of depth perception (12) and to provide it as depth information (18), and a control unit (20) that is designed to automatically set the focus (8) of the microscope (4) on the object surface (16) on the basis of the depth information (18), wherein the camera (10) has a greater field of view (S) and a greater depth of field (T) than the microscope (4). In addition, the present disclosure relates to a method for setting a focus (8) of a microscope (4) of a medical system (2) and to a use of a camera (10) with depth perception (12) of a medical system (2).

The invention discloses an adaptive shadow generation method based on planar projection guidance and depth perception diffusion, and belongs to the technical field of computer vision and image synthesis. The method comprises the following steps: in a first stage, generating a hard shadow mask of a foreground object under a virtual light source through physical projection calculation, and providing geometric position and shape priori; in the second stage, multi-modal conditions such as a hard shadow mask, a background depth image and a foreground and background fusion image and noise latent variables are uniformly coded into a Token sequence, the Token sequence is input into a Diffusion Transform model, detail rendering is carried out through a self-attention mechanism, and a composite image with a realistic shadow is output. According to the method, the problems of shadow geometric distortion, inconsistent illumination, insufficient texture fitting and the like in the prior art are solved through a mixed frame combining physical guidance and neural rendering.

The application belongs to the technical field of artificial intelligence, and specifically relates to an intelligent device online language teaching translation system based on image recognition, which discards the disadvantages of isolated recognition of characters or objects in images in the prior art, fuses visual information with accurate geographical positions, queries a scene-based expression rule set associated therewith, generates scene-based translation data flow and returns to a terminal device, first realizes deep perception and semantic understanding of a real physical environment by a translation system, and introduces individualized correction operation before driving and displaying at the terminal device, so that the final translation result is most applicable and understandable to a current user while taking into account an environmental context, the interaction behavior of the user is recorded synchronously, the decision logic of a scene cultural context is dynamically adjusted, the system realizes an intelligent closed loop of self-evolution, and long-term and continuously evolving individualized learning experience is provided for the user.

The application belongs to the technical field of medical devices, and particularly relates to an automatic needle insertion depth control system based on depth perception. The application discloses an automatic needle insertion depth control system based on depth perception, which comprises a multi-modal environment perception unit, a high-sensitivity mechanical monitoring unit, a dynamic deformation estimation unit, a depth decision control unit and a precision mechanical driving unit. The multi-modal environment perception unit is used for acquiring 3D point clouds and reconstructing surface topology. The high-sensitivity mechanical monitoring unit is used for monitoring a puncture load curve in real time. The dynamic deformation estimation unit is used for calculating a tissue displacement vector field. The depth decision control unit is used for determining a needle tip physical distance according to deformation and mechanical characteristics and generating a correction instruction. The precision mechanical driving unit is used for executing needle insertion closed-loop control. The application effectively solves the problem of accurate tracking under tissue dynamic deformation by fusing depth perception and mechanical feedback, realizes real-time and accurate control of needle insertion depth, significantly reduces the risk of misinsertion, and improves the safety of surgery.

The application relates to the technical field of artificial intelligence, in particular to a deep fake face detection method and system based on deep perception, which comprises the following steps: acquiring a face image, generating a classification label and a depth map label of the face image; extracting a multi-scale feature map from the face image through a shared feature encoder, generating a final depth map through a main depth estimation module for multi-scale feature map regression, generating an enhanced feature map based on a high-level feature map in the multi-scale feature map through a depth collaborative attention module, and generating a logic value representing a true or false prediction according to the enhanced feature through a classification prediction module; all learnable parameters in the model are optimized by minimizing a multi-task loss function value, and a trained model is obtained; a face image to be detected is sequentially passed through the shared feature encoder, the depth collaborative attention module and the classification prediction module to obtain a prediction logic value, and a classification result is generated according to the prediction logic value; and the application can improve the generalization ability of deep fake face detection.

The invention relates to the technical field of medicine circulation and automation, in particular to an intelligent medicine labeling method, system and equipment based on depth perception and multi-modal feature fusion and a medium. The method comprises the steps that a visible light image and three-dimensional depth information of a medicine box are collected; carrying out multi-modal feature fusion on the visible light image and the three-dimensional depth information, carrying out drug identity verification based on a fusion result, extracting drug identification information, and carrying out comparison verification on the drug identification information and to-be-labeled information; the medicine in-place state and labeling feasibility verification is executed based on the three-dimensional depth information; medicine features are extracted based on the visible light image and the three-dimensional depth information, labeling tracks and parameters are generated based on the medicine features, and a mechanical arm is controlled to execute labeling operation; and labeling result quality detection is carried out based on the visible light image, and the mechanical arm is controlled to execute correction operation when defects are detected. The accuracy, reliability and efficiency of labeling operation can be remarkably improved.

The application discloses a scene-driven high-generalization federated reinforcement learning automatic driving system and method, an imitation learning expert learns implicit expert preferences from expert demonstrations, combines dynamic driving suggestions to realize feature extraction of a reinforcement learning student, modeling of a reward function, construction of a loss function, multi-angle guidance of group optimization, and solves the problem of alignment of a reinforcement learning target; the imitation learning expert enhances the connection between multiple learning tasks through cross attention, realizes the bird's eye view reconstruction process under the guidance of scene depth perception and scene semantic perception embedding; through cross-scene optimization, the specific experience sharing between different expert data distributions is realized, and a high-universality imitation learning expert is trained; the reward function decoder outputs motion planning, reward feedback and dynamic driving suggestions as the basis for training and optimization of the reinforcement learning student, and the learning of driving rules is realized; through the time sequence processing module combined with knowledge distillation, the representation ability of the original image input is improved.

A depth sensing system includes a light-emitting device, a sensing module and a controller. The light-emitting device is configured to emit a light beam toward to a scene. The sensing module is configured to receive a reflective beam reflected from the scene to generate a scene image. The controller is electrically connected to the light-emitting device and the sensing module. The controller is configured to calculate an IQ image of the scene according to the scene image. The controller is configured to calculate confidence values of each pixel of the IQ image to generate a confidence image. The controller is configured to calculate a calibrated IQ image according to the confidence values, and then calculate a depth image of the scene. A depth sensing method is also provided.

The invention discloses a field depth information sensing method based on a single light-operated memristor array. The method comprises the following steps: 1, constructing a light-operated memristor array; 2, projecting the optical pulse to the light-operated memristor array, and writing and storing optical information; step 3, erasing the stored optical information; 4, performing one-time conductivity reading on the light-operated memristor array to obtain a two-dimensional current mapping graph; and step 5, sending the obtained two-dimensional current mapping graph as input data into a pre-trained artificial neural network so as to realize rapid identification of the motion direction and depth-of-field information of the target object. According to the invention, plane motion and depth-of-field information are extracted in the same array at the same time, hardware configuration is simplified, through a single-array structure, dependence on multiple arrays or additional sensors is not needed any more, and system conciseness is ensured. By using the light-operated memory characteristic of the array element, the planar motion perception is successfully expanded into the depth-of-field perception, and the function expansion is realized.

The invention provides a VR depth perception training method, device and equipment based on Unity, and belongs to the technical field of virtual reality. The VR depth perception training method comprises the following steps: acquiring a user code input by a subject on a graphical user interface, wherein the user code is used for recording behaviors of the subject and binding a depth perception test result of the subject; generating a depth perception training environment corresponding to the training mode by using a Unity engine, wherein the depth perception training environment comprises a control body, a plurality of capsule bodies and a VR simulation background; controlling the plurality of capsule bodies to randomly move and zoom in the VR simulation background within a preset time; after the capsule body is static, the control body is kept to track the moving state of the handle of the subject on the sagittal axis in real time, so that the control body and the selected capsule body are aligned in the vertical direction; and generating a depth perception test result of the subject according to the depth perception test rule and the movement information of the control body. The problem that an actual motion scene is difficult to simulate in the prior art can be solved.

The application discloses a kind of viewgraph generation methods of cross-modal fusion and multi-frequency coding under ultra-low orbit scene, solve the ultra-low orbit scene of aircraft target, based on the generation of view condition and single picture three-dimensional reconstruction method of priori diffusion, because of insufficient condition information fusion ability, radiation or illumination representation is limited and cross-view geometric constraint is insufficient, it is difficult to obtain the problem that visual coherence and geometric consistency of new view image with texture details under large range of view and scale change simultaneously;The application effectively captures the multi-order angle information of direction distribution to the perception coding of light direction vector, then encodes the position of light starting coordinate, finally encodes the pose of camera pose parameter, can depict the influence of relative attitude on depth perception and perspective distortion in image generation process;The above three encoding results are used in the decoding stage of latent diffusion model, effectively improve the geometric consistency of detail recovery under new view.

The invention discloses a transparent object depth completion method based on input robustness, and belongs to the field of computer vision and deep learning. According to the method, an RIDCNet network is constructed, and the RIDCNet network comprises an RGB-to-Normal branch and a Depth-to-Depth branch. The RGB-to-Normal branch is used for predicting a surface normal from the RGB image; the double-branch features are fused through a feature fusion module FFModule, the module adopts a double-branch attention mechanism, and RGB geometric information and depth measurement information are dynamically integrated through self-enhancement and collaborative enhancement; and finally, the Depth-to-Depth decoder is used for outputting a complemented depth map. The core purpose of the method is to solve the problem that the RGB-D camera fails to sense the depth of transparent objects such as glass, and the method can be applied to tasks requiring robust depth sensing, such as robot grabbing of the transparent objects, augmented reality (AR) scene fusion, automatic driving obstacle recognition and 3D scene reconstruction. According to the method, the dependence on defect depth input is remarkably reduced, and the robustness and precision of the model in a cross-sensor scene and under the condition that input disturbance exists are improved.

The invention discloses a binocular stereo matching depth estimation method and system based on hyperspectral reconstruction feature enhancement. The method comprises the following steps: S1, receiving left and right binocular RGB images; s2, performing high-dimensional spectral feature reconstruction on the left binocular RGB image and the right binocular RGB image by using a multi-stage spectral transformation module, and outputting a multi-channel hyperspectral feature map covering a visible light wave band by learning spectral reflectivity priori of a material; s3, adaptive channel selection and feature recombination are carried out on the multi-channel hyperspectral feature map through a learnable spectrum dimension reduction layer, and a left pseudo-color feature map and a right pseudo-color feature map with enhanced material physical attribute differences are generated; and S4, performing geometric feature extraction and parallax iterative optimization based on the left and right pseudo-color feature maps by using a stereo matching module, and outputting a final compact parallax map. According to the method, optical physical priori is learned through model training, so that low-cost RGB hardware can reproduce high-fidelity spectral features, and the problem of depth perception under complex materials and extreme shadows is effectively solved.

The invention discloses an end-to-end three-dimensional target detection method based on double-vision-field representation and multi-modal fusion, and the method comprises the steps: carrying out the forward projection of an image space based on depth uncertainty perception, and generating an original image stereoscopic pixel feature and a forward projection image aerial view feature; performing aerial view space back projection based on depth perception space cross attention, and generating bidirectional projection aerial view fusion features; carrying out multi-modal fusion of the stereoscopic pixel field, carrying out intra-modal and cross-modal fusion on the original image stereoscopic pixel features and the original point cloud stereoscopic pixel features by utilizing a Mamba model, and carrying out high compression to generate stereoscopic pixel field enhanced image aerial view features and stereoscopic pixel field enhanced point cloud aerial view features; and bird-eye view view field multi-modal fusion: performing intra-modal and cross-modal fusion on the bidirectional projection image bird-eye view features, the original point cloud bird-eye view features, the stereoscopic pixel domain enhanced image bird-eye view features and the stereoscopic pixel domain enhanced point cloud bird-eye view features by using a Mama model to generate double-view field fused bird-eye view features.

The application discloses a multi-modal depth perception and grasping system based on transparent objects, and relates to the field of robot operation, comprising a multi-spectral perception module, a depth correction module, a grasping posture generation module and a control module. The application comprehensively acquires visual information and thermal radiation information of the transparent object by combining two perception methods of RGB-D image and thermal imaging (TIR) image, and analyzes systematic errors of the depth map to detect error sources of the RGB-D camera and the TIR camera. The system adopts an encoder-decoder model as a depth correction core, the model extracts complementary features of the RGB-D image and the TIR image through a modal exclusive encoder, and utilizes a feature fusion module to complete feature alignment and integration, thereby effectively improving the depth estimation accuracy on the transparent surface. Meanwhile, a Bayesian optimization method is adopted to optimize hyperparameters of the depth correction model, through hyperparameter optimization and model fitting processing, the system effectively avoids overfitting and underfitting problems, and further improves the robustness of the depth correction model and the grasping accuracy of the transparent object.

As deep neural networks are increasingly used for generating dense depth maps, depth perception has become an increasingly popular topic in the image community. However, the application of depth perception estimation can still be limited due to the need for large amounts of dense ground truth depth for training. It is contemplated that a self-supervised control strategy can be developed for estimating depth maps using color images and data provided by a sensor system (e.g., sparse lidar data). Such a self-supervised control strategy can utilize superpixels (i.e., groups of pixels that share common characteristics (e.g., pixel intensity)) as locally planar regions to regularize surface normal derivatives according to estimated depth and photometric loss. The control strategy can be operable to produce dense depth maps that do not require dense ground truth supervision.

The method comprises the following steps: 1) collecting continuous frame images; 2) building an optical flow estimation network, the input of two continuous frame images is respectively extracted by a depth perception feature encoder to obtain a fusion feature map, then zero optical flow is taken as an initial optical flow estimation, and multiple iteration optimizations are carried out: in each iteration, a self-adaptive feature alignment module is entered for feature alignment, a global-local collaborative refinement module is entered for global-local feature collaborative refinement, and an optical flow prediction module is entered for optical flow updating, the optical flow of one-half resolution of the input image is up-sampled to the original image resolution, and a final optical flow map is output; 3) training the optical flow estimation network; 4) inputting the continuous frame image pairs in a test set into the optical flow estimation network for optical flow estimation. The application alleviates the feature matching difficulty in strong occlusion and large range weak texture area, and improves the precision and robustness of optical flow estimation.

The present application relates to the technical field of measurement, and relates to a kind of fast measurement of desktop horizontal reference, and it is used to carry out horizontal reference and flatness measurement to the plane to be measured of hand-held scanning, processing unit is used to obtain the depth perception unit of the spatial point depth information data stream of the local area of the plane to be measured, inertial measurement unit is used to obtain the data stream of the motion state of measurement device and the attitude information reflecting measurement device relative to gravity direction, processing unit is connected with depth perception unit and inertial measurement unit, the pose of measurement device is estimated based on depth information data stream and inertial measurement unit and constructs the three-dimensional point cloud map of the plane to be measured, and global horizontal calibration is executed based on the three-dimensional point cloud map constructed, geometric analysis is carried out to the three-dimensional point cloud map after calibration, to determine the overall levelness and local flatness of the plane to be measured, the present application realizes the rapid three-dimensional modeling of the plane to be measured and accurate level and flatness analysis.

The invention provides a heterogeneous robot collaborative operation system, method and device and a medium, and relates to the technical field of robots, and the system comprises the steps that a task planning unit splits an operation task into a plurality of operation sub-tasks; when the task execution unit determines that an operation blind area exists according to the environment perception data, a collaborative operation request is sent to the dynamic viewpoint planning unit; the dynamic viewpoint planning unit obtains a three-dimensional semantic map from the fused map building unit when receiving a collaborative operation request; a cooperative control instruction is generated based on the operation blind area and the three-dimensional semantic map and sent to the slave operation robot through the first cooperative communication module; the slave operation robot moves to a target position according to the cooperative control instruction and is adjusted to a target posture, and an area image of an operation blind area is obtained and sent to a fusion map construction unit; and the fusion map construction unit fuses the regional image and the three-dimensional semantic map to obtain a panoramic fusion map. According to the invention, depth perception fusion and physical collaborative operation of the master-slave operation robot are realized.