78 results about "Visual motion" patented technology

The invention discloses a joint denoising method for robot visual motion prediction, and the method comprises the steps: constructing a unified generative model through fusing an image and a depth map collected by a depth camera, motion data collected by CAN line communication of a Piper mechanical arm, and a tactile image collected by a Gelsight Mini tactile sensor; the method comprises two steps of data acquisition and input coding, and joint denoising and generation: firstly, multi-modal data are coded into low-dimensional potential representation, and then future images, depth maps, tactile data and robot actions are cooperatively predicted through a joint denoising framework based on Transform. A mask self-attention mechanism is innovatively introduced, information interaction between modes is dynamically adjusted, action generation is guided through tactile feedback, and the force control precision is improved. The model adopts a de-noising diffusion probability loss function to jointly optimize multi-modal prediction, so that the output consistency is ensured. According to the method, the robustness and the accuracy of flexible operation of the robot are remarkably improved.

The invention discloses a non-aligned spatio-temporal data driven multi-sensor fusion method and system, and the method comprises the steps: building a matching measurement function through collecting radar point cloud density and visual image texture, and constructing a synchronous gradient field through employing a space depth change parameter and a visual motion contour, and determining a data synchronization standard; detecting a time sequence abnormal accumulation area based on a synchronization standard, and identifying an abnormal starting point and propagation time delay through a backtracking track to form an asynchronous sensor coordinate table; iMU angle increment and sensor response delay are collected, and sampling imbalance information is generated through time domain matching of an attitude switching signal and a data collection inflection point; performing association mapping on the asynchronous coordinate table and the sampling imbalance information to generate a feature offset, determining a matching core area, and creating a hierarchical calibration sequence; performing internal and external source difference identification analysis on the calibration sequence to obtain an offset transfer speed to form an offset coupling factor; and a fusion enhancement region is determined based on the offset coupling factor, and high-precision multi-source data fusion is realized through cascade analysis and gain coefficient coding.

The invention provides an AGV global positioning controller based on double-filter arbitration and space-time verification. The AGV global positioning controller comprises a visual motion estimator, a global absolute positioner and an intelligent arbitration center. The visual motion estimator generates relative pose and visual positioning confidence through data of a depth camera, an inertial measurement unit and a wheel type odometer; the global absolute locator outputs an absolute pose and a global positioning confidence coefficient through ultra-wideband base station ranging data; the intelligent arbitration center dynamically selects a fusion strategy according to the confidence degree, performs weighted fusion in a normal state, and adopts speed interpolation or drift compensation to realize positioning source switching in an abnormal state, so that the problem of positioning jump caused by failure of a single sensor in a complex environment is effectively solved; the positioning stability can be improved, smooth switching and abnormity correction are realized, and the AGV global positioning precision is further improved.

The invention relates to the technical field of numerical control machine tool fault diagnosis, in particular to a remote video-based numerical control machine tool remote fault diagnosis method and system. The method comprises the following steps: collecting video data of the numerical control machine tool under multiple view angles through remote video equipment; motion basic parameters are calculated according to the video data; performing motion feature analysis on the motion basic parameters to obtain an axis motion feature vector; performing axis motion stability calculation on the axis motion characteristic vector to obtain an acceleration stability parameter; performing trajectory curvature analysis according to the acceleration stability parameter and the axis motion feature vector to obtain a visual motion smoothness index; working condition feeding response processing is carried out according to the visual motion smoothness index, and a smoothness-feeding response curve is obtained. According to the method, fault diagnosis is improved from passive response to active prediction through a numerical control machine tool fault diagnosis technology, the non-planned downtime is greatly shortened, and the equipment operation reliability and the production efficiency are improved.

The invention provides a display method, electronic equipment and a storage medium, and relates to the technical field of display. The method comprises the steps that in response to a first operation, the electronic equipment displays a first interface; displaying the target animation in a preset area of the first interface; the target animation comprises animation elements, the animation elements move or are static according to the motion state of the electronic equipment, and the target animation is used for mapping the motion state of the electronic equipment in real time. In the implementation mode, the animation elements in the target animation change according to the motion state of the electronic equipment, so that the displayed target animation can map different motion states of the electronic equipment in real time, the motion information sensed by the eyes of a user is consistent with the motion information sensed by a vestibular system, and the user experience is improved. Namely, the motion state of the target animation in the display interface seen by the eyes of the user is consistent with the motion state sensed by the vestibular system. Therefore, the conflict between the user vision and the motion state sensed by the vestibular system can be relieved, and the motion sickness of the user is relieved.

The invention provides a multi-configuration robot-oriented model training method and device and a storage medium, and relates to the technical field of robot control, and the method comprises the steps: constructing an initial task dynamic representation model, training an initial unified visual motion network according to a mechanical arm operation data set for a multi-configuration robot, and obtaining an initial task dynamic representation model; obtaining a target unified visual action network; performing feature extraction on the mechanical arm operation image sequence and the corresponding mechanical arm state sequence by adopting a target unified visual motion network to obtain a hidden space visual motion representation; and the hidden space visual motion representation serves as an auxiliary supervision signal, the initial VLA strategy network is trained according to the mechanical arm operation image sequence, the corresponding mechanical arm state sequence and the corresponding natural language instruction, and a target VLA strategy network is obtained. The robustness of model strategy learning and the model universality are enhanced, and the execution effect of the configuration robot is improved.

The invention relates to the technical field of artificial intelligence, and discloses a spiking neural network-based robot control method, which comprises the following steps of: acquiring dynamic visual information of an external environment, and encoding the dynamic visual information to generate a dynamic visual pulse stream; collecting motion state information of the robot, and encoding the motion state information to generate a motion state pulse code; through a pulse attention gate control mechanism, computing resources of the pulse neural network are dynamically allocated according to task priorities; inputting the dynamic visual pulse streams and the motion state pulse codes into a pulse neural network, processing the dynamic visual pulse streams and the motion state pulse codes by the pulse neural network according to computing resources, and outputting dynamic visual features and motion state features; fusing the dynamic visual features and the motion state features to generate a visual-motion pulse signal; and converting the vision-motion pulse signal into a motion control instruction, and controlling the robot to execute a corresponding action according to the motion control instruction.

An electric vertical take-off and landing (eVTOL) aircraft can enhance energy efficiency, safety, and operational range. A deployable wing structure can provide aerodynamic lift during horizontal flight, reducing reliance on energy-intensive propellers. Integrated flexible solar panels capture solar energy, contributing additional power and optimizing energy management. The wing system also includes an emergency descent mode, doubling as a glide-assist device for controlled landings during critical failures. The system offers modular configurations for various missions, ensuring adaptability and improved flight performance. The eVTOL can be implemented with systems and methods for mitigating motion sickness. The systems integrate tactile feedback systems into wearable devices and environmental components. Sensors detect motion and environmental changes, and a computing device can generate corresponding tactile feedback signals. Tactile actuators embedded in the devices or components provide non-visual motion cues, such as pressure, vibration, and haptic feedback, to resolve sensory mismatches between the vestibular and proprioceptive systems.

The invention provides a robot mobile mechanical arm coordinated motion planning method and system which can be applied to the technical field of robot control and the like. The method comprises the steps that space-time joint trajectory planning is achieved based on a multi-type algorithm, a moving base incomplete constraint path and a moving mechanical arm tail end path are generated through a three-dimensional occupation map obtained through environment perception, an improved preset trajectory optimization algorithm is adopted to constrain speed / acceleration, and smoothness and obstacle avoidance capacity are improved; a vision-motion collaborative optimization scheme is provided, joint speed and holder control quantity are solved at the same time through quadratic programming, and perception information gain maximization and motion stability are considered on the premise that mechanical constraints and an environment distance field are met; meanwhile, a re-planning mechanism of dynamic risk perception is developed, and when the environment changes or enters a visual blind area, real-time adjustment of a local track is achieved by enhancing safety distance constraint and forcing visibility requirements.

The present disclosure provides a humanoid robot control method and system based on action mapping, which relates to the technical field of humanoid robots, and includes: constructing a humanoid robot animation model according to the servo distribution of the humanoid robot to be controlled and performing bone binding; obtaining the human body visual motion capture data of the action to be reproduced, performing joint-level action mapping on the human body visual motion capture data and the humanoid robot animation model to obtain the action data of the mapped humanoid robot model; grading the task priorities of the humanoid robot torso, arms and legs, and combining the smoothed action data to solve the joint torque through inverse dynamics; using the joint torque as a feedforward quantity to perform reproduction control of the actions of the humanoid robot; through the human body visual motion capture data and the humanoid robot animation model, the present invention significantly improves the efficiency of generating anthropomorphic actions of the humanoid robot, solves the technical problems in the coherent action mapping of the multi-degree-of-freedom humanoid robot, and reduces the design difficulty.

A technique of camera exposure control for vision-based navigation of an unmanned aerial vehicle (UAV) includes acquiring an aerial image of a ground area below the UAV with an onboard camera system of the UAV, estimating a visual motion factor based on a speed of the UAV and an altitude of the UAV, and adjusting an exposure control setting of the onboard camera system based on the visual motion factor.

The invention discloses a visual motion control method and device for an automobile automatic charging system, a storage medium and a program product. The invention provides a visual motion control method for an automobile automatic charging system, the automobile automatic charging system comprises two visual sensors, a mechanical arm, a controller and a charging socket, the visual motion control method comprises the following steps: a diffusion model structure selection step: selecting a structure of a noise prediction network of a diffusion model based on a Transform structure; a training data collection and processing step: collecting expert demonstration videos of plugging and unplugging of a charging gun by a mechanical arm of the charging system, and processing the collected data, including video segmentation, key frame labeling and inspection of time synchronism of visual data and operation action data; and a diffusion model training and verification step: performing end-to-end training on the noise prediction network and the visual encoder to optimize the loss function.

The invention discloses a multi-dimensional sensing system, device and method based on an end-side large model, and the method comprises the steps: obtaining multi-modal sensor data in real time, including environment data, audio data and visual motion data; based on the signal variation, the historical standard deviation and the preset environment weight of each sensor, an environment anomaly index is calculated in combination with the time period factor; calculating a user intention confidence coefficient based on a cross-correlation value of the audio data and the visual motion data, a visual motion information entropy and a personnel existence probability; and when the environment anomaly index is greater than a preset safety threshold value or the user intention confidence coefficient is greater than a preset intention threshold value, triggering an end-side large model to analyze multi-modal sensor data and generate a control instruction. The invention relates to the technical field of multi-dimensional perception, and solves the technical problems of overhigh equipment energy consumption and potential safety hazard early warning lag caused by large-model real-time reasoning in a smart home scene.

The invention relates to the technical field of auditory body sense training, and discloses an earphone and video auditory body sense training linkage control system. The system comprises a multi-modal data acquisition module, a data fusion processing module, a dynamic feature extraction module, an adaptive weight distribution module, an anomaly detection engine module and a real-time feedback adjustment module. The multi-modal data acquisition module synchronously acquires earphone real-time audio feature data and video training system visual motion feature data; the data fusion processing module aligns the two types of data according to timestamps to generate a space-time incidence matrix; a dynamic feature extraction module extracts auditory sense-somatosensory sense coupling feature vectors from the dynamic feature extraction module; a self-adaptive weight distribution module adjusts a coupling feature vector weighting coefficient according to the target parameter of the current training stage; the anomaly detection engine identifies an abnormal mode based on the deep residual network; and the real-time feedback adjustment module generates an audio and video parameter adjustment instruction according to the detection result. The system realizes linkage control of auditory sense and body sense training, and is suitable for scenes of rehabilitation training, exercise guidance and the like.

This invention belongs to the field of image processing technology and discloses a power equipment status recognition method based on multi-source data fusion. The method includes: acquiring multi-source data of a target transmission line segment and performing time synchronization processing to obtain aligned multi-source data; acquiring line structure parameters and, combined with aligned electrical vector data, analyzing excitation intensity indicators reflecting the intensity of periodic electromagnetic interaction between adjacent lines; determining whether preset suspected risk conditions are met; if so, generating a trigger command and collecting corresponding multi-source data to form an event data packet; extracting motion features from the event data packet to obtain the line's visual motion features; performing correlation analysis based on the corresponding electromagnetic excitation features to obtain physical verification results; and analyzing the risk level of the current line's dynamic swaying based on the physical verification results and visual motion features. This invention effectively warns and prevents the risk of insufficient conductor phase-to-phase distance caused by dynamic power flow attraction.

The invention relates to a mobile robot motion estimation method based on pose decoupling. The method comprises the following steps: multi-sensor synchronization and decoupling state modeling; performing attitude-position decoupling pre-integration based on an IMU (Inertial Measurement Unit); visual observation modeling and reprojection constraint construction are carried out; carrying out pose optimization solution based on the decoupling factor graph; pose recovery, motion track output and local re-initialization are carried out; compared with the prior art, the method has the advantages that independent modeling and decoupling estimation are carried out on the pose rotation component and the translation component, the propagation effect of rotation errors to translation solution in traditional visual motion estimation is effectively inhibited, and the stability of translation estimation is remarkably improved. Under visual degradation scenes of weak texture, low parallax, illumination variation, motion blur and the like, estimation drift can be reduced, track continuity and consistency can be improved, and robustness of the system to abnormal matching and noise disturbance can be enhanced; besides, the decoupling mechanism provided by the invention has good universality, can be integrated with various types of visual SLAM or visual-inertial fusion frames, and provides more reliable motion estimation capability for stable autonomous navigation of the mobile robot in a complex environment.

The embodiment of the invention discloses a video linkage tracking method and device, electronic equipment and a storage medium, potential targets and motion trends are analyzed and accurately identified based on a panoramic video frame sequence, target position distribution and the motion trends are combined to dynamically divide and adjust a region of interest in real time, a fixed region division mode is replaced, and the tracking efficiency is improved. The fast moving target is always in an attention range, and the defect that tracking response is not timely is overcome; secondly, configuring a multi-dimensional dynamic triggering condition including a motion speed, a stay time length and an overlapping degree for each dynamic attention area, and adapting to target motion characteristic differences to avoid the limitation of intrusion triggering of a single area; meanwhile, by extracting target vision, motion and category exclusive multi-dimensional features and combining a multi-modal tracking algorithm, precise tracking is achieved, the defect that the precision is poor in a single-device view blind area and a multi-target parallel scene is overcome, false triggering and missed triggering are effectively reduced, and the accuracy and reliability of video linkage tracking are remarkably improved.

Provided is a system, method, and computer program product for analyzing spatio-temporal medical images using an artificial neural network. The method includes capturing a series of medical images of a patient, the series of medical images comprising visual movement of at least one entity, tracking time-varying spatial data associated with the at least one entity based on the visual movement, generating spatio-temporal data by correlating the time-varying spatial data with the series of medical images, and analyzing the series of medical images based on an artificial neural network comprising a plurality of layers, one or more layers of the plurality of layers each combining features from at least three different scales, at least one layer of the plurality of layers of the artificial neural network configured to learn spatio-temporal relationships based on the spatio-temporal data.

The invention relates to the technical field of robot control, and discloses a visual motion joint modeling and prediction method, device, equipment and medium, and the method comprises the steps: constructing a unified input data set, carrying out the preprocessing and time alignment of a visual image and a motion state vector, and generating standardized input data; inputting the visual image into a visual backbone network to obtain a visual feature sequence; the visual feature sequence is sent to a time sequence modeling module to generate feature representation fused with a time sequence context; inputting the feature representation fused with the time sequence context into an action prediction branch to generate an action prediction vector; generating a training loss item set based on the motion prediction vector and the demonstration label, and jointly updating the model to obtain an optimization model; constructing a contrast model based on the optimization model and generating an evaluation record; and determining optimization configuration based on the evaluation record and updating the optimization model. According to the method, through unified perception and prediction and dynamic optimization, the precision and efficiency are improved, and the autonomous operation capability in a complex environment is enhanced.

The invention discloses a video call forgery detection method based on a high-frequency sound signal, and the method comprises the steps: enabling a terminal device to actively transmit a high-frequency sound signal which is imperceptible to human ears in a video call process, and embedding a random sequence code in the high-frequency sound signal; the method comprises the following steps: synchronously acquiring an audio signal and a video signal of a video call, verifying a random sequence carried in the audio, extracting a human body motion feature of a target user reflected by high-frequency sound response, and extracting a corresponding visual motion feature from a video picture; and judging whether the video call is a real-time real call or not by analyzing the time sequence consistency between the acoustic motion feature and the visual motion feature. According to the method, a high-frequency sound active detection mechanism of random sequence coding and cross-modal motion consistency check are utilized, and playback and synthesis class forgery in a video call are defended from the physical level. The system can be deployed in terminal equipment such as a computer, a smart phone and the like with a loudspeaker, a microphone and a camera.

The present application relates to a kind of vehicle speed detection methods based on outer pole face image, belong to the field of automatic driving.The method includes: S1: periodically capture the image of the road in front of vehicle, obtain time-space distance image;S2: the curve cluster surface of the time-space distance image obtained is divided into several parts;S3: the curve corresponding to each part obtained;S4: by its analytic differential, the speed curve of each part is obtained;S5: the generation of full interval speed curve, according to the displacement of feature point, the speed of vehicle is calculated.The present application is based on the principle of visual kinematics, using the image obtained by vehicle-mounted camera and distance sensor, the speed of vehicle is calculated by observing the motion of feature point in image.Compared with the existing speed detection method based on physical sensor, the method is lower in cost, but the principle is simple, and the accuracy is also higher.The method can be used alone, can also be combined with the data of other sensors, improve detection accuracy and stability.

The invention discloses a generative AI training scene synthesis method for vestibular rehabilitation. The invention belongs to the technical field of medical artificial intelligence, and the method comprises the steps: obtaining vestibular function defect features of a subject, the features comprising a cold and hot test asymmetry ratio, a vHIT gain value or spontaneous nystagmus intensity; the features are coded into scene control vectors, the scene control vectors comprise the visual motion direction, the angular velocity amplitude, the acceleration change rate and the space complexity, and the angular velocity amplitude is dynamically calculated according to the patient compensation threshold value and the function retention rate and covers 60%-90% of the patient compensation threshold value and the function retention rate; inputting the vector into a pre-trained conditional diffusion model or a conditional generative adversarial network (cGAN) or other generative AI models to generate a customized VR scene; and rendering the scene to a head-mounted display device for rehabilitation training. According to the invention, one person has one scene and accurate stimulation is realized, the problems of fixed scene, uncontrollable stimulation, high development cost and the like of the existing VR rehabilitation scene are solved, and the training safety and effectiveness are improved.

The invention discloses a mobile robot formation finite time control method based on visual servo. According to the method, firstly, a relative kinematics model between a navigation robot and a following robot is established under a world coordinate system, the relative kinematics model is converted into a relative vision kinematics model in an image space in combination with a pinhole camera model, and an image error vector is obtained through normalization; for the error model, a non-linear observer is designed to estimate the unknown speed of the piloting robot, a parameter adaptive updating law is provided to process the uncertain feature height, and a finite time controller based on the linear speed and the angular speed is further designed to realize accurate tracking of the piloting robot. The information required by the method can be acquired from the image, the method does not depend on local communication or GPS between the robots, the communication and sensor loads are small, the tracking control precision is good, the system response speed is high, and the method is suitable for efficient cooperative control in the multi-robot formation task.

This invention discloses a method, system, and robot for generating 3D visual motion strategy training data based on task decomposition. The method includes: constructing a Task-Edit fully synthetic framework; firstly, parsing and reconstructing single-source human demonstration data to recreate the operation process in a simulation environment and generate a baseline template; then, decoupling the task into three major components: scene, skill, and object; generating massive and diverse "scene-skill-object" configuration data through independent editing and flexible combination of the attributes of each component; subsequently, completing Sim2Real data adaptation through depth map repair, point cloud denoising and fusion to narrow the gap between simulation and real-world scenes; finally, using high-quality structured data for 3D visual motion strategy training. This invention overcomes the limitations of traditional object center transformation, significantly improves model generalization ability, effectively enhances adaptability to complex scenes and long-term tasks, reduces data generation and deployment costs, and provides an end-to-end solution for 3D visual motion strategy training for complex robot operations.

The present invention relates to the field of motion control technology and discloses a visual motion control method, comprising: acquiring a first initial atlas using a first camera, acquiring a second initial atlas using a second camera in real time, performing motion denoising on the second initial atlas to obtain a second motion atlas; extracting a first pose feature from the first initial atlas, extracting a second pose feature from the second motion atlas, performing three-dimensional reconstruction based on the first pose feature and the second pose feature to obtain a three-dimensional model of the environment, performing object segmentation and semantic recognition on the three-dimensional model of the environment to obtain a semantic set of environmental objects; performing motion annotation on the three-dimensional model of the environment using the semantic set of environmental objects, extracting target motion points and target three-dimensional coordinates of the target motion points, and controlling a robotic arm to perform assembly motion based on the target three-dimensional coordinates. The present invention also proposes a visual motion control device. The present invention can improve the accuracy of data risk identification.

The application relates to the technical field of blind guiding robot interactive navigation, and particularly discloses a blind guiding robot interactive navigation system combining vision, inertial navigation and voice, wherein the blind guiding robot interactive navigation system synchronously acquires environment vision, motion and sound data through a multi-modal perception data acquisition module, analyzes the data through a context-user joint state feature extraction module, combines a user potential intention deduction module, predicts the user intention, generates an adaptive guiding strategy, adjusts a navigation path according to the strategy through an adaptive guiding path optimization module, ensures safety and efficiency, and finally provides voice and tactile feedback through a situational multi-modal interactive instruction output module, thereby enhancing the interactive experience and navigation autonomy. The application solves the travel problem of the visually impaired in a complex environment, improves the individualization, situational perception ability and safety of blind guiding, and brings more natural and efficient navigation assistance to the visually impaired.

A technique of camera exposure control for vision-based navigation of an unmanned aerial vehicle (UAV) includes acquiring an aerial image of a ground area below the UAV with an onboard camera system of the UAV, estimating a visual motion factor based on a speed of the UAV and an altitude of the UAV, and adjusting an exposure control setting of the onboard camera system based on the visual motion factor.

This application discloses a method, device, storage medium, and vehicle for controlling a motion sickness-preventing seat, relating to the field of vehicle seat adjustment and control technology. The method includes: detecting that an in-vehicle display screen is turned on; extracting visual motion acceleration from the content being played on the display screen and determining the user's perceived motion acceleration; identifying a first user viewing the display screen and determining a first seat occupied by the first user; and controlling and adjusting the first seat based on the visual motion acceleration and the perceived motion acceleration. This method can actively align visual perception of motion with bodily sensation feedback while the user is watching in-vehicle entertainment content using visual motion acceleration. This eliminates sensory conflict, ensuring consistent sensory signals, significantly reducing the incidence of motion sickness. The enhanced immersion in the content through motion feedback improves user satisfaction. The system automatically starts and stops, making it convenient to use, and the hardware cost is controllable.

The application discloses a kind of robot simulation scene and trajectory automatic generation method and system, receive natural language operation instruction, and through semantic analysis, visual asset generation, three-dimensional reconstruction and physical credible layout generation three-dimensional simulation scene that can be used for robot operation training;Utilize video generation model to synthesize the continuous image sequence of target operation task, and extract two-dimensional visual motion trajectory in target object area;By depth estimation, three-dimensional motion fusion, grasping posture generation and inverse kinematics mapping, two-dimensional visual motion trajectory is converted into robot end effector trajectory and corresponding robot joint trajectory, the simulation scene and expert trajectory generated are paired, form training data set for robot imitation learning strategy training, and support the strategy obtained by training is deployed to simulation environment or real robot hardware.The application can reduce the cost of robot training data acquisition and artificial trajectory design, improve the degree of automation of simulation scene and operation trajectory generation.

A gap measuring device for elevator inspection and detection relates to the technical field of elevator detection tools, a threaded rod is screwed and inserted at the front end of a mounting sleeve through a bearing, the rear end of the threaded rod is screwed at the rear end of the mounting sleeve through a bearing, a mounting seat is screwed and sleeved on the threaded rod through a thread, a connecting sleeve is fixedly arranged on the mounting seat, and the connecting sleeve is fixed on the mounting seat. The rear end of the connecting sleeve is arranged at the rear end of the mounting sleeve in a penetrating manner, the connecting bolt is fixedly arranged on the probe, and the connecting bolt is screwed and inserted into the rear end of the connecting sleeve through a thread; by arranging a probe with adjustable standard parameters and an elastic telescopic support structure, the probe can be simply inserted into a gap and touches the bottom when the probe is inserted for detection, and meanwhile, the device is also provided with a visual motion conversion structure, so that the movement of the probe can be converted into a simpler swing display structure, and the detection accuracy is improved. Therefore, the visual depth detection of the gap of the elevator workpiece can be realized through the device.