4371results about "Image data processing" patented technology

The invention relates to the technical field of transformer substation intelligent patrol, and provides a transformer substation three-dimensional fusion patrol method and system based on digital twinborn and autonomous identification. According to the method, a fused three-dimensional model is constructed through multi-source data acquisition and a three-dimensional Gaussian splash algorithm, and in combination with deep learning-based point cloud semantic segmentation and clustering, an equipment-patrol means coverage relationship is generated. Creating a virtual inspection proxy object based on a three-dimensional virtual environment, and controlling terminals such as an unmanned aerial vehicle to collect real-time video image data; the system carries out automatic identification on pictures, automatically completes equipment level alignment and standard point location identification, generates fine control of camera zooming, horizontal rotation, pitching and the like, and realizes standardized view finding and acquisition. By combining an enhanced recognition algorithm, traditional image processing and a deep learning model are fused, model self-evolution is realized through incremental learning, flexible expansion and collaboration of various patrol terminals are supported through a unified interface, and refined, real-time and intelligent patrol operation and maintenance requirements of an intelligent substation are met.

Systems and methods are described for enabling asynchronous experience sharing between users visiting the same environment at two different times. First image data is received, captured during a first time period, wherein the first image data comprises data characterizing the environment in three dimensions. Stored second image data is accessed, the stored second image data captured during a second time period earlier than the first time period, the stored second image data characterizing the environment in three dimensions. A display image is caused to be rendered at a user device during the first time period based on the first image data, the display image comprising an object from the second image data.

The invention provides a navigation instruction generation method, device and system based on multi-modal environment understanding, and the method comprises the steps: obtaining multi-modal perception information comprising the three-dimensional image data and three-dimensional point cloud data of an environment where a current unmanned aerial vehicle is located, and the task text information of the unmanned aerial vehicle; and performing semantic fusion extraction on the multi-modal perception information and the task text information based on a visual language fusion model to obtain a multi-modal embedded representation, constructing a large language model Prompt based on a plurality of prior navigation templates and the multi-modal embedded representation, and inputting the large language model Prompt into the large language model. According to the method, the navigation instruction text output by the large language model is obtained, then the navigation instruction text is analyzed into the control instruction sequence which can be recognized and executed by the unmanned aerial vehicle, the control instruction sequence is issued to the unmanned aerial vehicle, and the accuracy, continuity and stability of navigation can be maintained in a complex, dynamic and GNSS limited environment.

The invention discloses a large-span truss hoisting and splitting system and method based on digital twinning, and belongs to the technical field of building construction, and the system comprises a physical entity unit, a data acquisition and transmission unit, a digital twinning model unit, a function service and data analysis unit and an application and interaction unit. According to the invention, through real-time sensing of a physical entity state, multi-source data fusion transmission, high-precision digital twinning modeling, intelligent simulation analysis and decision optimization, and fusion of multi-modal man-machine interaction, a virtual control system which is in whole-course synchronous mapping and bidirectional interaction with a physical entity is constructed; according to the method, real-time visual monitoring, trend prediction and risk early warning of the hoisting and splitting process can be achieved, scheme preview and dynamic optimization are supported, virtual-real fusion operation guidance is achieved through the AR technology, and the safety, precision and efficiency of hoisting and splitting operation of the large-span truss are improved.

The invention relates to the technical field of chemical enterprise production management, in particular to a chemical enterprise safety production informatization management system which comprises a sensing network layer architecture, a digital twin management platform, a risk management and control and decision center, a process closed-loop management center and an emergency cooperative processing platform. According to the chemical enterprise safety production informatization management system and method, through construction of a chemical knowledge graph, on the basis of a dynamic risk matrix and an AI prediction model library, equipment remaining service life prediction, process tiny anomaly capture and accident consequence simulation processing are realized; meanwhile, a root cause analysis module is used for associating low-level alarm and traversing a knowledge graph to automatically infer a fault root, physical equipment and a virtual model are bound through an entity-model mapping engine, the model state is dynamically updated in combination with a real-time rendering technology, and technological parameter optimization, accident deduction and scheme verification can be completed in an analogue simulation engine; and the management intuition and the decision-making scientificity are effectively improved.

The invention discloses a cultural relic three-dimensional dynamic scene display system based on a virtual-real fusion technology, and relates to the technical field of mixed reality, and the system comprises a data collection module which obtains original signals of a to-be-displayed cultural relic and a surrounding environment, and generates an environment feature vector and an environment fusion index; the twinborn simulation module is used for generating health state feature vectors of the cultural relics in combination with finite element grids and carrying out prediction simulation on the deterioration state of cultural relic materials; the model creating module is used for creating a three-tense model resource package and generating time axis metadata by adopting a vertex interpolation and mapping mixing technology; the story layout module is used for constructing an environment event mapping rule base, obtaining a ranked trigger event list through priority ranking, and generating a scene script sequence; according to the method, the health state and the degradation process of the cultural relics under different environmental conditions can be dynamically reflected, and three-dimensional scene display changing along with time evolution is provided.

The invention discloses an operation AR intelligent auxiliary management system based on voice AI interaction driving, which is applied to the field of intelligent warehousing and comprises an environment perception type voice interaction module, an AR multi-mode interaction compensation module, a dynamic noise robustness data processing module, an operation management module and a database module. The environment perception type voice interaction module is used for collecting voice signals of operating personnel and inhibiting environment noise; the dynamic noise robustness data processing module is used for carrying out noise reduction processing, semantic understanding and closed-loop optimization on the voice signal and generating feedback information and a control instruction; the AR multi-modal interaction compensation module is used for receiving feedback information of the dynamic noise robustness data processing module and providing visual and tactile interaction compensation, and the job management module is used for scheduling management equipment according to a control instruction; according to the method, the operation efficiency can be effectively improved, the accuracy is high, the real-time performance is high, and interaction experience optimization can be realized.

The disclosed system for interacting with objects in an extended reality (XR) environment generated by an XR system may include (1) neuromuscular sensors configured to sense neuromuscular signals from a wrist of a user and (2) at least one computer processor programmed to (a) determine, based at least in part on the sensed neuromuscular signals, information relating to an interaction of the user with an object in the XR environment and (b) instruct the XR system to, based on the determined information relating to the interaction of the user with the object, augment the interaction of the user with the object in the XR environment. Other embodiments of this aspect include corresponding apparatus, and computer programs recorded on one or more computer storage devices, each configured to perform the actions of the methods.

The invention discloses a buccal needle positioning navigation method and system based on multi-mode perception driving, and the method comprises the steps: collecting the image information and depth information of the face of a patient; generating an individualized face three-dimensional model according to the image information and the depth information; determining a standard meridian map according to a meridian database, and registering the standard meridian map to the individualized face three-dimensional model through non-rigid deformation to obtain acupuncture point positioning; according to the acupuncture point positioning and a pre-stored organization hierarchy model, needle insertion path guiding information is calculated and generated, and in an interactive interface, the acupuncture point positioning and the needle insertion path guiding information are displayed in an overlapped mode so as to execute the needle insertion guiding process; and in the needle insertion guiding process, collision detection is carried out according to the needle insertion path guiding information and the organization level model, and safety early warning is carried out when a collision risk is detected. The precision and safety of the acupuncture treatment process can be effectively improved.

The invention discloses a scheduling management method and system for an unmanned finished product warehouse area, and the method comprises the following steps: analyzing material attributes and order information when materials are warehoused, selecting a raw material area FIFO order scheduling strategy or a finished product area customer order clustering strategy according to the type of the warehouse area, candidate stack positions are generated in combination with a warehouse-area-row-column-layer-stack position five-level coding system, the optimal stack position is determined after stack safety verification, and a crown block operation instruction is generated; when the crown block works, acquiring starting and target coordinates, loading the digital twin model of the reservoir area to generate an initial path, and re-planning a high-risk path by adopting an improved path planning algorithm introducing a load coefficient and a congestion index through conflict risk assessment; and during multi-vehicle collaborative operation, task collaboration and path conflict avoidance are realized based on a distributed multi-agent architecture and a preset communication protocol.

The invention, which relates to the technical field of plant safety monitoring, discloses an AI-vision-based plant safety monitoring and emergency linkage system comprising a sensing acquisition module, a feature completion module, a behavior analysis module, a risk assessment module, a deduction planning module, a linkage execution module and a model updating module. The sensing acquisition module is used for arranging a multi-mode sensing unit in a factory area, acquiring a visible light image, infrared thermal imaging data and millimeter wave radar point cloud, performing timestamp alignment and space coordinate mapping on the three types of data, and outputting an original sensing data set; the feature completion module is used for judging environment quality according to brightness and contrast indexes of a visible light image, extracting a human body point cloud cluster based on millimeter wave radar point cloud when the indexes are lower than a preset threshold value, generating three-dimensional attitude key points through clustering and a skeleton regression algorithm, performing fusion verification in combination with a human body thermal contour in an infrared image, and obtaining a fusion verification result. And outputting the human body posture feature map.

The invention provides a visual language navigation and model training method, device, equipment and medium, and the method comprises the steps: obtaining a training sample, processing a to-be-executed instruction and environment information in the training sample through a core model, obtaining a navigation feature corresponding to the to-be-executed instruction, and inputting the navigation feature into an action prediction head. Obtaining predicted action information corresponding to the navigation features, and inputting the navigation features into a thinking chain generation head to obtain predicted thinking chain information corresponding to the navigation features; and then, based on the predicted action information, the reference action information in the training sample, the predicted thinking chain information and the reference thinking chain information in the training sample, training the core model, the action prediction head and the thinking chain generation head so as to take the trained core model and the trained action prediction head as a visual language navigation model. According to the method, the reasoning efficiency and the reasoning accuracy can be improved.

The invention provides a virtual reality three-dimensional space data correction method, which comprises the following steps of: firstly, acquiring an image, depth and inertia measurement data of a virtual reality terminal, and a pose, a speed and a force feedback signal of an interaction body, and carrying out time sequence alignment under a unified time reference; an interaction event is identified through collision detection, and parameters such as a contact point, a normal direction, a relative speed, a penetration depth and a contact duration are extracted. Then, physical consistency constraints including no penetration, contact stability, friction consistency, momentum conservation and energy non-increase are constructed, and geometric consistency, depth consistency and physical consistency residual functions are established in the local contact area; and solving surface normal, depth, scale, pose, calibration parameters and other corrections through a robust nonlinear optimization method so as to update the point cloud, the grid and the calibration model. According to the method, the consistency and the stability of geometric modeling and physical interaction in the virtual reality scene can be remarkably improved, and the immersion and the interaction precision are improved.

The invention relates to the technical field of night monitoring, and discloses a night dynamic target tracking and image restoration method, which comprises the following steps of: establishing a heat-motion correlation model of a target through a space alignment algorithm in combination with heat source contour and intensity information acquired by a thermal imaging sensor and motion parameters detected by a millimeter wave radar; outputting an initial position and a motion state; inputting a target initial state and radar point cloud data based on an LSTM network, predicting a future N-frame trajectory, and dynamically correcting a prediction result by using real-time radar data; inputting the position information of the occlusion area and the trajectory prediction value into a GAN network to generate a complemented image, fusing the complemented image with the original heat source image, and recovering the complete contour of the target; and superposing and rendering the complemented image and the trajectory prediction into an AR picture, and when the target distance triggers a preset threshold value, realizing multi-channel safety early warning through vibration feedback, AR highlight warning and voice prompt. According to the invention, high-precision tracking and safety early warning of the night dynamic target can be improved.

One embodiment of an image processing device according to the present invention comprises a setting unit and a display control unit. The setting unit sets a movement route for when a viewpoint position inside a virtual space is automatically moved. The display control unit displays, on a display, an image which is inside the field of view and in which the line-of-sight direction from the viewpoint position is oriented rearwards along the movement route while the viewpoint position is automatically moved forward along the movement route.

The invention provides a visual operation and maintenance method and a visual operation and maintenance system for hidden pipelines of a building by using an AR (Augmented Reality) technology. The method comprises the following steps: receiving input engineering construction plan information of a target area, and evaluating based on the engineering construction plan information to obtain a construction type and type definition of a sub-area aligned with a current AR visual angle; determining a display depth based on the construction type and the type definition of the sub-region, and determining a hidden pipeline group to be displayed according to the display depth in a model region corresponding to the current AR visual angle in the building pipeline BIM model; and outputting and displaying the hidden pipeline group in the AR equipment. The method can effectively solve the problem that the existing AR technology is difficult to dynamically regulate and control the pipeline display depth, enables the pipeline display to more accurately meet the construction requirements, improves the accuracy and efficiency of construction planning, and reduces the risk of mistakenly touching the pipeline.

The disclosed system for tracking user movements while a user is interacting with a virtual object may include (1) obtaining tracking information by tracking, via a sensor, a position of an arm of the user and (2) in conjunction with tracking the position of the arm, obtaining wrist information by detecting, via a neuromuscular-signal sensor, (i) a movement of a wrist of the arm and (ii) assigning one or more motion characteristics to the virtual object in accordance with the tracking information and the wrist information.

The invention relates to the technical field of virtual reality and indoor and outdoor design, in particular to a VR indoor and outdoor design virtual material selection system based on eye movement tracking. Comprising a VR display and eye movement tracking module used for presenting a virtual design space to a user and capturing eye movement data of the user in real time; the fixation point recognition and target object judgment module is used for calculating real-time fixation point coordinates of the user based on the eye movement data; the dynamic material menu management module is used for outputting the target object information according to the judgment module; the sight line interaction and material rendering module is used for receiving a selection instruction of a user on a specific material in the dynamic material menu through a sight line; and the central processing module is used for coordinating and controlling data streams and execution logics of the modules. By integrating the eye movement tracking technology and the dynamic material menu management, the intelligent material selection interaction driven by the fixation point is realized.

The invention discloses a target-level three-dimensional point cloud cross-modal semantic retrieval method and device and electronic equipment. The target-level three-dimensional point cloud cross-modal semantic retrieval method comprises the steps that independent three-dimensional target point clouds are segmented from three-dimensional scene point clouds, and unique identifiers are given to the independent three-dimensional target point clouds; projecting each target point cloud to three orthogonal two-dimensional observation planes to generate a multi-view composite image; a pre-trained vision-language basic model is adopted to code and fuse the synthesized image, and a unified multi-modal feature vector is generated; a vector database in which the feature vectors are associated with their identifiers is constructed. And encoding a natural language query text into a text query vector by using the model, calculating the semantic similarity between the text query vector and the feature vector in the database, and positioning and returning the corresponding three-dimensional target point cloud. According to the method, accurate and efficient cross-modal retrieval from a natural language to a three-dimensional point cloud target is realized, the problem that a traditional method is difficult to support semantic fine-grained retrieval of the three-dimensional target point cloud is solved, and a key technical support is provided for training data management in the fields of intelligence and the like.

The invention relates to the cross technical field of biomedical engineering and psychotherapy, in particular to an automatic control system for classified exposure therapy of phobia based on naked eye 3D. The system comprises an electroencephalogram signal feature extraction module, a heart rate variability feature extraction module, a skin electric response feature extraction module, an eye movement feature extraction module, a physiological load state feature extraction module, an exposure therapy process feature extraction module, a first feature fusion module, a second feature fusion module and a stimulation parameter adaptive control module. The method combines multi-mode biological signal real-time analysis and naked eye 3D stimulation parameter self-adaptive adjustment, and is suitable for clinical psychological treatment mechanisms and psychological health intervention scenes.

The invention relates to the technical field of intelligent urethral surgical instruments, and particularly discloses a urethral dilation path planning system integrating visual SLAM and force perception, which synchronously acquires image data of the internal environment of the urethra and contact force data between the instrument and tissues through an image sensor and a force sensing unit integrated at the front end of the dilation instrument; constructing a three-dimensional environment model in the urethra by utilizing a visual SLAM technology, and positioning the position of an instrument in real time; the force data and the three-dimensional model are fused to generate a dynamic force distribution diagram, and a safe expansion path is planned based on the distribution diagram and the current position of the instrument; dynamically adjusting the advancing direction and the expansion force of the instrument through a closed-loop control system to enable the instrument to move along a planned path; meanwhile, path deviation and force abnormity are detected in real time through a monitoring mechanism, and model updating and path re-planning are triggered.

The invention relates to an AR projection interaction treadmill virtual scene generation method and device, equipment and a medium. The method comprises the following steps: acquiring motion inertia data and running belt motion parameters of a user on a running machine; respectively carrying out motion state extraction and gait phase extraction according to the motion inertia data and the running belt motion parameters to obtain a current motion state vector and gait phase information of the user; inputting the motion state vector and the gait phase information into a pre-trained motion state prediction model to obtain a preset predicted motion state of a plurality of frames in the future; based on the predicted motion state, performing layered scene generation through a scene generator to obtain a corresponding virtual scene; and performing projection mapping transformation processing on the virtual scene, and obtaining real-time AR projection data in combination with low-delay compensation. By adopting the method, a highly immersive and interactive virtual training environment can be provided in an AR projection mode.

The invention relates to the technical field of medical data processing, and discloses a non-invasive hemodynamic assessment system for coronary artery stenosis degree, comprising an image acquisition module used for acquiring a coronary artery computed tomography angiography image of a patient; the physiological signal acquisition module is used for synchronously acquiring physiological signals of a patient, and the physiological signals comprise electrocardiosignals and non-invasive arterial blood pressure signals; and the data processing module is used for preprocessing and segmenting the coronary artery computed tomography angiography image. Electrocardiogram and non-invasive arterial blood pressure signals are collected through the sensor, risks and complications caused by invasive examination of traditional coronary artery radiography are avoided, the problem that patients unsuitable for invasive examination such as contrast agent allergy and renal insufficiency are difficult to evaluate is solved, the pain of the patients is relieved, and the acceptability of the patients is improved.

A system and method for contextual three-dimensional messaging in augmented reality (AR) environments is disclosed. The system receives chat messages with specified real-world destinations and stores them associated with those locations. When a user wearing an AR device enters a destination location, the system detects their presence using techniques like GPS, Wi-Fi positioning, or computer vision. It then generates a 3D visual representation of the message and determines an appropriate spatial position within the physical environment based on environmental analysis and object detection. The 3D message is displayed at the determined position in the AR view. The system can analyze message content to identify topics and match them to detected real-world objects for contextual placement. Users can interact with displayed messages through gestures or voice commands to reply, forward, delete, or reposition messages. This enables immersive, location-aware messaging experiences that seamlessly blend digital content with the physical world.

The invention discloses a neurosurgical operation virtual simulation training system based on multi-modal data fusion, and relates to the technical field of operation simulation. The data acquisition and preprocessing module acquires data such as images and force feedback, and standardized storage is performed after filtering and registration; the multi-modal data fusion module performs weighted fusion on the data by using an attention algorithm to generate a three-dimensional model; the virtual operation scene construction module generates a high-fidelity environment through ray tracing and a physical engine; the surgical operation interaction simulation module realizes accurate interaction through positioning tracking and force feedback; the training evaluation and feedback module quantifies operation performance and provides guidance; and the system management and iteration module guarantees operation and supports data updating and optimization. According to the method, a high-reality scene is constructed through multi-modal fusion, interaction and multi-dimensional evaluation help skill improvement, dynamic difficulty and personalized paths are adaptive to different users, collaborative training and privacy data collaboration are supported, operation training standardization is promoted, and the risk of errors in an operation is reduced.

The invention relates to an AR (Augmented Reality) space true north calibration method, device and equipment and a medium. The method comprises the following steps: acquiring fixed solution positioning data in a moving process by utilizing RTK (Real-Time Kinematic) equipment; based on the fixed solution positioning data, selecting a first positioning point and a second positioning point, calculating a coordinate azimuth angle between the two positioning points by using Gaussian positive and negative calculation, and calculating a meridian convergence angle at the first positioning point; calculating an included angle between a connecting line of the first positioning point and the second positioning point and the true north direction based on the coordinate azimuth angle and the meridian convergence angle to obtain a geographic true north direction angle; and continuously monitoring data of a gyroscope of the RTK equipment and a gyroscope on the AR running equipment, and dynamically adjusting the true north direction angle in the AR space by adopting a double-loop control structure based on the geographic true north direction angle, so as to ensure that the AR space is consistent with the true geographic direction. Compared with the prior art, the method has the advantages that the AR space direction precision is improved from + / -5 degrees to + / -0.1 degrees, and the surveying and mapping standard is achieved.

The invention relates to the technical field of endoscopes, and provides a spectral image processing method for a three-dimensional endoscope. The method comprises the following steps: synchronously acquiring single-band spectral images returned by multiple sensors of the endoscope; after the single-band spectral image is corrected, a two-dimensional spectral image is generated by combining multi-level fusion of the spectral weight; constructing a three-dimensional point cloud coordinate set through parallax calculation; mapping the two-dimensional spectrum fusion image to a three-dimensional point cloud based on a back projection sub-pixel mapping algorithm to obtain a three-dimensional spectrum point cloud; and performing Poisson fusion driven inter-block splicing fusion on the three-dimensional spectrum point cloud, and outputting an interactive three-dimensional navigation model. The technical problem that image details are lost and three-dimensional reconstruction is inaccurate due to the fact that spectral image fusion precision is insufficient in an existing three-dimensional endoscope image processing method is solved, registration and reconstruction of high-precision three-dimensional spectral point clouds are achieved through multi-modal image collaborative correction and optimization fusion, and the image fusion precision is improved. And the accuracy and the real-time performance of navigation in the endoscope are improved.

In some embodiments, a method comprises obtaining a video stream of a portion of a geographic area, the video stream comprising a plurality of video frames, each of the plurality of video frames captured at a respective first time. Contextual metadata is obtained, the contextual metadata associated with one or more objects located in the portion of the geographic area at a second time, the second time being before each of the respective first times. The contextual metadata is inserted into one or more of the plurality of video frames, thereby causing the contextual metadata associated with the one or more objects to be overlaid on one or more corresponding portions of the one or more of the plurality of video frames.

Examples in the present disclosure relate to hand chirality estimation. Tracking data captured by one or more sensors associated with an extended reality (XR) device is processed to determine positions of a plurality of joints of a hand of a person. A reference vector is generated based on a first subset of the positions. The first subset of the positions includes positions of at least two metacarpophalangeal joints. A plurality of bending angles is determined based on at least a second subset of the positions. Each bending angle represents an angle between a respective pair of articulating bones that is measured in relation to the reference vector. An estimated chirality of the hand is identified based on the plurality of bending angles. Operation of the XR device is controlled using the estimated chirality of the hand.

According to the new energy Internet of Things multi-level analysis sharing platform based on digital twinning, the intelligent perception and edge access module collects data through a multi-mode sensor, and reliable data is provided for subsequent analysis through edge calculation preprocessing and hierarchical transmission. And the digital twinning construction and dynamic updating module constructs equipment-level, station-level and regional-level digital twinning models based on a Bayesian optimization algorithm to realize dynamic calibration of the models. And the multi-level analysis decision module realizes equipment fault diagnosis, generation power prediction, energy storage scheduling and power grid optimization through a four-level architecture. The digital twin mirror image regulation and control module establishes a bidirectional control channel and supports remote configuration of equipment parameters and automatic triggering of fault plans. The cross-domain data security sharing module adopts a three-layer security protection system and a federal learning mechanism to guarantee data security and sharing. According to the platform, full-process intelligent management of the new energy system is realized, and the operation efficiency and safety of the system are improved.