3669 results about "Multiple view" patented technology

The invention provides a road bridge crack detection method and system, and the method comprises the steps: collecting a bridge surface multi-view image, and constructing a training data set containing crack feature labeling through quality screening and standardized labeling; preprocessing the image by using a multi-scale feature fused deep convolutional neural network and carrying out semantic segmentation, initially identifying a suspected crack region and generating a segmentation mask; and constructing a BeNNS proxy model based on the mask, and establishing a mapping relationship between the detection result and the bridge structure topology, the stress flow field and the service function chain so as to evaluate the result reliability. And inputting an evaluation result into a hybrid evaluation mechanism, performing online real-time detection and offline batch verification to optimize precision, and outputting a verified crack region. Finally, morphological analysis is conducted on the area, geometric parameters and danger levels of cracks are extracted and integrated to a bridge health monitoring system, a crack evolution tracking algorithm and an early warning mechanism are established, and dynamic tracking early warning is achieved. The problem of low detection precision in a complex environment can be solved.

The invention relates to the technical field of image processing, and discloses an industrial part alignment method and system based on visual analysis and a storage medium. The method comprises the steps that a three-view camera collects an industrial part image, and preprocessing is carried out through gradient magnitude local contrast enhancement to obtain an enhanced image; performing hierarchical feature extraction to identify edge contours and key control points to form a multi-dimensional feature set; and establishing a dynamic reference coordinate system based on the feature set to obtain a part space attitude matrix. And the attitude deviation is compensated through Z-axis offset and rotation coupling error analysis. Posture adjustment is decomposed into a plurality of sub-stages, an alignment track is optimized by adopting a variable speed planning strategy, and accurate alignment of the parts is achieved. The problems that multi-view visual information fusion is insufficient, a special recognition algorithm for geometrical characteristics of the industrial parts is lacked, and Z-axis offset and rotation coupling error compensation is inaccurate in the posture adjustment process are solved, and the precision and stability of alignment of the industrial parts are improved.

The present disclosure relates to the field of computer vision and discloses a three-dimensional dynamic scene reconstruction method and apparatus, and a storage medium. The three-dimensional dynamic scene reconstruction method comprises: acquiring synchronized videos of a plurality of viewpoints of a dynamic scene; computing matching points between video images of different viewpoints, and estimating intrinsic and extrinsic parameters of each camera; obtaining a Gaussian splatting point set {p0} on the basis of a sparse point cloud constructed according to the depth of each matching point; for the first image frame of each video, using {p0} to perform static training thereon, to obtain a Gaussian splatting point set {p}; for the remaining image frames, dividing {p} into a static point set {S} and a dynamic point set {D}, performing dynamic training on {D}, and constructing a dynamic Gaussian splatting point set {P} from {p}, {S}, and the final {D}; and, in view of the intrinsic and extrinsic parameters of each camera, rendering {P} using a Gaussian splatting rendering pipeline, to obtain rendered images at different moments from new viewpoints.

The invention relates to the technical field of quality monitoring, and discloses a diamond high-strength micro-powder quality detection method and system based on artificial intelligence. The method comprises the steps of obtaining a two-dimensional projection image sequence of diamond micro-powder particles, calculating a projection matrix based on camera calibration parameters and geometric constraints, obtaining a multi-view image data set of the particles, establishing a pixel-level corresponding relation, extracting three-dimensional space coordinates of the surfaces of the particles, and reconstructing dense point cloud data of the particles. Establishing a local coordinate system based on the dense point cloud data, determining attitude parameters of particles in a three-dimensional space, if the attitude parameters deviate from a normal range, performing attitude compensation processing to obtain standardized point cloud data, and performing three-dimensional grid model construction on the standardized point cloud data; and calculating geometrical characteristic parameters of the particles based on the three-dimensional grid model, performing defect detection on the surfaces of the particles, and generating a crystal integrity evaluation report of the particles. The quality detection accuracy of the diamond high-strength micro-powder particles is improved.

The invention discloses a pump shell welding seam quality detection method based on image segmentation. The method comprises the following steps: generating a steady-state pump shell welding seam image flow under the driving of motion compensation; obtaining a domain adaptive DINOv2 visual embedded feature map; performing adaptive pyramid fusion and cross-scale attention operation on the domain adaptive DINOv2 visual embedded feature map to generate a semantic form segmentation map; generating a semantic-texture fusion mask; performing uncertainty weighted optimization on the semantic-texture fusion mask in combination with the pixel-level confidence map to obtain a weld defect instance map; generating an interpretable texture anomaly heat map; and through multi-view supplementary shooting or manual auditing, supplementary pump shell welding seam image data is obtained, and the steady-state pump shell welding seam image flow is updated. According to the method, the system can continuously keep accurate positioning of the pixel-level segmentation boundary in a weak-label or even non-label migration scene, and boundary drift and area missing detection of a segmentation result are effectively avoided.

The invention relates to the field of welding, and discloses a non-standard intelligent customized welding system based on vision and surface gradient, which comprises a visual perception module used for iteratively approaching a workpiece from a preset initial height through a self-adaptive high angle shooting exploration mechanism, and combining layered grid division based on a camera view and a progressive multi-angle expansion scanning mode, collecting multi-view three-dimensional point cloud data of the workpiece; and the point cloud processing module is used for performing regional progressive registration on the multi-view three-dimensional point cloud data, implementing threshold constraint based on point cloud curvature characteristics by dynamically adjusting registration step length, and eliminating accumulative errors in combination with pose map optimization. Through self-adaptive high-angle shooting and layered grid scanning, multi-view point cloud can be automatically collected without manually presetting a workpiece model, regional registration and curvature constraint are combined, the workpiece model is constructed, a weld joint structure is automatically extracted based on surface gradient features, traditional manual labeling is replaced, a welding track is generated according to weld joint topology, and the precision is dynamically corrected.

The invention discloses a data enhancement method and system based on multi-agent self-evolution and hybrid evaluation, and relates to the technical field of artificial intelligence, and the method comprises the steps: employing a teacher model as a multi-agent cooperation system, and carrying out the interaction of a reasoning agent, an evaluation agent, a reflection agent and a memory management agent through a reasoning agent, an evaluation agent, a reflection agent and a memory management agent; iteratively generating a high-quality reasoning data set with a traceable reasoning path and a self-verification label, and performing multi-task supervision fine tuning on a learning model; a diversity sampling strategy based on reinforcement learning is adopted to generate multiple groups of outputs, and a weak point data set is screened by using a consistency score; and for the weak point data set, the multi-view instruction rewriting agent performs diversity rewriting on the instruction and then returns to perform deep reasoning distillation to generate new enhanced data and combine the new enhanced data to high-quality reasoning data. According to the method, multiple agents are arranged, so that the model is iteratively synthesized, evaluated, reflected and modified, the obtained enhanced data can be directly used, a manual auditing step is omitted, and the synthesis efficiency is improved.

The invention provides a multi-view-oriented three-dimensional scene image reconstruction registration and optimization method and system, and relates to the technical field of image processing, and the method comprises the steps: obtaining multi-frame three-dimensional scene image data, extracting a multi-level feature set, constructing a cross-view-angle semantic association graph, building a feature corresponding relation, and calculating a multi-view-angle spatial transformation relation parameter. Performing coordinate system alignment on the image data to generate an initial three-dimensional reconstruction result, and performing optimization in combination with a multi-target joint optimization function and a dynamic adaptive weight regulation and control mechanism. According to the method, the precision and robustness of three-dimensional scene reconstruction are improved, and the problem of registration errors caused by large view angle difference in a complex scene is solved.

The invention discloses a ship-shore cooperative tracking and positioning method based on multi-modal sensor fusion, and belongs to the technical field of target positioning. The method comprises the steps that multi-sensor layout and sensor fusion calibration are carried out on a target ship set and a shore end respectively, multi-view image sequence data and three-dimensional point cloud data are obtained, and the target ship set comprises a plurality of target ships; performing data fusion based on the multi-view image sequence data and the three-dimensional point cloud data to obtain fusion data of the target ship set, establishing an adaptive motion state model and an adaptive observation model based on the fusion data, and performing state prediction, state updating, data association and tracking management on the target ship set by adopting an unscented Kalman filtering algorithm; and establishing a space-time diagram model based on the Kalman filtering fusion observation factor and the Kalman filtering state prediction factor, and performing pose optimization on the target ship set in combination with the GPS factor. According to the method, the accuracy of ship and ship-shore cooperative positioning is improved.

The invention discloses a high-resolution three-dimensional reconstruction method of a fusion diffusion model, which belongs to the technical field of image data processing, and comprises the following steps: constructing an original data set D; constructing an enhanced training set; constructing a three-dimensional reconstruction network which comprises a text encoder, a renderer, a VAE encoder, a conditional diffusion model, a VAE decoder and an MVS module; training and fine-tuning the conditional diffusion model in three stages to obtain a three-dimensional reconstruction model, acquiring an image sequence and a text instruction of a scene to be reconstructed, and performing reconstruction by using the three-dimensional reconstruction model. According to the method, highly consistent geometric and color reduction can be kept under the multi-view condition, and splicing artifacts are remarkably reduced. Through semantic guidance optimization, texture details and structural consistency of the reconstruction model are greatly improved. Conditional diffusion sampling enables the model to accurately restore local details in a complex scene, and the stability of real-time rendering is improved.

The invention relates to a substation operation risk identification method based on a multi-view video and high-precision positioning. Acquiring video data of a working site through a plurality of cameras with fixed visual angles and mobile video acquisition equipment; a high-precision positioning system is used for obtaining three-dimensional space coordinates of operators and equipment in real time; establishing a three-dimensional digital twinborn model of the substation equipment, and performing dynamic scene reconstruction based on the multi-view video stream to generate a real-time three-dimensional scene of the operation site; fusing the positioning data and the three-dimensional scene by adopting a space-time fusion algorithm to generate a dynamic digital portrait of the operator; and carrying out real-time analysis on behaviors and positions of operators by using a risk assessment algorithm based on a preset risk rule, calculating to obtain a risk assessment value, and setting a feedback mechanism to continuously optimize positioning and scene reconstruction precision. According to the invention, efficient, accurate and real-time identification and early warning of the operation risk of the transformer substation are realized, and the safety management level of an operation site is effectively improved.

The invention discloses a body-in-white welding seam recognition and automatic welding method based on a machine vision technology, particularly relates to the technical field of computer vision and image processing, and is used for solving the problem of welding seam track recognition accuracy caused by insufficient processing capability of an existing three-dimensional vision recognition method on incomplete and uncertain point cloud data. Through the steps of multi-view point cloud acquisition and registration, probabilistic confidence evaluation, region growth of track continuity constraint, multi-track fusion optimization and the like, accurate identification of a body-in-white welding seam track under a complex working condition is realized; firstly, multi-view point cloud data are obtained, probabilistic registration is carried out to generate a confidence evaluation result, then candidate tracks are generated based on confidence weighting and semantic constraint, finally, an optimal track is generated through intelligent optimization and converted into a welding instruction which can be executed by a robot, and the accuracy and robustness of weld joint recognition are effectively improved.

The invention discloses a cross-source data three-dimensional reconstruction method and system based on improved Gaussian sputtering, and the method comprises the steps: collecting an unmanned plane inclined image and a ground panoramic image of a target region, and constructing a time-space correlation data set; based on multi-view geometric constraints, space-time coding matching point pairs are established through an adaptive feature pyramid, intelligent incremental cross-source data sparse reconstruction is carried out, and point cloud and camera parameters are output; adopting improved Gaussian sputtering, compressing a three-dimensional Gaussian kernel into a two-dimensional Gaussian primitive through double tangent vector constraint, and fitting surface geometry to realize multi-scale reconstruction; and optimizing primitive parameters by using a differentiatable renderer, completing multi-scale fine reconstruction through gradient back propagation, and generating a high-precision three-dimensional model. According to the method, multi-scale accurate geometric prior input and accurate camera poses are provided for three-dimensional reconstruction, the dependence on professional manual operation in a traditional three-dimensional reconstruction method is greatly reduced, and meanwhile, the geometric accuracy and visual fidelity of a reconstruction result are remarkably improved.

The invention relates to the technical field of panoramic image construction, in particular to a panoramic image reconstruction method and system based on multi-angle imaging. The method comprises the following steps: collecting a multi-angle original image based on a distributed multi-camera array, carrying out adaptive filtering denoising and adaptive panoramic imaging adjustment, and constructing a multi-angle imaging geometric constraint network; performing multi-view semantic information deviation elimination based on a multi-angle imaging geometric constraint network, and performing global semantic feature fusion to obtain a unified semantic space representation framework; identifying illumination feature information of different visual angles, performing multi-angle illumination corresponding compensation on the multi-angle original image, performing image semantic distortion correction based on a unified semantic space representation framework, and constructing a multi-angle illumination compensation image; and performing multi-scale texture structure analysis on the multi-angle illumination compensation image to generate a high-fidelity texture fusion image. According to the invention, a natural and seamless panoramic image is provided, a panoramic scene is perfectly presented, and the immersive visual experience of a user is improved.

The invention discloses a complex terrain three-dimensional modeling and earthwork volume calculation method based on a multi-source fusion point cloud, belongs to the technical field of surveying and mapping and engineering surveying, and mainly solves the problems of low terrain modeling precision and insufficient earthwork calculation efficiency in a complex scene. The method comprises the following steps: constructing a ground-air integrated multi-source sensing network to synchronously acquire laser point cloud, multi-view images and positioning data; adopting PointNet + +-based initial registration and multi-scale ICP fine registration fusion to generate a unified point cloud; combining semantic segmentation and penetration probability filtering to accurately extract a digital elevation model; constructing a similar triangular prism voxel model by using a constrained Delaunay triangulation network; and finally, the earth volume is rapidly calculated by adopting a GPU parallel voxel cutting and filling algorithm, so that high-precision terrain modeling and rapid engineering quantity calculation are realized, and the method can be efficiently and reliably applied to large-scale projects such as roads and mines.

The invention provides an EEG (electroencephalogram) classification method based on multi-domain feature fusion. A multi-domain feature extraction network is constructed, the multi-domain feature extraction network mainly comprises a frequency domain feature extraction module and a space-time feature extraction module which are deployed in parallel, a feature fusion module and a classifier module, multiple view features such as a time domain, a frequency domain and a space domain can be separated, and electroencephalogram signal classification is achieved. According to the method, an efficient solution is provided for solving the problem of insufficient multi-domain feature utilization of the electroencephalogram signals, the cross-scene classification precision can be remarkably improved while the model efficiency is kept, and a technical foundation is laid for personalized deployment of brain-computer interfaces.

The invention discloses an irregular particle three-dimensional shape measurement device and method based on speckle imaging, and belongs to the field of irregular particle three-dimensional shape measurement, and the irregular particle three-dimensional shape measurement device comprises an irregular particle shakeout device which is used for enabling irregular particles to fall in a sparse free falling body form, and is used for forming lamellar laser vertical to the falling direction of the irregular particles, the laser sheet generating module is used for forming complete speckle distribution on the surfaces of the irregular particles; the laser sheet generating module is used for generating a laser sheet, the multi-view image collecting module is used for collecting speckle distribution on the surfaces of irregular particles in a multi-view mode and forming speckle images, the synchronous control system is used for controlling the laser sheet generating module and the multi-view image collecting module and synchronizing the time of the laser sheet generating module and the multi-view image collecting module, and the three-dimensional reconstruction processing unit is used for obtaining three-dimensional shape data of the irregular particles according to the speckle images. According to the invention, the irregular particles can be rapidly, non-destructively and in-situ measured so as to meet the requirements of aeroengine erosion research on the precision, efficiency and integrity of irregular particle data.

The invention discloses a robot obstacle avoidance method and system based on millimeter wave radar sparse point cloud, and relates to the technical field of obstacle avoidance recognition. A robot obstacle avoidance system based on millimeter wave radar sparse point cloud comprises a point cloud acquisition module, a negative obstacle identification module, a weak obstacle identification module, a point cluster identification module, a risk map module, a tentative verification module and an obstacle avoidance decision module. According to the invention, suspected obstacle point clusters are extracted based on a reflection intensity threshold and a spatial proximity relation in an enhanced point cloud, a theoretical parallax model of a real static obstacle is constructed under the constraint of a robot motion trajectory, and Doppler velocity distribution of each frame is combined with a static obstacle Doppler physical law for comparison. And classifying the point clusters which do not meet the multi-view geometric consistency or Doppler physical law, and distinguishing multipath false point clusters from dynamic point clusters.

The invention provides a coal mine operation and maintenance monitoring method and device, electronic equipment and a storage medium. Environmental parameters and equipment state data are collected in real time through the Internet of all things deployed in an underground coal mine; preprocessing the environment parameters and the equipment state data by utilizing an edge computing node; the method comprises the following steps: acquiring point cloud data and a multi-view image of an underground coal mine, and splicing the point cloud data and the multi-view image to construct a three-dimensional scene model; the preprocessed data and the three-dimensional scene model are dynamically fused, and the equipment state features, the environment features and the risk early warning parameters are displayed in the three-dimensional scene model in a three-dimensional visualization mode in an overlapping mode; under the abnormal condition, abnormal data are received, maintenance steps or fault points are marked in a virtual interface of the monitoring center, data islands are broken, and multi-source data are deeply fused; and real-time guidance can be provided through interaction in modes of gestures, voice and the like, so that the maintenance efficiency is improved.

The invention relates to the technical field of image compensation, in particular to a reflection compensation method based on image processing, and provides the following scheme: acquiring an environment panoramic image by using a panoramic camera to establish a scene global coordinate system, and acquiring a multi-view original image in combination with a camera array arranged in the circumferential direction of an object; determining a rotation symmetry axis according to the multi-view contour features, reconstructing a three-dimensional geometric body, and calculating normal distribution and curvature change; generating a prediction image based on diffuse reflection and specular reflection hypothesis in the surface expansion domain through virtual visual angle disturbance, identifying a reflection region according to color and gradient consistency, and generating a reflection mask; and performing texture reconstruction on the reflective area by using geometric registration and multi-view compensation, and finally performing splicing and fusion to obtain a non-reflective high-fidelity panoramic image. The method does not need to change the field illumination condition, and can achieve the precise recognition and compensation of the complex curved surface reflection in the cultural relic in-situ collection environment.

The invention relates to the technical field of three-dimensional imaging, in particular to a four-eye structured light stereoscopic vision imaging method, which comprises the following steps of: arranging four cameras and synchronously acquiring multi-view image data with structured light stripes; carrying out image preprocessing and stripe code identification on the obtained four-view-angle structured light image data, and extracting structured light stripe center line positions and corresponding space projection information under each view angle; generating three-dimensional point cloud data of the high-precision target sample by using parallax calculation and a three-dimensional reconstruction algorithm, and completing spatial registration and filtering optimization of point cloud; fusing the point cloud and the light intensity data by combining the reflection intensity information of the multi-view structured light stripes to generate a composite data set containing space and spectral information; and constructing a dense parallax field and executing three-dimensional consistency verification, and generating a high-fidelity three-dimensional reconstruction model with a complete topological relation and sparse shielding compensation capability. According to the invention, the problems of insufficient view angle coverage, shielding area information loss, difficult edge structure matching and the like of traditional stereoscopic vision imaging can be solved.

The invention belongs to the technical field of three-dimensional scene reconstruction, and particularly discloses a multi-view three-dimensional Gaussian densification method and system for adaptive density control, and the method comprises the following steps: collecting a multi-view original image, and carrying out the preprocessing of the multi-view original image; complexity features are extracted, a pixel-level complexity heat map is generated, and a globally unified three-dimensional complexity field is constructed; performing back projection on the reconstruction residual error, high-frequency inconsistency and depth / geometric consistency cost of each view angle, generating three-dimensional error popularity, determining a candidate newly-added set and a candidate pruned set, generating a weak label to train a lightweight multilayer perceptron classifier, outputting a ternary probability corresponding to newly-added / pruned / maintained, and obtaining a new / pruned / maintained three-dimensional perceptron classifier; and performing Gaussian densification operation on the newly added region. By adopting the technical scheme, fine point adding is carried out on the complex area, effective pruning is carried out on the simple area, and meanwhile, the synthesis quality, the global consistency and the calculation efficiency of the new view angle are improved.

The invention discloses a cross-platform virtual-real fusion scene construction method and system based on AI space calculation, and relates to the technical field of artificial intelligence and space calculation, and the method comprises the steps: carrying out the multi-scale feature fusion based on a received cross-modal conversion instruction set, and generating an initial image sequence; carrying out implicit field coding on a target object by combining a three-dimensional reconstruction algorithm to obtain an initial parameterized model; performing space-time alignment on the multi-view video stream, loading a digital scene asset package in combination with physical sensing data and a preset spatial index structure, and establishing a bidirectional data channel between a virtual scene and a physical sensor; performing rendering and illumination parameter adjustment on the initial parameterized model to obtain an optimized parameter model; performing differential coding processing on the optimization parameter model to obtain a target virtual-real scene fusion model; and distributing the target virtual-real scene fusion model to a preset terminal. The invention provides a virtual-real fusion construction method for end-to-end collaborative optimization, which is suitable for cross-platform live broadcast or dynamic interaction scenes.

The invention discloses an unmanned aerial vehicle inspection method and system applied to foundation pit accumulated water monitoring, and the method comprises the steps: determining an optimal route node sequence, and generating an unmanned aerial vehicle control instruction set; acquiring internal and external parameters of a multi-source sensor, and performing time alignment compensation on image frames and sensor data; acquiring a ponding area image coordinate set and attribute data; a three-dimensional projection point set of the ponding area is obtained, and the real coverage area of the ponding area is calculated; generating a three-dimensional overview map of the foundation pit by using the multi-view image of the foundation pit area; determining a three-dimensional coordinate point set and an area estimation result of the ponding area; and projecting the three-dimensional coordinate point set of the ponding area to the three-dimensional overview map of the foundation pit to generate a hidden danger monitoring report. By integrating the unmanned aerial vehicle, the sensor data and the advanced image processing technology, the accuracy, stability and visualization effect of foundation pit accumulated water monitoring are greatly improved, more reliable and comprehensive technical support is provided for safety management of foundation pit engineering, and the safety risk in the construction process is remarkably reduced.

The invention provides a scene reconstruction and semantic understanding method and system based on a visual large model, and relates to the field of computer vision and three-dimensional reconstruction, and the method comprises the steps: obtaining multi-view image data of a target scene, inputting a pre-training visual large model, and outputting a joint feature representation and attention weight matrix; generating three-dimensional coordinate values and semantic probability distribution of spatial sampling points in a three-dimensional space according to the joint feature representation, and constructing a spatial semantic field; clustering the spatial sampling points by using the attention weight matrix, performing semantic consistency enhancement, and converting the spatial sampling points into deterministic semantic tags; constructing a geometric optimization objective function, and adjusting three-dimensional coordinate values; and extracting continuous space sampling points with the same semantic tag to form an object boundary, constructing a scene topological graph, deducing a scene functional structure, and generating a navigation path. According to the method, the unification of accurate geometric reconstruction and deep semantic understanding of the scene is realized, the three-dimensional reconstruction precision and semantic analysis accuracy are improved, and reliable support is provided for intelligent navigation.

The invention provides a physical training posture correction method based on machine vision, which comprises the following steps: acquiring an athlete training image sequence through a multi-view image acquisition system and preprocessing the athlete training image sequence, extracting three-dimensional posture key points of a human body by utilizing a depth posture recognition model, constructing a skeleton model and calculating real-time kinematics characteristic parameters, and performing multi-dimensional comparison with standard parameters to generate a posture deviation evaluation report, thereby generating a personalized multi-modal correction feedback scheme, and establishing a personal athlete movement feature database to realize adaptive standard parameter optimization. According to the method, the training postures of athletes can be accurately recognized and corrected in real time, the training effect can be improved, sports injuries can be prevented, and training individuation and scientificity are improved.

The invention relates to a deformation online measurement and control method for multi-robot collaborative assembly. The method comprises the steps that multi-view surface images of workpieces in the assembly process are collected in real time through distributed robots and cameras arranged at the tail ends of the distributed robots; each view angle surface image is input into a deep learning model trained based on a digital image related technology, the optimal pixel displacement corresponding to each view angle surface image is output, and the optimal pixel displacement corresponding to each view angle surface image is converted into a spatial displacement label; the deep learning model takes an encoder-decoder as a trunk network; fusing the spatial displacement labels corresponding to the view angle surface images to obtain a fused displacement field; based on the fusion displacement field and the nominal path planning point, the tail end pose of the distributed robot is determined; the assembly error is calculated based on the reference target positioning and the tail end pose, and the PID controller adjusts the joint space of the distributed robot based on the assembly error. According to the method, the calculation overhead is remarkably reduced, and the measurement precision and robustness are improved.

The invention relates to the technical field of artificial intelligence, can be applied to business scenes such as automatic driving and intelligent transportation, financial science and technology, medical health and the like, and discloses a three-dimensional scene event analysis method, device, equipment and medium. Generating a three-dimensional bounding box for a dynamic target in each frame of multi-view image data, fusing the information of the three-dimensional bounding box with the features of the semantic graph, generating a refined three-dimensional voxel semantic graph, aggregating multiple frames of refined three-dimensional voxel semantic graphs, constructing a motion track of the dynamic target, and detecting a space overlapping event based on the motion track. And generating a collision event description based on the relative motion state, and generating an attribution analysis result of the event. According to the method, the accuracy and automation level of collision detection and responsibility analysis in a dynamic scene are improved by fusing the multi-frame image information and the three-dimensional voxel data and combining the target position and the motion state.

The invention belongs to the technical field of three-dimensional scene understanding, and particularly relates to a training method and device for a three-dimensional open vocabulary semantic segmentation model. The training method comprises the steps of obtaining multi-view RGB-D images of a target area, performing multi-stage reasoning on each image through a visual language model, generating a target vocabulary list, prompting a two-dimensional segmentation model to establish a pixel-level text label, performing depth mapping on the images to generate a first point cloud, and generating a second point cloud; mapping the text tag to the first point cloud to generate a point-by-point text tag; pre-training a neural network model with a sparse encoder-decoder structure by taking the point-by-point text label as a supervision signal, and generating a three-dimensional segmentation model on the first point cloud; and for the second point cloud of the complete scene of the target area, matching point feature embedding and text embedding with the highest similarity in the shared vision-language feature space, generating a credible point-text tag pair, and finely adjusting the three-dimensional segmentation model based on the credible point-text tag pair.

A deep neural network(s) (DNN) may be used to detect objects from sensor data of a three dimensional (3D) environment. For example, a multi-view perception DNN may include multiple constituent DNNs or stages chained together that sequentially process different views of the 3D environment. An example DNN may include a first stage that performs class segmentation in a first view (e.g., perspective view) and a second stage that performs class segmentation and / or regresses instance geometry in a second view (e.g., top-down). The DNN outputs may be processed to generate 2D and / or 3D bounding boxes and class labels for detected objects in the 3D environment. As such, the techniques described herein may be used to detect and classify animate objects and / or parts of an environment, and these detections and classifications may be provided to an autonomous vehicle drive stack to enable safe planning and control of the autonomous vehicle.