78 results about "Projection mapping" patented technology

The invention, which relates to the technical field of image processing, discloses a city component automatic three-dimensional reconstruction method based on a sub-meter high-resolution remote sensing image, comprising the following steps: acquiring a sub-meter high-resolution remote sensing image of a target area, and establishing an initial ground-image projection mapping relation based on a rational polynomial coefficient model; acquiring a ground control point, correcting the mapping relation, and outputting a registration image; performing multi-view geometric matching on the registered image to generate dense earth surface point cloud, and inputting the dense earth surface point cloud into a semantic neural radiation field network to generate dense point cloud; performing cross-modal feature alignment and fusion on the dense point cloud and the vector data, and outputting a fused multi-source dense point cloud; according to the method, through fine correction of the multi-source remote sensing image, the problems of large registration error, more point cloud sparse noise and semantic deficiency are solved, and high-precision and automatic three-dimensional reconstruction of urban components is realized.

The invention discloses a naked-eye 3D display optimization method based on real-time eyeball tracking, and particularly relates to the technical field of naked-eye 3D display, and the method comprises the steps: capturing eyeball movement data in real time through a visual angle tracking sensor, and collecting illumination information in combination with an ambient light sensor; depth perception parameters are calculated based on pupil diameter variation and frequency, and a time sequence prediction type dynamic compensation coefficient is generated in combination with eyeball movement acceleration; acquiring an initial fixation point coordinate by using an improved spherical projection mapping model, and performing compensation coefficient correction to obtain a real-time coordinate; and finally, according to the real-time coordinates, dynamically adjusting the refractive index distribution of the nanostructure layer, the rotation angle of the polarizer, the focal length of the optical lens and other optical modulation parameters. The real-time distance is calculated through the binocular parallax algorithm, the depth mapping value is generated by combining the focal length of the camera and the baseline distance, the method can adapt to the illumination change and the user view angle, and the 3D display effect is optimized.

The invention discloses a multi-modal large model based on dynamic decision hybrid expert mapping, which comprises a visual coding group comprising a plurality of visual coding structures with different functions, a visual fusion and compression module, a dynamic decision hybrid expert mapping module and a large language model, the dynamic decision hybrid expert mapping module comprises a hybrid expert module and a shared expert module which respectively process the multi-scale image features to obtain high-discrimination features and comprehensive features, and a decision expert module which is used for carrying out importance estimation on the multi-scale image features; the decision connection module and the self-adaptive projection mapping layer are used for connecting the high-discrimination features and the comprehensive features according to importance estimated values, and the dynamic decision hybrid expert mapping module intelligently distributes parameters to different expert networks and dynamically adjusts weight configuration of experts. And the internal structure of the model can be automatically optimized according to different inputs, so that higher processing precision is achieved.

The invention relates to an AR projection interaction treadmill virtual scene generation method and device, equipment and a medium. The method comprises the following steps: acquiring motion inertia data and running belt motion parameters of a user on a running machine; respectively carrying out motion state extraction and gait phase extraction according to the motion inertia data and the running belt motion parameters to obtain a current motion state vector and gait phase information of the user; inputting the motion state vector and the gait phase information into a pre-trained motion state prediction model to obtain a preset predicted motion state of a plurality of frames in the future; based on the predicted motion state, performing layered scene generation through a scene generator to obtain a corresponding virtual scene; and performing projection mapping transformation processing on the virtual scene, and obtaining real-time AR projection data in combination with low-delay compensation. By adopting the method, a highly immersive and interactive virtual training environment can be provided in an AR projection mode.

The invention provides an insulator defect identification method based on cross-modal data fusion. The insulator defect identification method comprises the following steps: acquiring and preprocessing an insulator visible light image, a thermal infrared image and point cloud data which are synchronously acquired; projecting all points in the point cloud to obtain a projection image and a projection mapping matrix; registering the visible light image and the thermal infrared image with the projection image to obtain a corresponding affine transformation matrix, and constructing a shielding matrix for shielding the background according to the projection image; performing feature extraction on the pre-processed visible light image, thermal infrared image and point cloud data to respectively obtain a texture feature map, a temperature feature map and a geometric feature map; fusing the texture feature map, the temperature feature map and the geometric feature map according to the shielding matrix and the affine transformation matrix to obtain cross-modal fusion features; and the cross-modal fusion features are used to discriminate and position insulator defects. According to the method, the problem of leak detection of insulator defects in different scenes by a traditional single-mode method can be solved.

The invention relates to the technical field of model training learning, in particular to a training method of an intention recognition model for understanding user requirements in an intext software scene, which comprises the following steps of: traversing paths to determine levels and establish variable-dimension projection mapping, generating semantic orthogonal conflict samples, obtaining variable-dimension features by utilizing matrix transformation, and obtaining an intention recognition model for understanding user requirements in an intext software scene. And quantizing the brother node overlapping degree to apply orthogonal constraint, updating the weight by minimizing the positive sample distance and maximizing the conflict sample distance, and constructing an intext software intention recognition optimization model. According to the method, a mapping mechanism of a hierarchical position and a variable-dimension projection matrix is established through software architecture topology, a semantic orthogonal logic conflict sample is constructed, brother node feature branch orthogonal constraint is introduced, and feature representation of parallel architecture branches is forcibly separated in a vector space; the logic conflict sample spacing is maximized, semantic overlapping of similar function nodes is eliminated, and a decision boundary with high distinction degree is constructed to accurately analyze a complex nested instruction.

The invention provides a speed measurement method based on dynamic zooming of an unmanned aerial vehicle and an inspection system. The method comprises the following steps: acquiring a video stream acquired by the unmanned aerial vehicle in real time through a camera for dynamically adjusting focal length parameters; inputting the video stream into the detection model frame by frame; outputting a target detection frame set of the corresponding current frame; performing cross-frame association on the detection frames of the adjacent frames through a ByteTrack algorithm, and distributing a unique instance identifier for the same target; acquiring real-time parameters of the unmanned aerial vehicle, establishing a real pavement coordinate system and a pixel imaging coordinate system, and performing coordinate system projection mapping; and calculating the real speed of the target. The method has the beneficial effects that the coordinate mapping error is greatly reduced through dynamic parameter fusion and unmanned aerial vehicle displacement compensation, and particularly, the improvement is obvious in a pitch angle or zoom scene; through introduction of central point position transformation, image center offset caused by digital zooming is dynamically calibrated, and pixel-physical coordinate mapping distortion caused by pure image cutting or interpolation amplification is effectively eliminated.

The invention discloses a laser radar real-time semantic segmentation method and system based on image and point cloud multi-view fusion knowledge distillation, and the method comprises the steps: obtaining laser radar point cloud data and camera RGB image data, building a projection mapping relation between point cloud and image pixels, and enabling the mapping relation to be used for searching a common effective projection point; laser radar point cloud data is converted into a voxel grid view and a distance plane view, a main and auxiliary double-branch architecture is constructed in combination with camera RGB image data, hierarchical compression features are fused through three-level mutual attention gating, and multi-modal multi-view fusion features are generated; respectively distilling the multi-modal multi-view fusion features to a main branch and an auxiliary branch by adopting a bidirectional knowledge distillation strategy, and synchronously optimizing the dual-branch feature extraction capability; the auxiliary branch is unidirectionally distilled to the main branch, so that the performance of the main branch is further improved; and migrating the semantic segmentation capability on the point cloud multi-view and image-point cloud multi-modal fusion features to a point cloud voxel network, thereby realizing real-time high-precision semantic segmentation of a radar point cloud single mode. According to the method, the precision and the real-time performance of point cloud semantic segmentation in a dynamic scene are remarkably improved.

The invention provides a condition image generation method based on point rasterization and a related device, and the method comprises the steps: carrying out the coloring processing of a target point cloud set according to an environment image corresponding to a target moment; performing foreground and background separation processing on the target point cloud set after coloring processing to determine a dynamic foreground region and a static background region in the target point cloud set; complementing the incomplete area in combination with a historical point cloud set in the time window; projecting the complemented target point cloud set to an image coordinate system of the vehicle by using a perspective projection mapping relation to obtain a first projection image; and rendering the grating area in the first projection image to obtain a point rasterization condition image. According to the method, frame-by-frame analysis and image plane mapping are carried out on original laser radar point cloud data, point rasterization processing is combined, a geometric condition image with pixel-level precision is constructed, and structural prior support is provided for subsequent new view synthesis and controllable video generation.

The invention provides a three-dimensional semantic scene reconstruction method and device, electronic equipment and a storage medium, and belongs to the technical field of artificial intelligence. The method comprises the following steps: determining three-dimensional coordinates of RGB image pixel points in a world coordinate system; extracting two-dimensional image features of pixel points in the RGB image to obtain a two-dimensional feature image corresponding to the RGB image; based on the three-dimensional coordinates of the pixel points in the RGB image under the world coordinate system, mapping the pixel points in the RGB image to the three-dimensional occupied grid sub-model under the world coordinate system to obtain a target three-dimensional occupied grid sub-model; determining a grid feature vector of a three-dimensional grid in the target three-dimensional occupied grid sub-model based on the target three-dimensional occupied grid sub-model, the two-dimensional feature image corresponding to the RGB image and the projection mapping matrix; and calling the three-dimensional semantic prediction sub-model, processing the grid feature vector of the three-dimensional grid in the target three-dimensional occupation grid sub-model, and obtaining a good reconstruction result of the target indoor scene.

The invention relates to the technical field of strain measurement, in particular to a three-dimensional full-field strain measurement method and system based on multi-view collaboration, and the method comprises the steps: constructing a sparse topology skeleton model of a measured target under a global coordinate system, and building a bidirectional projection mapping relation between the global situation and each local situation; the method comprises the following steps: synchronously acquiring a time sequence texture feature image of a measured target, calculating a luminosity geometric coupling confidence tensor of each pixel point and extracting an initial texture feature field under each view angle; utilizing a skeleton model to guide the initial texture feature field to perform limited local matching, and resolving to obtain a local independent displacement field and residual energy distribution of each view angle; performing multi-view weighted fusion based on a bidirectional projection mapping relation, and reconstructing a global three-dimensional displacement field; and performing differential calculation on the global three-dimensional displacement field to obtain a deformation gradient tensor and generate three-dimensional full-field strain distribution. According to the method, gradient mutation caused by view field splicing errors can be effectively inhibited, and physically real and full-field continuous three-dimensional strain distribution is obtained.

The application provides a speed measurement method and an inspection system based on dynamic zoom of a UAV, comprising: acquiring a video stream collected by a camera of the UAV in real time by dynamically adjusting a focal length parameter; inputting the video stream into a detection model frame by frame; outputting a target detection frame set corresponding to a current frame; performing cross-frame association on detection frames of adjacent frames by a ByteTrack algorithm to assign a unique instance identifier to the same target; acquiring real-time parameters of the UAV, establishing a real road surface coordinate system and a pixel imaging coordinate system, and performing coordinate system projection mapping; and calculating a real speed of the target. The application has the beneficial effects that dynamic parameter fusion and UAV displacement compensation greatly reduce coordinate mapping errors, especially in a pitch angle or zoom scene, and the improvement is obvious; by introducing a center point position transformation, the application dynamically calibrates image center deviation caused by digital zoom, effectively eliminating pixel-physical coordinate mapping distortion caused by simple image cropping or interpolation enlargement.

The invention discloses a closed data space model test self-adaption method based on double-stage feature whitening, and belongs to the field of model self-adaption, and the method comprises the steps: inputting each enhanced view in an enhanced data set composed of a first enhanced view and a second enhanced view into a data processing model, obtaining an intermediate layer activation feature and an output result respectively corresponding to each enhanced view; determining a first whitening feature corresponding to each intermediate layer activation feature, and determining a second whitening feature corresponding to each first whitening feature after projection mapping based on all the first whitening features; and performing parameter updating on a target data layer in the data processing model based on joint model loss constructed by the output result and the second whitening feature. According to the method, a two-stage feature whitening mechanism is introduced, so that the model can obtain better adaptive ability in a closed data space, and the generalization performance and reasoning stability of the model in a complex test environment are improved.

The present application provides a kind of conditional image generation method and related device based on point rasterization, according to the environment image corresponding to target time, target point cloud set is colored processing;The target point cloud set after coloring processing is carried out front and background separation processing, to determine the dynamic foreground region and static background region therein;Combining the historical point cloud set in time window, the incomplete area is completed;Using perspective projection mapping relationship, the target point cloud set after completion is projected to the image coordinate system of vehicle, to obtain first projection image;The raster area in first projection image is rendered, to obtain the conditional image of point rasterization.It is analyzed frame by frame to original laser radar point cloud data and image plane mapping, and combined with point rasterization processing, constructs the geometric conditional image with pixel level precision, provides structure priori support for subsequent new view synthesis and controllable video generation.

The invention relates to the field of medical image processing, and discloses a method and device for fusing a DR image and an ultrasonic image, computer equipment and a storage medium, and the method comprises the steps: obtaining a DR image containing a DR detection array position; the position of the DR detection array is calibrated through optical positioning equipment; acquiring an ultrasonic image containing position data of a probe detection array; the position data of the probe detection array are calibrated through optical positioning equipment; determining a projection mapping relation between the DR image and the ultrasonic image according to the position data of the DR detection array and the position data of the probe detection array; and fusing the DR image and the ultrasonic image according to the projection mapping relation to obtain DR ultrasonic bimodal fusion data. According to the method, the precision and efficiency of DR image and ultrasonic image fusion are effectively improved, and reliable bimodal visualization support is provided for medical image diagnosis.

The present application relates to the technical field of projectors, and specifically provides a kind of intelligent projector control method based on color correction and depth correction, image and depth information of projection plane are collected by optical perception device, and color deviation tensor and depth deviation tensor between it and ideal plane are calculated, color correction and depth correction are carried out on pre-projection image according to color deviation tensor and depth deviation tensor, so that projected image will not be affected by color and flatness on projection plane.The present application avoids color interference and flatness influence of projection plane, ensures that projector can realize flat, achromatic projection under visual effect in extreme conditions such as plane and space irregularity, color stain and the like, and solves the problems such as precision limitation, single scene and high cost in existing projection mapping application.

The invention relates to a projection mapping treadmill surface dynamic guiding method and device, equipment and a medium. The method comprises the following steps: acquiring a continuous image sequence on the surface of a treadmill and calculating the displacement change of key feature points to obtain a velocity vector; performing pose calculation based on the velocity vector to generate initial pose estimation data; constructing a three-dimensional surface mesh model which can be updated in real time according to the attitude data; extracting curvature and normal vector information when the updating frequency of the model reaches the standard; calculating a projection distortion degree based on the geometric features and generating a correction matrix; performing affine transformation on the original projection content through the correction matrix to obtain projection image data matched with a treadmill posture; and generating a projection guide instruction according to the transformed image. By adopting the method, the surface deformation of the treadmill can be adapted in real time, the projection distortion is accurately corrected, and the dynamic matching of the motion guide information and the geometric attitude of the treadmill is ensured.

The application relates to the technical field of projectors, and particularly provides an intelligent projector control method based on color avoidance and depth correction, candidate regions on a pre-projection image are determined according to a color deviation tensor of a projection plane and a global color fitting plane, a projection region is divided in the candidate regions according to the shape of the pre-projection image, depth correction is performed on the projection region according to a depth deviation tensor of the projection region and a global depth fitting plane, and the pre-projection image after the depth correction is projected on the projection region obtained through color avoidance division. The application avoids the influence of color interference and flatness, guarantees that the projector can realize flat and achromatic projection under visual effect in the case of plane and space irregularity, color stains and other extreme conditions, and solves the problems of precision limitation, single scene and high cost in the existing projection mapping application.

The invention relates to a marine unstructured camera video stitching method, which comprises the following steps: S1, acquiring video stream data of a shipborne camera, the video stream data being initial data required for calculating stitching parameters; s2, the initial data obtained in the step 1 are processed, splicing mapping parameters are obtained, and the splicing mapping parameters comprise cylindrical projection mapping parameters, homography matrixes between adjacent cameras, self exposure compensation parameters and optimal suture lines; splicing mapping parameters are stored in a file form, and the parameters are loaded to a memory in a GPU mapping table form during subsequent splicing; and S3, on the basis of the splicing mapping parameters obtained in the step S2, carrying out online splicing on the video images. According to the method, the problem that the ship camera array is difficult to calibrate is solved, a more stable splicing effect is achieved in different illumination scenes, and compared with a traditional video splicing method, the method is higher in dynamic scene adaptability.

The application relates to the technical field of data labeling, and discloses a labeling method for model training data, which comprises the following steps: obtaining a multi-modal original data packet for model training; constructing a projection mapping matrix from a three-dimensional point cloud coordinate system to a two-dimensional image pixel coordinate system by using an internal parameter matrix of a camera acquisition device and an external parameter matrix of the camera acquisition device relative to a point cloud acquisition device; based on the projection mapping matrix, aligning an image frame in image data with a point cloud frame in point cloud data by using a timestamp sequence of the image data and the point cloud data, generating a spatio-temporal alignment fusion data frame; and labeling the fusion data frame. Based on the projection mapping matrix and the timestamp sequence of the image data and the point cloud data, the spatio-temporal alignment fusion data frame is generated, an objective geometric basis for data labeling is constructed, and the labeling accuracy of the labeled data is significantly improved. The application further discloses a labeling system for model training data, an electronic device and a crowdsourcing data labeling platform.

The present application relates to the technical field of computer vision, in particular to a method for fast processing 2D video and color fusion of real scene reconstruction three-dimensional model based on GPU, comprising the following steps: data acquisition and space-time reference calibration preprocessing; real scene three-dimensional basic model construction; GPU parallel space-time joint pose compensation projection mapping; re-projection error calculation and pose compensation iterative optimization; GPU accelerated color fusion optimization output. The present application designs the GPU parallel space-time joint pose compensation projection mapping, combines the acquisition timestamp difference and device motion parameter to calculate the pose compensation amount, matches the re-projection error iterative optimization, adapts the real scene three-dimensional reconstruction dynamic acquisition scene, corrects the space-time asynchronization and pose offset, accelerates the GPU parallel operation and guarantees the projection mapping accuracy; at the same time, relying on the GPU parallel rendering engine to complete the color difference correction, color tomography smoothing and edge transition processing, realizes the uniform fusion of 2D video color and three-dimensional model, and solves the problems of uneven color and connection trace.

The application belongs to the field of three-dimensional reconstruction, and particularly relates to a micro-morphology analysis method based on wavelet multi-layer frequency band decomposition. The method comprises the following steps: firstly, a multi-focus image sequence of a micro scene to be measured is collected, and multi-scale spatial features are obtained through feature coding and projection mapping; then, a two-dimensional discrete wavelet transform method is introduced, the spatial features are decoupled into a low-frequency approximate subband representing global geometry and a high-frequency detail subband representing fine texture, and structure feature correction and directional texture enhancement are respectively implemented; on this basis, inverse discrete wavelet transform and residual fusion are used to construct time-frequency enhanced features, a deep regression network is used to analyze pixel-level focus probability distribution to generate a continuous depth map; finally, a cascading mechanism driven by three-dimensional geometric reconstruction for scene morphology perception is established, and depth geometric edge constraints are used to reversely guide the high-fidelity generation of a full-focus image. Through the wavelet multi-layer frequency band decomposition strategy, the application effectively solves the problem of confusion between texture details and out-of-focus noise in micro imaging, and realizes the synchronous robust analysis of high-precision geometric structure and full-depth clear texture of micro morphology.

Some embodiments described herein relate to a method (e.g., a computer-implemented method) that includes receiving data associated with an operative field that includes a subject from an optical sensor. A three-dimensional (3D) virtual model associated with at least one of the subject or an object in the operative field can be accessed, and an observed mesh that includes a representation of the subject, based on the data received from the optical sensor can be defined. A virtual 3D environment, including the virtual model can be defined. The virtual model can be registered to the observed mesh, or the observed mesh can be registered to virtual model. A rendering of the virtual model can be projected, in real time, into the operative field such that the rendering of the virtual model is scaled and oriented relative to the at least one of the subject or the object in the real-world operative field as it appears in the virtual 3D environment.

The invention discloses a laser radar point cloud marking method and device, equipment and a medium, and relates to the technical field of image processing, and the method comprises the steps: constructing a projection mapping relation between an omnidirectional visual image and a laser radar point cloud based on a target space position relation; performing image recognition on the omnidirectional visual image by using a preset image recognition model to obtain a plurality of first target detection frames including object category labels; and determining a target area of the first target detection frame in the laser radar point cloud according to the projection mapping relation, performing point cloud clustering processing in the target area to obtain a target point cloud cluster, and fitting the target point cloud cluster to obtain a second target detection frame to complete marking operation of the laser radar point cloud. The detection frame with the category label is obtained through the image recognition model and mapped to the point cloud for clustering, and the technical problem that the point cloud processing delay is large is solved.

To more accurately identify object types in projection mapping. [Solution] The projection system 100 includes a temperature measuring device 20 that measures the temperature of an object, a projection device 40 that projects an image onto the object, and a control device 10 that controls the image projected onto the object by the projection device 40 based on the temperature of the object measured by the temperature measuring device 20.

The invention provides a distribution network branch line identification and positioning system and method fusing laser radar and a camera, and the system comprises a man-machine interaction terminal and a hot-line work robot, and the hot-line work robot is composed of an industrial robot, a visual perception module, a communication module, and a control module. The laser radar and the camera are fixedly installed on a tail end shaft of the industrial robot through mechanism connection, and the scanning dimension and position are driven by the mechanical arm to be adjusted to obtain visual data. And the control module processes the obtained visual data and robot attitude data, identification and positioning of the distribution network branch line are completed by adopting a projection mapping-data parameterization target point identification and positioning method, an obstacle and a target branch line can be accurately distinguished, and the operation point of the branch line is accurately positioned. The system has the advantages of high efficiency and economical efficiency, a full-automatic solution is provided for a distribution network live working scene, and power operation and maintenance are promoted to be transformed to intelligence.

One disclosed example provides a videoconferencing system comprising a processor and a storage device storing instructions executable by the processor to obtain an image of a scene acquired via a camera, the image of the scene comprising image distortion arising from a camera pitch angle at which the image of the scene was acquired. The instructions are further executable to apply a projection mapping to the image of the scene to map the image of the scene to a projection comprising a tilt parameter that is based upon the camera pitch angle at which the image of the scene was acquired, thereby obtaining a corrected image, and output the corrected image.

The invention relates to the technical field of industrial automatic control and fluid surface treatment, in particular to an automatic test panel spraying system based on PLC control. Comprising a virtual feature analysis module which is used for acquiring three-dimensional digital model data and a preset spraying track of a virtual target curved surface and constructing a virtual geometric feature data set containing curvature features of discrete points and a normal vector field; the equivalent flux calculation module is used for generating a target three-dimensional flux distribution data matrix; the projection mapping compensation module is used for generating a composite compensation instruction sequence containing dynamic attitude deflection data, variable-speed motion data and variable-fan-amplitude air pressure data; and the execution driving control module is used for converting the digital instruction into a multi-axis servo driving signal and an analog quantity adjusting signal, driving an execution mechanism to execute a time-space coordination action on a physical plane base material, and reproducing fluid deposition characteristics of a virtual target curved surface. According to the system, the consumption of trial and error materials is greatly reduced, and the conversion time from a laboratory to a production line is shortened.

This invention belongs to the field of human-computer interaction technology. It discloses a calligraphy and painting appreciation experience cabin, an interaction method, and a device. Through a immersive digital experience cabin, a somatosensory gesture interaction system collects the user's hand spatial motion data stream. Combined with ultra-high-definition original image layers and pre-annotated semantic partition maps of the image, a precise appreciation interaction experience is achieved. Based on the spatial curvature sequence and velocity decay characteristics of gesture trajectories, user behavior is classified into three trajectory types: global exploration, regional wandering, and fixed-point approach. Gestures are associated with semantic partitions of the image through projection mapping, constructing a two-dimensional intent index key to retrieve matching professional content. The method also includes multi-scale cropping and progressive detail enhancement techniques, allowing image details to gradually appear as the user's attention increases, and achieving a smooth transition of intent conversion through spatial deviation rate monitoring. This invention achieves a natural, intelligent, and immersive calligraphy and painting appreciation experience.