2062 results about "Colour image" patented technology

The invention relates to the technical field of multi-target tracking, in particular to a dynamic multi-target tracking and trajectory prediction method. Comprising the following steps: acquiring a color image and a depth image, and preprocessing; constructing a dynamic threshold double-flow neural network, extracting features from the preprocessed color image and depth image to obtain a visual feature set and a depth feature set, fusing the visual feature set and the depth feature set through an adaptive attention mechanism to obtain a fused feature set, and combining a dynamic threshold to obtain an observation information list of a target; updating the observation information list through a self-motion decoupling mechanism to obtain a real information list; establishing space association and time association between the target and the track based on the real information list to obtain a newest track set; and analyzing target characteristics, environmental constraints, social behaviors and historical trajectories, generating a prediction trajectory of each target and a corresponding multi-factor score, generating a time-varying influence graph, and performing planning and decision making by adopting a double-layer decision-making mechanism.

The invention provides an aluminum alloy surface oxidation spot defect identification method and device based on machine vision, and relates to the field of intelligent manufacturing and industrial automation, and the method comprises the steps: obtaining an aluminum alloy surface color image, and carrying out the preprocessing of the image, so as to extract a brightness component image; self-adaptive threshold segmentation of local contrast enhancement is carried out on the brightness component image, a defect area binary mask is generated, morphological connected domains are extracted according to the mask, and three basic feature indexes of the area pixel value, the contour Fourier descriptor complexity and the area gray scale standard deviation contrast of each connected domain are calculated; and extracting and marking a connected domain boundary, verifying a boundary closed topological structure, and dynamically generating a curvature-driven self-adaptive sampling point through multi-scale B-spline curvature extreme value detection. Through optical-algorithm-process three-level collaborative innovation, the curved surface reflection false alarm rate is reduced, the pinhole detection rate is increased, and the boundary precision is + / -0.2 pixel.

The invention provides a self-adaptive nonlinear image enhancement method and system for a low-light scene of a mobile terminal, and relates to the technical field of image enhancement, and the method comprises the steps: carrying out the image preprocessing and noise reduction, and carrying out the graying and noise suppression of an input color image through a local variance self-adaptive algorithm; adaptive down-sampling is carried out, and the down-sampling proportion is dynamically adjusted according to the image resolution and the content complexity, so that the processing efficiency is improved; brightness adaptive enhancement is carried out, and the overall brightness of the image is rapidly improved by adopting an Otsu method and a lookup table; contrast nonlinear enhancement: enhancing image details and contrast in combination with a Laplace operator and local mean adjustment; and color restoration: restoring the resolution through bilinear interpolation and performing weighted fusion to realize natural color reconstruction. And finally, a high-quality image of which the brightness, the contrast ratio and the color are remarkably improved is output. According to the invention, the recognition accuracy and processing efficiency of the low-illumination image are improved.

The invention belongs to the technical field of image analysis, and discloses an old people health state prediction and alarm method based on AI image analysis, and the method comprises the steps: obtaining a depth image, a color image and a pressure image of an activity environment of old people, carrying out the combined analysis of the depth image and the pressure image, recognizing a terrain semantic region in a monitoring region, and carrying out the prediction of the health state of the old people; and carrying out dynamic segmentation on the gait contour of the old people. And the terrain gait coupling degree is calculated by analyzing the depth change characteristics and the motion period characteristics of the gait sub-regions, so that the abnormal gait sub-regions are screened out. And for the abnormal regions, the terrain gait coupling degree, the dynamic texture features of the color image and the pressure distribution features of the pressure image are fused, and a cross-modal health risk vector is constructed. Based on the vector, the health state of the old people is accurately predicted, and a corresponding alarm signal is generated. According to the invention, precise evaluation and timely early warning of the health state of the old people are realized, and intelligent guarantee is provided for the home safety of the old people.

The invention discloses a mechanical arm autonomous grabbing system based on a visual language model and a control method of the mechanical arm autonomous grabbing system. The method comprises the steps that firstly, a mechanical arm is initialized, and a color image and a depth image of a working interval are collected; secondly, based on the color image and the natural language description, generating a target object bounding box and confidence, and judging whether a target object exists in the scene or not; and when the target object does not exist, instance segmentation and capture feasibility detection are carried out on the visible object in the current scene. And then the depth image and the object binary mask passing through grabbing feasibility detection are sent to an interaction object selection strategy and an action selection strategy to obtain an object to be interacted and a pushing action direction. And finally, center coordinates of the to-be-interacted object and center coordinates of the mechanical arm are obtained, the obstacle is moved in the pushing action direction until the target object appears and can be grabbed, and the mechanical arm grabs the target object and places the target object at a designated position. According to the method, the object specified by the human natural language can be accurately recognized and grabbed.

The invention belongs to the technical field of face recognition, and particularly relates to a face recognition algorithm adaptive to illumination variation, which uses Gaussian filtering to reduce noise caused by illumination variation, convert a color image into a grey-scale image, reduce calculation complexity, perform adaptive histogram equalization, estimate global illumination conditions in the image and improve the face recognition accuracy. Calculating the main direction and intensity of illumination in the image, predicting the current illumination condition, and enabling the image brightness to adapt to different environment illumination; identifying a face region in the image, using a deep learning method to position face related points, using an LBP to extract local features insensitive to illumination variation, and using a deep convolutional neural network to extract global features; introducing a multi-illumination data set; the Euclidean distance is used for comparing the extracted feature vectors, the similarity between the extracted feature vectors and known faces in a database is recognized, according to the similarity score, threshold judgment is adopted for AQW to obtain a final recognition decision, and the method has the effect of being capable of accurately adapting to facial features under different illumination conditions in real time.

The invention discloses a tunnel detection method based on a color image, and relates to the technical field of tunnel detection, and the method comprises the following steps: obtaining color image data in a tunnel, and carrying out the illumination condition calibration of the image data, so as to recognize the illumination change caused by natural light, an artificial light source and a light source fault. According to the method, through illumination calibration, adaptive color correction and dynamic contrast enhancement technologies, the problems of image color distortion and detail loss under the complex illumination condition of the tunnel are solved, accurate extraction of color features and clear presentation of details of each region are ensured, and the robustness and precision of defect detection are remarkably improved. In combination with edge detection, color feature extraction, shape analysis and a machine learning algorithm, automatic application of a multi-modal identification technology is realized, misjudgment areas are automatically filtered, a defect detection report with high confidence is generated, and the defect detection report comprises crack length, leakage area and deformation classification. Accurate and comprehensive data support is provided for tunnel safety assessment and maintenance decision.

The invention belongs to the technical field of three-dimensional reconstruction, and relates to an inspection scene multi-view three-dimensional reconstruction method based on adaptive feature enhancement, which comprises the following steps of: (1) constructing a depth map fusion framework based on laser radar and monocular camera self-calibration, obtaining an inspection scene color image and depth map data, the method comprises the following steps: (1) constructing a routing inspection scene multi-view three-dimensional reconstruction network based on adaptive feature enhancement, (3) carrying out transfer learning fine tuning on the routing inspection scene multi-view three-dimensional reconstruction network, and (4) dividing a routing inspection scene data set and configuring an experimental environment, and carrying out network model training and testing. According to the multi-view three-dimensional reconstruction network provided by the invention, the integrality index on a public data set is superior to that of other network models in a pre-training stage, and each evaluation index on an inspection scene data set is lower than values of a reference network and before fine adjustment in a transfer learning fine adjustment stage, so that the network is more suitable for inspection scene reconstruction through fine adjustment; and an ideal effect is achieved.

The invention relates to the technical field of foamed silicone rubber surface detection, and discloses a foamed silicone rubber surface detection method based on image visual identification, which comprises the following steps: acquiring a color image of a foamed silicone rubber surface, carrying out graying and normalization processing, and generating a diffuse reflection effective detection area by using a polarization filter difference technology; color abnormity is detected based on an image gridding and gray statistical method, meanwhile, pore and microbubble defects are detected in combination with minimum value seed point extraction and an eight-neighborhood region growing algorithm, finally, various masks are fused, the defect area and number are counted, and a quality inspection report is generated; the method can effectively inhibit highlight interference and accurately identify uneven colors and pore microbubbles, the detection process does not need external training, calculation is efficient, and the method is suitable for rapid detection and quality control of surface defects of foamed silicone rubber.

The invention provides a depth map generation method and device based on a large model, a three-dimensional reconstruction method and device, electronic equipment and a storage medium, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, can be applied to real-time road scene depth perception, environment three-dimensional reconstruction and obstacle avoidance, and can be applied to real-time road scene depth perception. And virtual and real scene fusion and other scenes can be realized. The specific implementation scheme is as follows: performing visual coding on a monocular image to obtain a coded image; inputting the coded image and the target text into a pre-trained large language model for fusion to obtain fusion features; generating global guide features based on the fusion features, wherein the global guide features comprise joint semantic information of visual features and text features; adding noise to the color image of the monocular image to obtain a noise feature sequence; de-noising the noise feature sequence under the condition of the global guide feature, and generating an implicit feature matched with the joint semantic information; a depth map is generated based on the implicit features.

The invention discloses a thyroid intraoperative real-time navigation method and a thyroid intraoperative real-time navigation system based on multi-mode optical fusion. The thyroid intraoperative real-time navigation method comprises the following steps: outputting visible light through an endoscope and coupling light waves of a narrow-band multispectral light source to irradiate an operative field, exciting parathyroid glands to generate near-infrared fluorescence, and receiving reflected visible light, split light, near-infrared fluorescence and laser speckle signals through an endoscope probe. And separating the composite optical signal into four channels, and respectively generating an anatomical structure color image, a blood vessel spectroscopic image, a parathyroid gland near-infrared fluorescence image and a laser speckle image. Performing decorrelation processing on the laser speckle image to generate a blood flow dynamic pseudo-color decorrelation speckle image; an anatomical structure, a blood vessel center line, a parathyroid gland contour and blood flow dynamic feature points are extracted through a multi-modal registration technology, after affine transformation space alignment is conducted, a comprehensive imaging map containing the anatomical structure, blood vessel distribution and parathyroid gland function marking information is generated through a wavelet fusion algorithm, and intraoperative multi-dimensional real-time tissue navigation is achieved.

The invention discloses a multi-mode teenager idiopathic scoliosis screening method based on a back RGB-D image, and belongs to the technical field of image processing and scoliosis screening, and the method specifically comprises the steps: collecting an RGB-D image pair of a subject in an Adams anteflexion posture, carrying out the manual marking of an RGB color image, constructing and training a back key point detection model, and carrying out the detection of the back key point. And establishing a calculation rule of a maximum trunk rotation angle, a minimum trunk rotation angle and a back contour asymmetry index, and establishing an AIS classification rule based on the RGB-D data set, thereby completing AIS classification of the to-be-detected subject. According to the non-contact radiation-free AIS screening method, comprehensive utilization of back two-dimensional and three-dimensional information is achieved by means of data complementation of the RGB mode and the D mode, the problem of feature deficiency in a single mode is solved, and the method has the advantages of being fast, efficient, robust and objective.

The invention provides a commodity identification method and device, and the method comprises the steps: carrying out the multi-view collection and preprocessing, and constructing a second color image set and a depth image set; performing three-dimensional reconstruction and volume estimation based on the second color image set and the depth image set, and judging a commodity packaging form; extracting a commodity packaging semantic and price information area based on the second color image set, constructing a space-semantic joint code, and generating global packaging semantic features; based on the second color image set, a multi-view visual coding network is adopted, and a visual prediction SKU index is output; carrying out structure-semantic fusion modeling based on multi-source heterogeneous features, and outputting commodity category labels; and matching a commodity price based on the commodity category label, generating an interaction prompt sign in combination with a confidence threshold, and triggering interaction feedback. According to the invention, commodity vision, space and price activity information are fused, the identification accuracy and price verification intelligence are improved, and the method is suitable for the field of intelligent cashier and retail automation.

The invention relates to the technical field of computer vision and three-dimensional space modeling, and discloses a real-time multi-instance segmentation method and device based on a Gaussian splash radiation field model. The method comprises the following steps: based on a two-dimensional Gaussian splash radiation field model, rendering a visual angle with continuous spatial change to obtain an image sequence, and obtaining a multi-visual-angle consistent two-dimensional instance segmentation mask of the image sequence; and assigning an instance tag to each Gaussian primitive based on the two-dimensional instance segmentation mask. And for a two-dimensional Gaussian splash radiation field model with an instance label, acquiring a coarse instance segmentation mask and a color image at any view angle by using a Gaussian splash algorithm, inputting the mask and the image into a lightweight post-processing network for edge detection and region connectivity repair, and outputting a target two-dimensional instance segmentation mask with a complete structure and a label consistent with a three-dimensional scene. The method does not depend on any training or distillation process, directly acts on a Gaussian splash radiation field model, and has the advantages of high reasoning speed, high semantic consistency, support of multi-target continuous tracking and the like.

The embodiment of the invention belongs to the technology of display equipment, and provides liquid crystal display equipment based on field sequential display and a color image display method thereof.The liquid crystal display equipment comprises a liquid crystal display screen, a backlight module and a processor, the liquid crystal display screen is divided into a plurality of backlight partitions used for displaying images, and the backlight module is used for emitting backlight; the backlight is used for adjusting the backlight brightness of the display image corresponding to the four-field sequence. And the processor is used for acquiring RGBW image pixel data of the image in the backlight partition, and acquiring backlight brightness data corresponding to the RGBW color based on a preset backlight extraction algorithm according to the RGBW image pixel data. And the processor is also used for determining image compensation pixel data corresponding to the RGBW color based on a preset pixel compensation algorithm and the backlight brightness data corresponding to the RGBW color. And the liquid crystal display screen is used for displaying a color image according to the image compensation pixel data corresponding to the RGBW color. The power consumption of the display equipment is reduced, and the user experience is improved.

The invention belongs to the field of agricultural information perception, and relates to a plant leaf scab segmentation method and system based on RGB-D cross-modal fusion, and the method comprises the steps: collecting a color image and a depth image of a crop through a synchronous collection device; registering the color image and the depth image to obtain a registered color image and a corresponding registered depth image; constructing an initial segmentation network, and performing model training on the initial segmentation network; the initial segmentation network comprises a cross attention-based feature aggregation module and an attention-guided adaptive feature fusion module; inputting the registered color image and the registered depth image into a trained segmentation network for segmentation to obtain a segmentation result; through heterogeneous data fusion of a depth sensor and a visible light camera, a multi-dimensional feature system covering two-dimensional textures and three-dimensional deformation is constructed; in combination with a cross-modal feature complementation mechanism, the recognition robustness of weakly dominant diseases and insect pests is enhanced, and the segmentation precision in a complex illumination and branch and leaf shielding scene is significantly improved.

The invention relates to a stereoscopic warehouse checking method and system based on an unmanned aerial vehicle, and the method comprises the steps: receiving a cargo checking task, and obtaining the layout information of a stereoscopic warehouse; determining the position of a to-be-checked goods location in the goods checking task in the stereoscopic warehouse; according to the layout information of the stereoscopic warehouse and the position of the to-be-checked goods location in the stereoscopic warehouse, determining a safe flight area and a hovering point of the unmanned aerial vehicle based on a path planning algorithm, and generating a flight task; issuing a flight task to the unmanned aerial vehicle, and calculating pose information of the unmanned aerial vehicle during execution of the flight task in real time through an inertial measurement unit; based on the observation image of the unmanned aerial vehicle visual system, correcting the accumulated error of the pose information through a Kalman filtering algorithm; rGB color images and depth images collected by the unmanned aerial vehicle at the to-be-checked goods locations are obtained in real time; determining whether the box stack is in an abnormal condition according to the depth image; and for the box stack which is not in the abnormal working condition, the number of goods in the box stack is determined according to the RGB color image.

The invention discloses a warehouse in-out rechecking management system based on automatic visual scanning, which relates to the technical field of warehouse rechecking management, and comprises a pose acquisition unit, a visual calibration unit, a cargo reading unit, a record management unit, a warehouse-in rechecking unit and a warehouse-out rechecking unit, the pose acquisition unit is used for acquiring a color image and a depth image of a target cargo in a to-be-identified area so as to obtain pose information of the cargo, and the visual calibration unit is used for obtaining a relation matrix of a camera coordinate system and a mechanical arm coordinate system by calculating a rotation matrix and a translation matrix. And space coordinates in an image collected by the camera are converted into a mechanical arm coordinate system. The automatic visual scanning and mechanical arm technology are combined, the intelligent level in the warehouse management process can be greatly improved, the method is more suitable for actual complex warehouse scenes, the cargo warehouse-in and warehouse-out speed can be increased, and the cargo checking accuracy can also be improved.

An image processing method, performed by a computer device, comprising: extracting sketch texture features at multiple scales from a sketch image; extracting image noise features at multiple scales from preset noise; determining color guide information corresponding to the sketch image; encoding, for each scale, a noise feature based on a sketch texture feature and the color guide information to obtain multi-scale image features; and performing multi-scale decoding on these image features to obtain a colored image comprising a sketch texture corresponding to the sketch image and a color based on the color guide information. A related training method and apparatus are also provided to develop models for this image processing technique.

The invention relates to the technical field of automatic spraying, and particularly discloses a spraying system based on multi-modal vision and artificial intelligence self-correction, comprising a multi-modal vision acquisition module for acquiring color image information, depth information and infrared feature information; the control module is used for generating a spraying sensing model according to the information of the multi-modal visual acquisition module; the control module carries out spraying area identification, track planning and spraying parameter decision making based on the spraying sensing model; the spraying execution module is used for spraying; and the feedback self-correction module iterates the spraying parameter decision in the control module based on a reinforcement learning algorithm according to the difference between the actual coating state information and the expected state. The control method based on multi-modal vision and artificial intelligence self-correction is applied to a spraying system based on multi-modal vision and artificial intelligence self-correction. The scheme is used for solving the problems that the spraying quality of complex workpieces is not high due to the single sensing dimension of an existing spraying system, and the production flexibility is poor due to the rigid control mode.

The invention discloses an electronic cabin assembly quality detection method based on three-dimensional vision. The method comprises the following steps: firstly, generating data for comparison during detection for a standard electronic cabin; shooting all to-be-detected objects in the to-be-detected electronic cabin to obtain a depth image and a color image of each to-be-detected object; according to the depth image and the color image of the to-be-detected object, combining with the data of the standard electronic cabin for comparison during detection; and whether screws of the electronic cabin to be detected are neglected and not installed in place, whether cable plugs are neglected and not installed in place, whether connectors and cable plugs in the wiring unit are wrongly matched, and whether the bending radius of cables is too small are detected in sequence. Through fusion of depth information, three-dimensional quantitative detection of the assembly quality is realized, and the detection precision, reliability and automation level are significantly improved.

The invention discloses a plant three-dimensional reconstruction method and a related device, and relates to the technical field of computer vision, the method executes multiple iterations to update a Gaussian ellipsoid set, when a color image under each view angle is rendered in each iteration, the depth value of each key point is obtained based on a depth image, and the color image at each view angle is rendered. The depth weight of the target Gaussian ellipsoid is obtained based on the depth value, the depth weight is inversely correlated with the depth value, that is, the contribution of the Gaussian ellipsoid with the small depth value to color rendering is improved, depth information distinguishing of a fine region is enhanced by introducing the depth weight, the contribution of Gaussian ellipsoids of different depth levels to rendering dyeing is accurately controlled, and the color rendering efficiency is improved. Therefore, the geometric accuracy and layering sense of three-dimensional reconstruction of the plant are improved, and the reduction degree of the three-dimensional reconstruction model to the plant is further improved.

The invention provides a real-time human body detection method and system based on a microwave radar, and is applied to the field of signal data processing. By constructing a sparse reconstruction model and combining multi-dimensional feature decomposition, the method effectively relieves the recognition difficulty caused by target signal aliasing, continuously outputs and receives radar signals, extracts distance, Doppler and angle features, constructs target distribution point clouds in a three-dimensional feature space, further utilizes angle continuity judgment and point cloud density analysis, and improves the recognition accuracy of target distribution point clouds. According to the method, the number and distribution of overlapped targets are accurately estimated, a Kalman filter is introduced for dynamic tracking for a scene in which a continuous area is not formed, continuous separation and pseudo-color image reconstruction of multiple targets are realized, and thus the resolution precision and target positioning capability of a radar in a multi-person close-range aggregation scene are remarkably improved.

The invention provides a diabetic retinopathy grading method based on a vision-language pre-training model and a ranking perception prompt strategy. According to the method, an eye ground color image and a corresponding DR grading label are converted into image features and text embedding through an image and text dual-coding mode, matching is carried out through the similarity between the image and the text, and accurate DR grading is achieved. In order to solve the problems of class imbalance and natural sequence modeling, a ranking perception prompt module and a similarity matrix smoothing module are introduced, ranking perception prompt ensures that the model can learn and maintain the sequence relation between different DR levels, and an SMS module effectively relieves the influence caused by data imbalance. The method disclosed by the invention can show relatively high accuracy and robustness in various DR grading tasks, especially under the condition of few samples or unbalanced categories. The method not only improves the grading accuracy, but also provides an effective auxiliary tool for clinical diagnosis, and has a wide application prospect.

The invention discloses a color printed matter defect automatic detection system and method based on machine vision, and relates to the technical field of computer vision, and the method comprises the steps: converting a color image into a gray level image, generating a boundary enhanced gray level image through a Canny edge detection algorithm, calculating the offset of edge pixels through sub-pixel recovery, and obtaining a color printed matter defect detection result. Extracting an edge pixel proportion based on the sub-region division table, performing amplification tracking on edge dense sub-regions, and analyzing RGB color deviation; through a multi-scale Gaussian pyramid decomposition and edge refinement technology, analyzing multi-scale features of the boundary enhanced grayscale image and optimizing edge precision; and classifying edge defects and color defects of each sub-region by utilizing a defect threshold value, calculating a comprehensive risk value, increasing a risk level according to a boundary condition, and generating a detection report. According to the method, the Canny edge detection algorithm is combined with sub-pixel recovery, so that the edge extraction precision and the positioning capability of color printed matter defect detection are improved.

The invention discloses an uncertainty mapping method and device based on selective learnable depreciation, and the method comprises the steps: S1, collecting color images and depth maps at different time points, and projecting the color images and the depth maps to a three-dimensional coordinate system; s2, predicting an evidence vector for each pixel by using an evidence generation network; s3, calculating total evidence intensity based on the evidence vector, and defining credibility and uncertainty based on the total evidence intensity; s4, constructing a noise mask to represent the observation quality, and predicting a damage coefficient by using a selective damage network; s5, the evidence intensity is adjusted in a zooming and depreciation mode; and S6, outputting a final probability, carrying out basic probability distribution of multi-frame fusion observation on the three-dimensional voxel grid, and constructing a semantic map containing uncertainty estimation. According to the method, the conflict rate and uncertainty in the semantic map fusion process can be effectively reduced, and the accuracy and credibility of the map are remarkably improved.

The invention discloses a robot autonomous disassembly method and system based on multi-source visual perception, and relates to the technical field of biological pharmacy, and the method comprises the following steps: obtaining initial image data of a target bagged product, including a color image collected by an RGB camera and a depth image collected by a depth camera, and respectively marking collection timestamps; synchronous alignment of image data is carried out based on timestamps, and spatial position information of different source images at the same moment is matched by constructing a unified time coordinate system. Through integration of technologies such as multi-source visual image synchronization, three-dimensional modeling and track control, residue detection and the like, high-precision autonomous disassembly of the biopharmaceutical bagged product by the robot is realized, the problem of misoperation caused by image time sequence asynchronization is solved, the system stability and the cleanliness control capability are improved, and good application value is achieved.

The invention relates to a target detection method and system based on improved YOLOv11, and aims to improve the grabbing precision and operation efficiency of a mechanical arm in a complex environment. The method combines an augmented reality technology, a target detection algorithm, a depth sensing technology and an advanced motion control system, and specifically comprises the following steps that an RGB-D camera is used for collecting data; establishing a three-dimensional space model, and performing accurate alignment processing on the depth image and the color image; based on a mercuric chloride Atlas 200I DK A2 development board, in combination with an improved YOLOv11 model, efficient detection and identification of a target are realized; a target is converted from a pixel coordinate system to a camera coordinate system through a coordinate conversion method, and then the target is further mapped to the three-dimensional space position of a mechanical arm base coordinate system; finally, grabbing operation is completed through the elephant mechanical arm MyCobot 280M5. By combining RGB-D information, a deep learning detection algorithm and a three-dimensional coordinate conversion method, efficient and accurate target grabbing of the mechanical arm under the complex environment is achieved through the system, and the system has the advantages of being high in precision and real-time performance, easy to maintain, easy to integrate and deploy and the like.

The invention discloses an AI-based child story video generation method and system, and relates to the technical field of artificial intelligence and multimedia crossing, and the method comprises the steps: generating a script from an original text input by a user through constructing an AI model fusing an emotion modeling capability; constructing an image generation combination model, defining a joint loss function, calculating an edge intensity graph of the contour image by using a Sobel edge detection algorithm, calculating an optical flow field of frame change by using a block matching algorithm, and performing color image dynamic frame alignment; a fine tuning WaveNet model is used to generate audio; through constructing an image generation combination model, combining a StyleGAN3-T model and an LDM model, defining a joint loss function, and using a Sobel edge detection algorithm and a block matching algorithm to calculate an edge intensity graph and an optical flow field, dynamic frame alignment of a color image is realized, and inter-frame continuity of a generated video is improved.

The invention discloses a LiDAR-IMU-camera tight coupling positioning and mapping method and device for a mobile platform, and belongs to the technical field of robot SLAM. Synchronously triggering the color camera, the laser radar and the IMU through hardware, and unifying timestamps; performing compensation and distortion removal on the laser point cloud motion by the IMU data; based on curvature, intensity and density self-adaptive downsampling, local plane fitting errors are used for distributing observation weights; the IMU pre-integration pose is used as an initial value, point-to-surface registration of the weighted point cloud and the local map is carried out, and a laser-IMU tight coupling odometer factor is obtained; the color image and the laser intensity graph are fused into a multi-mode loopback descriptor, and loopback factors are generated through global retrieval and geometric verification; and inputting an odometer, an IMU, a laser loopback factor and a visual loopback factor into an increment factor graph optimizer, jointly solving a global optimal key frame pose, and outputting a dense laser point cloud map and a color visual point cloud map. The method can be operated in real time on an embedded platform, effectively inhibits drifting, and realizes centimeter-level global consistent positioning and mapping.