12397 results about "Image generation" patented technology

A system and method are disclosed for generating hyperspectral images from multi-modal sensor data including RGB, LiDAR, thermal, and near-infrared inputs. Training data includes hyperspectral images and corresponding multi-modal measurements. Spectral band grouping is performed based on correlation coefficients. A multi-modal decomposition network with cross-modal attention mechanisms generate reconstructed hyperspectral images by fusing complementary sensor information. A fine-tuning network creates reconstructed RGB images. A comprehensive quality assurance system analyzes spectral consistency, cross-modal coherence, and fusion artifacts to generate quality metrics. Missing data compensation strategies handle corrupted sensor inputs using information from other modalities. The system includes temporal integration for video sequences and multi-resolution processing for different sensor resolutions. Quality metrics guide network weight adjustments to improve reconstruction accuracy while maintaining robustness to sensor failures and environmental variations.

The method can include: recording a neural state, determining a latent array, and determining an output based on the latent array. In variants, the method can function to enable a subject to use neural signals to navigate within a high-dimensional concept space to control an output (e.g., image generation, motor control, communication, etc.).

The invention discloses an intelligent multi-modal virtual digital human interaction system based on an AI language large model. The system comprises a high-authenticity face generation module; the high-authenticity face generation module uses an AdaAN network, based on adaptive feature fusion and voice driving and time sequence modeling of voice features, feature information related to voice is extracted, the extracted voice features are processed through a deep neural network, it is ensured that the voice and facial expressions are highly aligned in time and space, and the face recognition accuracy is improved. Collecting a bio-electricity signal, mapping the signal to facial muscle movement, generating a final facial expression, and interacting with a user; the system further comprises an intelligent interaction module, a training optimization and efficient generation module, an efficient integration module, a multi-modal data acquisition module, an AI large model core processing module, a digital human image generation and driving module, an interaction scene adaptation module and a feedback optimization module. The invention further discloses a multi-mode digital human interaction method which has wide application value.

The invention provides a scene reconstruction method based on delayed rendering and three-dimensional Gaussian. The method comprises the following steps: S1, generating initial three-dimensional point cloud data based on a multi-view image; s2, constructing a trainable structural body for three-dimensional Gaussian modeling; s3, normal initialization and residual optimization are carried out on the Gaussian ellipsoid primitives, depth consistency constraint is combined, and a differentiable and learnable normal reconstruction mechanism is realized, so that the geometric expression ability of illumination modeling is enhanced; s4, introducing a reflection training mechanism based on ambient light and a reflection direction, and generating a Gaussian attribute based on a visual angle; and S5, a final image is generated through a differentiable Gaussian sputtering rendering algorithm, and optimization is carried out through pixel loss of the final image and a real image. According to the method, the reality sense and geometric consistency of the Gaussian sputtering model under the complex illumination condition are remarkably improved, and the technical problems of unreal rendering effect, inaccurate surface normal estimation, weak propagation capability and the like of the existing three-dimensional Gaussian sputtering model under the complex illumination condition are solved.

The method can include: recording a neural state, determining a latent array, and determining an output based on the latent array. In variants, the method can function to enable a subject to use neural signals to navigate within a high-dimensional concept space to control an output (e.g., image generation, motor control, communication, etc.).

The invention discloses a multi-modal semantic and physical law driven remote sensing image generation method, belongs to the technical field of computer vision and remote sensing image generation, and aims to solve the problems of insufficient cross-modal semantic alignment, low reliability of a generation result and insufficient physical mechanism fusion. The four-stage method comprises the following steps: firstly, rejecting low-quality samples from original data and unifying a spatial scale; then, extracting a multi-modal semantic vector by adopting a BLIP model and a CLIP model, and introducing a remote sensing physical rule to carry out vector optimization; then position coding and physical constraint conditions are embedded in the submerged space, and multi-source information joint modeling is achieved through a cross-modal encoder; and finally, by taking text description, physical priori knowledge and diffusion time steps as joint conditions, performing de-noising reasoning based on a Transform architecture, and completing back diffusion reconstruction by means of a trans-attention mechanism. According to the method, physical rationality and semantic consistency are improved, and a more reliable technical normal form is provided for remote sensing image generation in the fields of disaster monitoring, military simulation and the like.

The invention discloses a construction progress dynamic optimization method and system based on BIM and computer vision, and particularly relates to the technical field of building construction management, and the method comprises the steps: carrying out the automatic registration of a BIM model and a construction site image; processing the construction site image by adopting a visual identification algorithm to generate a visual identification result; constructing a four-dimensional dynamic BIM model, and mapping a visual identification result to a corresponding component in real time through multi-feature similarity calculation; the progress deviation is monitored by using key path dynamic identification and a deviation propagation matrix, and the risk is predicted by combining a Bayesian network and Monte Carlo simulation. The BIM and computer vision technologies are fused, a construction progress optimization system integrating automatic registration, dynamic monitoring, risk prediction and intelligent decision making is constructed, and the problems that traditional manual inspection data collection is low in efficiency, progress monitoring is lagged, risk prejudgment is fuzzy and resource allocation is extensive are solved; accurate monitoring, risk early warning and resource optimization configuration of the construction progress are realized.

The invention relates to the technical field of industrial component detection, in particular to a transformer iron core detection method based on computer vision, which comprises the following steps of: acquiring an iron core image, extracting key pixel characteristics, screening a directional scattering abnormal region to generate an interference map, extracting a consistent gradient region correction image to generate a reconstruction map, and positioning a symmetric disturbance generation structure map by integral gray difference. And analyzing an overlapping relation by a superposition structure graph to generate an abnormal component graph, and evaluating a risk level by matching a reference index to generate an early warning graph layer. Interference reflection and structural features can be distinguished through linkage analysis of the pixel direction vector and the brightness change frequency, correction of a distorted area in an image is realized based on a gray statistical stable value, and the distortion of the image is corrected by constructing a symmetric point map and analyzing the change trend of a gradient difference value sequence. And the structural overlapping relation is quantitatively judged by combining a component mapping profile diagram, so that the relevance between an abnormal region and a key component is clearly expressed, and the grading evaluation capability of various fault risks in the iron core is improved.

The invention discloses a remote sensing image generation method based on a federal visual language model, which belongs to the technical field of machine learning and specifically comprises the following steps: receiving text instruction description by each client; extracting a multi-scale feature map from private remote sensing image data through a visual encoder, and generating a semantic embedding vector by text instruction description through a language encoder; inputting the semantic embedding vector and the multi-scale feature map into a dynamic attention mask generator to generate pixel-level space weight distribution; carrying out weighted fusion operation on the multi-scale feature map, and generating visual feature representation of text conditionalization; generating a remote sensing image according with the description of the text instruction through an image decoder; the client uploads model parameter increments of the visual encoder, the language encoder and the dynamic attention mask generator to the central server; the central server aggregates the model parameter increments, and distributes the updated global model parameters to each client; according to the method, the flexibility and semantic consistency of remote sensing image generation are effectively improved.

A data processing system implements receiving, via a user interface of a client device, images for generating a collage image; generating captions for the images; constructing a first prompt by appending the captions to a first instruction string including instructions to a generative language model to extract a theme from the captions; providing the first prompt to the generative language model and receiving the theme therefrom; constructing a second prompt by appending the theme to a second instruction string including instructions to a text-to-image model to use the theme to create a background image with placeholders; providing the second prompt to the text-to-image model and receiving the background image therefrom; identifying the placeholders in the background image; creating the collage image by fitting the images into the identified placeholders; providing the collage image to the client device; and causing the user interface to display the collage image.

The invention relates to an infrared and visible light image fusion method based on a cross-domain Transform, and belongs to the field of computer image processing. The method comprises the following steps: respectively carrying out preprocessing operation on an infrared image and a visible light image to obtain a training data set; an end-to-end image generator network is designed, an encoder module is used for extracting deep semantic features of an infrared image and a visible light image, a fusion module introduces an axial attention mechanism to enhance the global modeling capability of the features, and feature fusion is carried out in combination with information of a spatial domain and a frequency domain; the fused features are gradually recovered to an image space through a decoder module, and a fused image is generated; constructing a fusion loss function module, and guiding the network to focus a significant feature difference between the source image and the fusion image based on a comparative learning idea; and finally, inputting the infrared and visible light image Y channel into the network model, generating a fusion image, completing a training process, and realizing unified optimization of fusion performance and visual quality.

The invention provides a lightweight real-time semantic segmentation method for a three-dimensional Gaussian scene, and belongs to the field of computer vision and three-dimensional scene understanding. The process comprises the following steps: S1, data preparation: acquiring multi-modal information data including point cloud, images and poses, and preprocessing the data; s2, carrying out two-dimensional semantic segmentation, and generating a pixel-level semantic segmentation map and a semantic tag for the image by utilizing PP-LiteSeg; s3, performing three-dimensional scene modeling and semantic projection, fusing multi-modal information to complete three-dimensional Gaussian scene reconstruction, and projecting two-dimensional semantics to a Gaussian function to form preliminary Gaussian function semantic features; and S4, semantic optimization and segmentation output: considering image difference, semantic difference and smoothness, establishing a loss function, optimizing semantic features and outputting a segmentation result. According to the method, efficient, real-time and accurate three-dimensional Gaussian semantic segmentation work can be completed, and the method is suitable for scene understanding tasks of light-weight and resource-limited equipment.

The present invention relates to the technical field of cross-modal image signal reconstruction. Disclosed is a cross-modal joint source-channel coding and decoding method adaptable to changeable scenarios. The method comprises: first designing a Transformer encoder-based cross-modal channel coding and decoding optimization solution, so as to achieve the performance improvement and robustness of a channel encoder and a channel decoder; then designing a cross-modal source coding and decoding optimization solution for haptic-to-image generation based on a latent diffusion model, so that under an image signal loss scenario, haptic information is used to guide image generation; and finally, incorporating transfer learning technology, so as to reduce additional training costs caused by a system facing changeable cross-modal communication scenarios such as a changeable channel signal-to-noise ratio and different transmission tasks. Under cross-modal changeable communication scenarios, the joint source-channel coding and decoding method provided in the present invention can solve the problems of the inability of a receiving end to well complete image reconstruction, and additional model training costs caused by changeable channel environments and scenarios.

The invention discloses a method for generating an autonomous inspection orthoimage of an unmanned aerial vehicle, and relates to the technical field of unmanned aerial vehicle surveying and mapping and autonomous navigation. Global optimization of an air route is realized through a path optimization strategy fused by a genetic algorithm in combination with grid characteristics of an inspection area and image parameter constraints, and candidate waypoints are used as nodes; redundant waypoints are screened through leg smoothness factors, route complexity is reduced, a genetic algorithm takes flight height and waypoint spacing as constraints, a fitness function containing flight distance, turning times and overlapping rate is constructed, a global optimal path is found through population iteration, multi-target requirements are optimized and balanced, and it is ensured that the route meets the image acquisition precision requirement and also meets the requirement of image acquisition. The flight distance can be shortened, the turning frequency is reduced, the cruising ability of the unmanned aerial vehicle is adapted, efficient propelling of the inspection task is guaranteed, meanwhile, waypoint coordinates output through simulation directly adapt to a flight control system, and it is guaranteed that actual flight parameters are consistent with planning parameters.

Implementations relate to generating multi-modal response(s) through utilization of large language model(s) (LLM(s)) and other generative model(s). Processor(s) of a system can: receive natural language (NL) based input, generate a multi-modal response that is responsive to the NL based output, and cause the multi-modal response to be rendered. In some implementations, and in generating the multi-modal response, the processor(s) can process, using a LLM, LLM input to generate LLM output, and determine, based on the LLM output, textual content and generative multimedia content for inclusion in the multi-modal response. In some implementations, the generative multimedia content can be generated by another generative model (e.g., an image generator, a video generator, an audio generator, etc.) based on generative multimedia content prompt(s) included in the LLM output and that is indicative of the generative multimedia content. In various implementations, the generative multimedia content can be interleaved between segments of the textual content.

Artificial intelligence-based systems and methods are described for providing personalized skin product recommendations. Natural language data of a user is received using a conversation engine, such as a generative pretrained transformer AI model. The natural language data is input into a synthetic image generation model for generating a digital twin image depicting skin condition(s). A product recommendation model inputs the digital twin image and / or phenotype and / or demographic classifications of the user, and outputs a skin care product recommendation for the user. An image simulation model inputs the product recommendation, the digital twin image and phenotype and / or demographic classifications of the user to output a simulated image of the user having one or more graphical enhancements. The simulated image may be displayed to the user on display screen along with natural language data describing the simulated image as depicted skin as predicted to appear following treatment with the skin care product.

The invention provides a multi-modal image fusion method based on modal self-adaption and modal interaction compensation, and the method comprises the following steps: S1, obtaining a multi-modal image fusion data set, and obtaining a training data set through preprocessing; S2, analyzing the modal difference characteristics of infrared and visible light images, and evaluating the correlation characteristics of image pairs in different scenes; s3, capturing a cross-modal feature dependency relationship through a self-attention mechanism; s4, a differential feature extraction strategy is adopted, model parameters are optimized through iterative training, and multi-modal image fusion is completed; s5, a modal interaction compensation module is additionally arranged, unit dynamic balance common features and modal exclusive features are fused, feature complementation is achieved in channel and space dimensions, parameters of the modal interaction compensation module are optimized, the model is made to learn the optimal fusion weight of the multi-modal features in a self-adaptive mode, and multi-modal fusion image generation optimization is achieved through the model; according to the invention, multi-modal image fusion can be accurately and effectively carried out.

The invention discloses a small sample industrial defect detection system based on a multi-stage diffusion model. The small sample industrial defect detection system comprises a multi-stage diffusion generation module and a defect detection module based on multi-scale attention and physical constraint. The multi-stage diffusion generation module divides the diffusion generation process into three stages of global structure reconstruction, local detail refinement and texture feature synthesis through a stage control mechanism, and gradually guides feature evolution and improves the generation effect aiming at the quality and diversity problems of defect image generation under the small sample condition; the defect detection module based on multi-scale attention and physical constraint adopts a coding structure fusing local window attention and global attention, through multi-scale feature extraction and fusion, significant features of a defect area are effectively captured, gradient smoothing loss and edge energy consistency loss are introduced at a decoder end, and the defect detection accuracy is improved. Therefore, high-quality reconstruction of a normal area and effective suppression of an abnormal area are realized.

The invention provides a video generation control method and a computer readable storage medium, and is applied to the technical field of computers. According to the method, the structural defect of a split script is solved through a large split model, the consistency of split image features is ensured through image generation based on reference image feature constraint, audio and picture synchronization is realized in combination with emotional speech synthesis and dynamic mouth shape alignment, the video fidelity is improved through a split video cue word guiding multi-subject generation algorithm, and the user experience is improved. Therefore, the technical problems of missing emotion logic, poor consistency of split image features, difficulty in audio synchronization, insufficient controllability of video generation and the like in film and television creation in the existing AIGC technology are effectively solved, and the efficiency and the quality of video content production are improved.

According to the Mama and diffusion model-based OCTA fundus image generation system, method, medium and device provided by the invention, the unique Mama block is designed to effectively predict the accurate diffusion noise of the OCTA image from the OCT potential noise, the artifact of the generated image is reduced, and the contrast of the generated OCTA image is improved. Secondly, the proposed bidirectional Bow scanning scheme effectively alleviates the mechanism challenge of original Mama scanning, enhances the spatial continuity of the generated OCTA image, and improves the vascular extension of the generated image. Besides, the understanding ability of the OCT image is improved by combining a Mama block and a diffusion model through designed condition prompt, and the authenticity of the generated fundus OCTA image is further improved. According to experimental verification on two public data sets of OCTA500-3M and OCTA500-6M and one private data set, the SSIMs of 87.5% and 88.1% are achieved respectively, it is indicated that the fundus OCT image information can be fully utilized, the high-quality fundus OCTA image can be effectively generated, and new possibility is provided for low-cost screening of retinal diseases.

The invention relates to the technical field of evaluation and management, in particular to a classroom learning dynamic evaluation and management system, which comprises a multi-source heterogeneous data acquisition module, an acoustic sensor array, an optical motion capture unit, an intelligent writing track acquisition device and a brain wave monitoring terminal, the data preprocessing and feature fusion module is used for eliminating environment interference signals and generating preprocessed data; the dynamic learning state evaluation module is used for extracting three-dimensional features and outputting a dynamic evaluation matrix; the group learning portrait generation module is used for generating a multi-dimensional learning portrait in combination with a preset knowledge graph topological structure; the self-adaptive management strategy generation module is used for dynamically generating a management strategy set; and the intelligent feedback execution module is used for driving the self-adaptive illumination system, the wearable vibration reminding device and the AR enhancement projection equipment to execute physical environment regulation and control. According to the invention, personalized strategy dynamic generation is realized, and adaptive regulation and control of the classroom environment and the teaching rhythm are realized in combination with the intelligent Internet of Things equipment, so that an intelligent collaborative feedback closed loop is formed.

The invention discloses an interactive drawing generation method, system and device based on artificial intelligence, relates to the technical field of artificial intelligence, and realizes the generation of a high-quality image through multi-modal data fusion, semantic alignment, conflict detection and iterative correction. The scheme comprises the following core steps: acquiring multi-modal data such as texts, voices, sketches and the like, and processing the multi-modal data into a formatted feature data set; performing alignment and weighted fusion on the multi-modal features to generate unified semantic space features; detecting semantic inconsistency of the image and text description by using a vision-text attention mechanism, generating a conflict index matrix and triggering an alarm mark; and iteratively correcting the generated image based on the unified semantic space features, the conflict index matrix and the alarm marks. The system is composed of a data acquisition and preprocessing module, a multi-modal feature alignment and fusion module, a conflict detection and alarm module, an image generation and iterative correction module, a system control and scheduling module and the like.

The invention discloses a medical image generation method and device based on bimodal fusion, equipment and a medium, and the method comprises the steps: respectively extracting the visual features of a medical image and the semantic features of text description through an image encoder and a text encoder; mapping the two types of features to a shared semantic space by adopting comparative learning to realize cross-modal alignment; a first path captures hierarchical semantic information through a convolutional neural network, a second path retains local significant features through maximum pooling, and two paths of outputs are fused layer by layer to construct spatial context features; projecting the cross-modal features into a spatial feature map through a multi-layer perceptron, splicing the spatial feature map with coding features, and inputting the spliced spatial feature map into a decoder for up-sampling reconstruction; and performing joint optimization on the comparison loss and the structural similarity loss to realize end-to-end training. And finally, the SSIM index of the generated medical image is improved.

The invention relates to the technical field of industrial product surface quality detection, in particular to a flexible display module surface defect image recognition method, which comprises the following steps: acquiring a plurality of surface images of a flexible display module under different light sources and carrying out distortion removal processing on the surface images; reconstructing and generating three-dimensional reference point cloud data representing the current curved surface form of the module; re-projecting the distorted image to the ideal rigid plane according to the relationship, generating a plurality of corrected images, and generating a plurality of corrected images to eliminate geometric and luminosity distortion introduced by flexible deformation; obtaining a defect candidate area binary image; and extracting a multi-dimensional feature vector of the defect candidate region from the binary image of the defect candidate region, and classifying the feature vector by using a pre-trained defect classification model to obtain a defect identification result. Through the three-dimensional reference point cloud reconstruction and image re-projection technology, the problem of geometric distortion caused by surface deformation of the flexible display module is effectively solved, and misjudgment and missed judgment are avoided.

In various examples, systems and methods are disclosed relating to learning and segmenting objects in lidar point clouds. A system can generate, for an image depicting an environment, a set of segmentation masks using an image segmentation model and the image, and can generate a set of tokens for the image using an image-text encoder model. The system can generate a segmented point cloud by transferring the set of segmentation masks to the point cloud representing at least a portion of the same environment depicted by the image. The system can update, using the segmented point cloud and the set of tokens, a machine-learning model to generate output segmentations for input point clouds.

The invention discloses a full-process AI image creation method and system based on diffusion type image generation, and relates to the field of AI image creation, and the method comprises the steps: obtaining a natural language creation instruction inputted by a user, and analyzing and extracting a main body element data set and a visual style label data set; generating a sequential task sequence including script generation, split mirror design, picture generation and video synthesis; generating a script frame and a split script data set through the large model; calculating the semantic similarity between the split content and the main body element, and performing supplementary optimization; generating a diffusion model cue word and obtaining rendering parameters to generate a rendered picture; the video is synthesized after the visual coverage is verified, and final output is achieved through feedback adjustment; according to the method, a complete control link from creation guiding to generation output can be constructed, and the customized generation requirement in a complex creation scene is met.

The pattern generation method based on diffusion model fine tuning comprises the following steps: constructing a data set; the generation model comprises an edge detection model ControlNet and a stable diffusion model, the stable diffusion model comprises a text encoder CLIP, a diffusion model U-Net and an image decoder VAE, the CLIP text encoder converts an input natural language into word vector features and inputs the word vector features into a U-Net network in a submerged space, and the diffusion model U-Net and the image decoder VAE are connected with the edge detection model ControlNet and the diffusion model U-Net; the U-Net network is controlled to carry out iterative denoising on pure noise in a low dimension to generate a compressed image, and finally the compressed image is restored to an original pixel image through a decoder VAE, so that text content control image generation is realized; carrying out fine tuning training on the generative model by adopting a LoRA parameter efficient fine tuning technology; and intelligently generating a batik style pattern. The method has the advantages that the texture precision of the generated image can be improved, and the personalized style image can be generated.

A method of automated generation of contextually-relevant images for a music segment includes receiving at least one of basic metadata information and lyric information for the music segment, generating a first prompt for a computer-implemented machine-learning language model based on the at least one of the basic metadata information and the lyric information, receiving context information from the computer-implemented machine-learning language model in response to the first prompt, generating a second prompt for the computer-implemented machine-learning language model based on the context information, generating a third prompt by providing the second prompt as an input to the computer-implemented machine-learning language model, and generating an image descriptive of the music segment by providing the third prompt as an input to a computer-implemented machine-learning image generation model.

A method for problem inference based on multi-modal generative artificial intelligence includes receiving question information including an image and text, generating formal languages by parsing the image and text of the question information, respectively, based on a pre-constructed problem solving template, generating text-based intermediate inference information for the question information by inputting the generated formal language to a formal language inference unit, generating image-based inference information by inputting the text-based intermediate inference information, the text included in the question information (hereinafter referred to as “text question information”), and the image included in the question information (hereinafter referred to as “image question information”) to a multi-modal image generation model, and generating text-based inference information by inputting the text-based intermediate inference information, the image-based inference information, and the text question information to a multi-modal text generation model.

The invention relates to a defect picture generation method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining a workpiece image, text prompt information and a mask image, the mask image is generated based on the workpiece image and mask information, and the mask information is used for representing the shape and position of a to-be-generated defect on the workpiece image; the workpiece image, the text prompt information and the mask image are input into a pre-trained defect image generation model, a target defect image is obtained, the defect image generation model is used for extracting a first generation feature from the workpiece image and the text prompt information, extracting a first shape feature from the mask image, and generating a second generation feature; based on the first generation feature and the first shape feature, a target defect image is generated, and the defect generated in the target defect image meets the constraint of the mask information on the defect shape and the defect position. In this way, accurate control over the defect shape and the defect position can be achieved, and the target defect image meets the actual requirement.