141 results about "Perceptual image" patented technology

The invention discloses a target detection method based on Mama feature fusion, and relates to the technical field of image target detection. According to the method, the innovative implementation of the VSSA module is utilized, a selective scanning mechanism of the state space model is applied to 2D visual data processing, the long-distance dependency relationship in the image is effectively captured through state space modeling in four directions, the limitation of a traditional state space model in the two-dimensional visual data processing process is solved through the multi-direction processing strategy, and the processing precision of the 2D visual data is improved. The model can comprehensively perceive spatial dependency relationships in different directions in an image, the VSSA adopts learnable state space parameters to dynamically model a feature sequence, the ability of the network to understand a complex space structure is enhanced, the method is particularly suitable for processing scenes needing long-distance context information, and in addition, the method is combined with MTMHSA, so that the complexity of the network is reduced. And the fusion capability of different levels of features in target detection is further enhanced. Through the innovation, the model can better understand the target in the image, and the positioning and classification precision of the target is improved.

The invention discloses a three-dimensional medical image visual language model pre-training method for reconstructing a hybrid strategy, which belongs to the technical field of medical image calculation and comprises the steps of constructing a medical image text pair data set, a language text mask reconstruction strategy, a visual image mask reconstruction strategy, a semantic perception fusion strategy and multi-task joint learning. According to the method, the large language model is finely adjusted, the diagnosis and attribute information in the medical report is extracted by using the finely-adjusted large language model, the efficient prompt is generated, and the large language model has relatively high generalization ability, so that the cost of manual annotation is greatly saved; according to the semantic perception fusion strategy, the text feature obtained by the text encoder and the image feature obtained by the image encoder are combined to obtain the new text feature, so that the text perceives the diagnosis and attribute information of the image in advance, the alignment of the image and the text in the embedding space is further optimized, and the pre-training efficiency is improved.

The invention relates to the field of artificial intelligence (AI), in particular to an electrical stimulation method, an electrical stimulation device, electrical stimulation equipment and a storage medium. The method comprises the steps of obtaining a to-be-perceived image of a target object, inputting the to-be-perceived image into a trained electrical stimulation model to obtain electrical stimulation parameters output by the electrical stimulation model, and performing corresponding electrical stimulation operation on the target object to recover visual perception of the target object on the to-be-perceived image. An electrical stimulation generator in electrical stimulation is obtained by performing first link training and second link training based on a plurality of sample images and corresponding sample neural signals, the first link represents a verification link among electrical stimulation parameters, analog neural signals and reconstructed images, and the second link represents a verification link among the reconstructed images. The second link represents a verification link among the electrical stimulation parameters, the light magical view image and the sample image, so that the electrical stimulation model can greatly optimize the generation process of the electrical stimulation parameters, and the accuracy and generalization of restoring visual perception through electrical stimulation are improved.

The invention discloses an overwater target multi-dimensional fusion identification method and system, and the method comprises the following steps: S1, collecting perception image data in a water surface environment, and carrying out the preprocessing of the perception image data; s2, carrying out multi-modal fusion; s3, inputting the fused image data into an OfficientDet target detection network, extracting a feature map, and outputting a category label and spatial position information of a water target through a bounding box regression sub-network and a target classification sub-network; s4, constructing time sequence input, inputting the time sequence input to a Motionform network, and extracting motion trail features of the water target; s5, generating an identification report; and S6, performing online updating on the network parameters by adopting an incremental learning mode. According to the method, the OfficitDet network and the Motionform network are fused, accurate detection and dynamic recognition of the water target are achieved, and the method has the advantages of being high in precision, high in stability and capable of achieving self-adaptive optimization.

The invention relates to the technical field of image recognition and classification, in particular to a fine-grained image recognition method and system based on a shared visual backbone network. The method comprises the following steps: acquiring a fine-grained image data set; preprocessing the acquired image data; constructing a deep network model, including performing feature extraction by using the deep network model; object proposing and screening are carried out on the extracted feature map; performing feature pooling and information fusion on the screened features; performing global abstract extraction and gating context fusion on the fused features; projecting an object sequence in the gated and fused features to a feature space of a language model; training and optimizing the constructed deep network model; and performing fine-grained image classification by using the optimized model. According to the method, a parallel local feature extraction path is introduced, and a cross-attention fusion module is designed, so that the model can sense the global context and local discriminative details of the image at the same time.

The invention provides an unmanned aerial vehicle full-time perception image reconstruction method based on multi-mode cooperative reinforcement learning and degeneration decoupling. A frequency perception feature modulation model, a dual-mode dual-domain transformation module and a dynamic bidirectional guide mechanism are included. According to the system, firstly, feature information of different frequency bands is adaptively separated and modulated through a frequency sensing feature modulation model, and decoupling and compensation of composite unknown degradation are achieved; realizing cross-domain interaction and information fusion of visible light and infrared characteristics in a spatial domain and a channel domain by using a bimodal dual-domain transformation module; and finally, realizing collaborative enhancement of cross-modal degradation perception through a bidirectional dynamic guide mechanism, and generating an unmanned aerial vehicle visible light reconstruction image and an infrared super-resolution image with higher structural consistency and texture fidelity. According to the method, deep fusion and degeneration decoupling of multi-modal information can be realized in a complex degeneration environment, and the imaging quality and the environmental adaptability of an unmanned aerial vehicle full-time sensing system are remarkably improved.

Systems and methods for detecting phishing using image hashing include obtaining a plurality of images from different sources, generating a hash for each image, comparing at least one hash associated with a first image to one or more hashes associated with a second image, calculating a similarity score based on the comparing, and classifying the first image based on the similarity score.

The invention provides an efficient image super-resolution method and device based on dual guidance semantic enhancement, and relates to the technical field of image super-resolution, visual content generation and the like in computer image process.The method comprises the steps that an image super-resolution model containing a pre-training diffusion model backbone structure is constructed, and a pre-training diffusion model is constructed; a trainable LoRA module is inserted into a U-Net structure and a VAE encoder of the image super-resolution model; calculating the perception image loss of the generated image and the real image by adopting a dual-guidance quality enhancement training strategy; based on a semantic alignment method of score matching, performing semantic feature constraint on the generated image through a condition vector of a pre-training diffusion model to form semantic consistency loss aligned with real image distribution; and carrying out end-to-end training on the image super-resolution model under joint optimization of perceptual image loss and semantic consistency loss, and after the training is finished, realizing high-resolution generation of a low-resolution image only through one-time U-Net reasoning.

The disclosure relates to systems and methods automatically extracting an image and related image components, computationally determining an understanding of the image, and generating mathematical vector embeddings via sentence encoders based on the computationally determined understanding. The mathematical vector embeddings may be used for semantic image retrieval that enables image searching based on a semantic understanding of input images and / or input text. The mathematical vector embeddings may be used for training and executing generative Artificial Intelligence (AI) models to create new content that includes retrieved images and / or generate new images.

The invention discloses a progressive migration training decoding method based on GPT, and belongs to the field of optic nerve decoding. According to the method, a GPT-based category and text decoding model is constructed based on functional magnetic resonance signal data recorded when a subject watches a large number of visual perception images, and the method comprises an image encoder used for embedding image feature vectors extracted by adopting a pre-training model into a hidden feature space; prompt containing task information and position codes are embedded into a hidden feature space; the visual information encoder is used for mapping a brain visual region voxel signal to a hidden feature space by using a GRU; the whole-brain information encoder is used for guiding the selection of voxel signals of other regions of the brain by taking a voxel signal of a visual region of the brain as a context, and mapping global information to a hidden feature space; and the language generator fuses all the hidden features and selects the Token with the highest probability to generate the text of the corresponding content. According to the method, the decoding accuracy is improved, and the generalization ability of the model is enhanced.

The invention discloses a generative video coding method based on implicit inter-frame alignment, which comprises the following steps of: 1, intelligently modeling space-time characteristics between video frames through a self-adaptive space-time importance encoder, and dynamically allocating code rates; 2, extracting features of a previous frame through an implicit inter-frame alignment strategy, fusing the features with information of a current frame, and converting the information of the previous frame through the structure of the U-Net network to be aligned with the current frame; and step 3, combining the perception loss function with the perception image block similarity constraint. The method has the following three advantages: the generation quality of a key region is optimized through an adaptive space-time importance encoder, and meanwhile, the bit overhead of a secondary region is reduced; an implicit inter-frame alignment strategy is used for capturing an inter-frame potential time-space relationship, so that the calculation complexity of estimating explicit motion information is reduced, and the calculation overhead is reduced; and the perception quality of the video frame is effectively improved through the perception loss function.

The invention discloses a cleaning equipment control method and related equipment, and relates to the field of cleaning equipment control, the cleaning equipment comprises a grabbing mechanical arm, and the method comprises the steps that perception image information of a first target area is acquired; determining whether a to-be-grabbed object exists or not based on the perceived image information; under the condition that the to-be-grabbed object exists, obtaining to-be-analyzed image information of the to-be-grabbed object; determining grabbing parameter information of the to-be-grabbed object based on the to-be-analyzed image information; and based on the grabbing parameter information, a grabbing mechanical arm is controlled to execute grabbing operation on the to-be-grabbed object.

Systems and methods for view-conditioned diffusion for real-world vehicle gaussian splatting. A single perspective image can be transformed using image transformation techniques to generate a training dataset that addresses a domain gap between synthetic data and real-world data in a traffic scene. A pre-trained diffusion model can be finetuned with the training dataset to obtain a fine-tuned diffusion model. Perspective-aware images having different perspective views of an entity from the single perspective image can be generated using the fine-tuned diffusion model. A large generative model (LGM) can be trained using the perspective-aware images to generate a gaussian splatting model for the entity. View-conditioned simulations from the single perspective image can be generated by using the gaussian splatting model for downstream tasks.

The present application provides a method and device for processing rail network information based on real-time state perception images, relating to the field of information processing technology. The method comprises: determining an initial rail network image under normal conditions for all rail transit network information; the initial rail network image represents the attributes, state, and location of rail transit network equipment; and performing pixel-level updates on the initial rail network image based on the real-time state of the rail transit network to obtain a real-time rail network image. The method and device for processing rail network information based on real-time state perception images provided by the present application can provide a real-time and intuitive description of rail transit network information that is difficult to obtain.

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a diffusion neural network for mask aware image and typography editing. For example, in one or more embodiments the disclosed systems utilize a text-image encoder to generate a base image embedding from a base digital image. Moreover, the disclosed systems generate a mask-segmented image by combining a shape mask with the base digital image. In one or more implementations, the disclosed systems utilize noising steps of a diffusion noising model to generate a mask-segmented image noise map from the mask-segmented image. Furthermore, the disclosed systems utilize a diffusion neural network to create a stylized image corresponding to the shape mask from the base image embedding and the mask-segmented image noise map.

The invention provides a multi-task model training method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as automatic driving and the like. The method comprises the following steps: respectively inputting a perception image sample into a feature extraction network in a pre-trained single-task model and a feature extraction network in a to-be-trained multi-task model to obtain a single-task feature and a multi-task feature; one subtask branch of the multi-task model is the same as the task of the single-task model; determining distillation loss between the single-task feature and the multi-task feature based on the task type of the single-task model; based on the distillation loss, performing parameter updating on a feature extraction network of the multi-task model to train the multi-task model; the feature extraction network is shared by each sub-task branch in the multi-task model. According to the invention, the problem of multi-task negative migration is solved, and the performance of the multi-task model is improved.

The invention discloses an equipment identification and quality grade joint evaluation method for a transformer substation inspection image, and the method comprises the steps: inputting a collected original transformer substation image into an image identification and quality grade joint evaluation model; the model comprises a detection perception image enhancement module, a target detection module, an ROI extraction module and an image quality evaluation module which are connected in sequence. Performing enhancement processing on the original transformer substation image by using a detection perception image enhancement module to obtain an enhanced image; performing target detection on the enhanced image by using a target detection module to obtain a power equipment category label and ROI frame coordinate information; an ROI extraction module is utilized to cut the original substation image according to the ROI frame coordinate information to obtain a corresponding equipment ROI image; performing no-reference image quality evaluation on the ROI image of the equipment by using an image quality evaluation module to obtain a quality grade label; according to the method, the equipment identification accuracy is improved, and the image quality grade is credibly output.

The invention provides a semantic and structure perception image restoration method and device based on a diffusion model. The method comprises the steps of obtaining a to-be-processed image and a mask used for identifying a to-be-restored area; performing panoramic segmentation on the to-be-processed image to obtain a semantic feature map, and constructing a peripheral semantic candidate set for the mask boundary to constrain semantic distribution of the to-be-restored region; extracting a structural feature map of the to-be-processed image, wherein the structural feature map comprises at least one of edge information, line segment information and angular point information; and inputting the to-be-processed image, the mask, the semantic feature map and the structural feature map into a multi-conditional diffusion generation network, and generating repair content in the to-be-repaired area by the multi-conditional diffusion generation network under the constraint of the semantic feature map, the structural feature map and the peripheral semantic candidate set, and outputting the restored image.

The invention relates to the technical field of image detection, and provides an image forgery detection method based on a multi-modal large language model, which comprises the following steps: inputting a to-be-detected image into the multi-modal large language model; the multi-modal large language model outputs an authenticity identification result of the to-be-detected image; wherein the multi-modal large language model has three parallel expert branches of physical consistency, semantic consistency and underlying structure clue analysis. According to the method, semantic information in the image is perceived and the context logic and the physical logic of the image are understood by fully utilizing the powerful priori knowledge, the deep semantic understanding capability and the chained reasoning capability of the multi-modal large language model, and meanwhile, complex reasoning and explanation can be performed on a forged image through a natural language.

The invention discloses a ground penetrating radar high-quality imaging method based on an image recovery diffusion model, and the method comprises the steps: constructing a simulation image through an open source simulation tool gprMax based on a finite difference time domain method (FDTD), and constructing a contrast high-frequency clutter-free and low-frequency clutter data set through preprocessing and fusing actual measurement clutter simulation; a double-layer routing attention mechanism is introduced on a basic model to enhance association between features, conditional information is fully utilized, discrete wavelet transform is used to effectively separate low-frequency and high-frequency information, information loss is reduced, and the high-frequency information is enhanced through an adaptive wavelet transform high-frequency enhancement module. According to the method, good improvement is achieved on a synthetic data set, acceptable effects are achieved on multiple evaluation indexes including root mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), perceptual image similarity (LPIPS) and depth image structure and texture similarity (DISTS), and a certain underground target structure can also be recovered in actual measurement data.

According to embodiments of this disclosure, a method and apparatus for data processing are provided. The method includes: determining a reference region corresponding to a target traffic light indicated by map data in an image coordinate system corresponding to a perceived image; determining a set of candidate objects from a plurality of candidate objects identified based on the perceived image, based on the positional relationship between the reference region and a plurality of candidate regions, wherein the plurality of candidate regions correspond to the plurality of candidate objects identified as traffic lights; determining a first correlation between a first attribute of the target traffic light and a second attribute of the set of candidate objects, wherein the first attribute is determined based on the map data and the second attribute is determined based on the perceived image; and matching the target traffic light with the target objects in the set of candidate objects based on the first correlation. Based on this approach, embodiments of this disclosure can more accurately identify and match traffic lights.

The application discloses a three-dimensional medical image visual language model pre-training method of reconstruction mixed strategy, belongs to the technical field of medical image computation, and comprises the following steps: constructing a medical image text pair data set, a language text mask reconstruction strategy, a visual image mask reconstruction strategy, a semantic perception fusion strategy and multi-task joint learning; the large language model is fine-tuned, the fine-tuned large language model is used to extract diagnosis and attribute information in a medical report and generate efficient prompts, the large language model has strong generalization ability, and the cost of manual labeling is greatly saved; the semantic perception fusion strategy is used for combining text features obtained by a text encoder and image features obtained by an image encoder to obtain new text features, so that the text perceives diagnosis and attribute information of the image in advance, the alignment of the image and the text in an embedding space is further optimized, and the pre-training efficiency is improved.

A display system for a motor vehicle includes a projection device having a display device which is configured to output image-bearing light for projecting a display image in an eye area of a user via a display surface. A combiner surface reflects the image-bearing light into the eye area of the user. A device configured to obtain an eye pose of the user transforms a display image with the aid of a post-transformation matrix so that local flaws in a surface section of the combiner surface which result in a distortion or a fuzziness of a perception image perceptible by the user are compensated. A display image is displayed on the display surface using brightness values so that a homogeneous and high contrast representation results based on the eye pose of the user and the detail of the combiner surface. The transformed display image is output via the display device.

The invention relates to the field of vehicle energy management, and discloses a vehicle control method, device and equipment, a medium, a vehicle and a program product, and the method comprises the steps: determining the multi-dimensional features of the vehicle based on the driving data of the vehicle and a perception image; acquiring a torque control strategy matched with the multi-dimensional features; and controlling the output torque of the vehicle according to the torque control strategy. According to the method, the multi-dimensional features can be extracted from the driving data and the sensing image, then the torque control strategy is matched, optimal control over the output torque of the vehicle is achieved, and energy consumption of the vehicle is greatly reduced.

The application relates to a remote sensing image generation online map method, device, equipment and medium, a plurality of different transformed sample images are obtained by performing geometric transformation on remote sensing sample images in each group of sample image pairs, the plurality of transformed sample images are added to the corresponding sample image pairs to obtain a training data set, each training data set is used to train an online map generation model to obtain a trained online map generation model, an encoder in a perception image compression network is used to map remote sensing sample images and corresponding transformed sample images from a pixel space to a feature space, a forward diffusion and reverse denoising are performed on the feature space by a denoising diffusion bridge network, and then an output of the denoising diffusion bridge network is mapped from the feature space to the pixel space by a decoder in the perception image compression network, and a real-time remote sensing image is input into the trained online map generation model to obtain a real-time network map. By adopting the method, a map with clearer boundaries and brighter colors can be generated in real time.

The disclosure generally relates to methods and systems for image co-registration of multi-modal temporal sensing. Conventional techniques for image co-registration of multi-modality images focus on either the spatial or temporal domain and thus are not of high accuracy and do not preserve both global and local characteristics for the matching. The present disclosure solves the technical problems in the art for image co-registration of multi-modal temporal sensing using a deep-learning based multi-input-output encoder-decoder network with a Gabor Jet Model. The deep-learning based multi-input-output encoder-decoder network is utilized for the feature extraction. A distinctive Gabor-jet layer of the Gabor Jet Model is utilized for the similarity matching. The Gabor-jet layer generates a Gabor jet graph which provides sparse feature points for matching between matching images and reference images.

The invention belongs to the field of computer vision, and particularly relates to a multi-mode CNN-Transform fused image tampering detection method, which designs a CNN and Transform double-flow parallel feature extraction structure, effectively combines the advantage of CNN at capturing fine local features and the advantage of Transform in capturing long-distance dependency relationship and global semantic information, and improves the accuracy of image tampering detection. Local details and global tampering features in the image can be sensitively perceived at the same time, so that the detection capability and stability of image tampering are effectively improved. Processing the noise domain image filtered by the SRM through a CNN (Convolutional Neural Network), and capturing local texture and noise artifact features; meanwhile, a Transform branch processes an RGB spatial domain image, extracts rich global semantic information, and realizes deep interaction and advantage complementation of two types of modal information through a feature fusion module. The BAFM module provided by the invention can deeply mine feature information of different scales and different spatial directions, and more accurate spatial attention features are generated, so that the sensitivity and expression ability of the network to tampered regions are improved.

The invention discloses a visual SLAM method and system based on saliency prediction and a storage medium, and belongs to the technical field of synchronous positioning and mapping. Firstly, saliency prediction based on multi-modal fusion is carried out according to a current image frame and a depth frame, a grey-scale map of the current image frame and a saliency mask containing an effective structured region are obtained, and then feature extraction and matching are carried out to obtain matched feature points; then calculating the saliency entropy of the current image frame and judging a key frame, and creating map points for real-time grading to obtain a graded local map; and finally, performing global BA weighted optimization, and constructing a global map by continuously expanding and maintaining the graded local map. According to the method, a significance prediction technology based on multi-modal fusion is introduced, geometric information and depth information are fused to accurately perceive an effective structured region in an image, significant features are captured and accurately matched, the perception and association capability of the system is enhanced, and the stability and overall performance of the system are comprehensively improved.

The invention discloses a multi-task model training method. The method comprises the steps of obtaining a training data set of a plurality of sub-tasks corresponding to a multi-task model; inputting a first sample image in the image data set of the sub-tasks into a multi-task model, and performing multi-scale feature extraction on the first sample image by adopting a first sub-network of the multi-task model to obtain first feature maps of different scales; performing feature fusion processing on the first feature maps of different scales by adopting a second sub-network shared by each sub-task network in an associated task group to which sub-tasks belong in the multi-task model to obtain a second feature map; processing the second feature map by adopting a sub-task network corresponding to sub-tasks in a multi-task model to obtain first sensing images of the sub-tasks, so as to obtain the first sensing images of the sub-tasks; and iteratively updating network parameters of the multi-task model according to the first sensing image and the first sample image corresponding to each sub-task to obtain a trained multi-task model.

The invention discloses an illumination perception adaptive method and system for night pedestrian re-identification. The method comprises the following steps: S1, obtaining an input image; s2, processing the input image through an illumination perception image preprocessing network, and dynamically generating an enhancement coefficient graph and a noise confidence graph; s3, performing detail enhancement on the input image through an adaptive enhancement branch to obtain an enhanced image; performing noise suppression on the input image through a feature purification module to obtain a purified image; s4, performing feature extraction on the enhanced image and the purified image by using a shared backbone network to obtain enhanced features and purified features respectively; and S5, fusing the enhancement feature and the purification feature through an illumination perception controller to obtain a final identity representation. According to the invention, through end-to-end illumination perception adaptive processing, image enhancement and feature purification are effectively cooperated, and the discrimination capability and robustness of a night pedestrian re-identification model in a low-light environment are significantly improved.