167 results about "Perceptual image" patented technology

The invention discloses a target detection method based on Mama feature fusion, and relates to the technical field of image target detection. According to the method, the innovative implementation of the VSSA module is utilized, a selective scanning mechanism of the state space model is applied to 2D visual data processing, the long-distance dependency relationship in the image is effectively captured through state space modeling in four directions, the limitation of a traditional state space model in the two-dimensional visual data processing process is solved through the multi-direction processing strategy, and the processing precision of the 2D visual data is improved. The model can comprehensively perceive spatial dependency relationships in different directions in an image, the VSSA adopts learnable state space parameters to dynamically model a feature sequence, the ability of the network to understand a complex space structure is enhanced, the method is particularly suitable for processing scenes needing long-distance context information, and in addition, the method is combined with MTMHSA, so that the complexity of the network is reduced. And the fusion capability of different levels of features in target detection is further enhanced. Through the innovation, the model can better understand the target in the image, and the positioning and classification precision of the target is improved.

The invention discloses a vision-language pre-training general framework for realizing multi-granularity cross-modal alignment. Image-text pairs are used as training data to be respectively input into an image encoder and a text encoder to obtain coarse-granularity global features and fine-granularity local features; multi-granularity cross-modal alignment is carried out on the obtained global features and local features, and the aligned global features of the image and the text are used for cross-modal pre-training; the multi-granularity cross-modal alignment is composed of a text perception image block selection module and a cross-granularity alignment module; a text perception image block selection module in the step S3 selects the image block with the highest correlation through the correlation between text words or phrases and the image blocks to form a corresponding region or target object, so that more sufficient fine-grained alignment is realized; the cross-modal pre-training target comprises the multi-granularity cross-modal pre-training target constructed in the step S6 and universal image-text comparison loss, image-text matching loss and mask text modeling loss.

The invention discloses a three-dimensional medical image visual language model pre-training method for reconstructing a hybrid strategy, which belongs to the technical field of medical image calculation and comprises the steps of constructing a medical image text pair data set, a language text mask reconstruction strategy, a visual image mask reconstruction strategy, a semantic perception fusion strategy and multi-task joint learning. According to the method, the large language model is finely adjusted, the diagnosis and attribute information in the medical report is extracted by using the finely-adjusted large language model, the efficient prompt is generated, and the large language model has relatively high generalization ability, so that the cost of manual annotation is greatly saved; according to the semantic perception fusion strategy, the text feature obtained by the text encoder and the image feature obtained by the image encoder are combined to obtain the new text feature, so that the text perceives the diagnosis and attribute information of the image in advance, the alignment of the image and the text in the embedding space is further optimized, and the pre-training efficiency is improved.

The invention provides a micro robot motion control method and system, and the method comprises the steps: carrying out the synchronization and calibration processing of environment state information, and generating a comprehensive perception image; analyzing the comprehensive perception image to obtain a classification result, and predicting the environment state change influencing the motion track of the micro robot in a period of time in the future based on the classification result; generating a candidate path by using a fast random tree, and optimizing the candidate path in combination with an AI search algorithm to obtain a target path; constructing a three-dimensional dynamic probability graph, continuously updating the three-dimensional dynamic probability graph by using a reinforcement learning algorithm, and generating an instant decision map; and estimating the position of the micro-robot by using a Kalman filter to obtain the position state of the micro-robot, adjusting the speed and steering angle of a driving motor of the micro-robot, and generating trajectory tracking of the micro-robot. According to the technical scheme provided by the invention, the flexibility and intelligence of the micro robot are improved.

The invention relates to the field of artificial intelligence (AI), in particular to an electrical stimulation method, an electrical stimulation device, electrical stimulation equipment and a storage medium. The method comprises the steps of obtaining a to-be-perceived image of a target object, inputting the to-be-perceived image into a trained electrical stimulation model to obtain electrical stimulation parameters output by the electrical stimulation model, and performing corresponding electrical stimulation operation on the target object to recover visual perception of the target object on the to-be-perceived image. An electrical stimulation generator in electrical stimulation is obtained by performing first link training and second link training based on a plurality of sample images and corresponding sample neural signals, the first link represents a verification link among electrical stimulation parameters, analog neural signals and reconstructed images, and the second link represents a verification link among the reconstructed images. The second link represents a verification link among the electrical stimulation parameters, the light magical view image and the sample image, so that the electrical stimulation model can greatly optimize the generation process of the electrical stimulation parameters, and the accuracy and generalization of restoring visual perception through electrical stimulation are improved.

The invention discloses an overwater target multi-dimensional fusion identification method and system, and the method comprises the following steps: S1, collecting perception image data in a water surface environment, and carrying out the preprocessing of the perception image data; s2, carrying out multi-modal fusion; s3, inputting the fused image data into an OfficientDet target detection network, extracting a feature map, and outputting a category label and spatial position information of a water target through a bounding box regression sub-network and a target classification sub-network; s4, constructing time sequence input, inputting the time sequence input to a Motionform network, and extracting motion trail features of the water target; s5, generating an identification report; and S6, performing online updating on the network parameters by adopting an incremental learning mode. According to the method, the OfficitDet network and the Motionform network are fused, accurate detection and dynamic recognition of the water target are achieved, and the method has the advantages of being high in precision, high in stability and capable of achieving self-adaptive optimization.

The invention relates to the technical field of image recognition and classification, in particular to a fine-grained image recognition method and system based on a shared visual backbone network. The method comprises the following steps: acquiring a fine-grained image data set; preprocessing the acquired image data; constructing a deep network model, including performing feature extraction by using the deep network model; object proposing and screening are carried out on the extracted feature map; performing feature pooling and information fusion on the screened features; performing global abstract extraction and gating context fusion on the fused features; projecting an object sequence in the gated and fused features to a feature space of a language model; training and optimizing the constructed deep network model; and performing fine-grained image classification by using the optimized model. According to the method, a parallel local feature extraction path is introduced, and a cross-attention fusion module is designed, so that the model can sense the global context and local discriminative details of the image at the same time.

The invention relates to a rapid and video-memory-friendly image segmentation method based on an SAM model, and the method comprises the following steps: 1, building a U-SAM model based on the training of the SAM model; the U-SAM model is trained by the following steps: step S1, establishing an SAM model and a U-SAM model; a semantic amp of the U-SAM model is obtained through an image encoder of the knowledge distillation compression SAM model; the scale perception image encoder is used for improving the semantic perception capability of the SAM; and S2, freezing related parameters of the SAM model, retaining prompt segmentation capability, training a double-token mask decoder of the U-SAM model, and solving the problem that prompt embedding is difficult to control. 2, using the U-SAM model to realize prompt segmentation and general image segmentation; therefore, prompt and universal segmentation tasks can be realized through a rapid and video-friendly framework.

The invention provides an unmanned aerial vehicle full-time perception image reconstruction method based on multi-mode cooperative reinforcement learning and degeneration decoupling. A frequency perception feature modulation model, a dual-mode dual-domain transformation module and a dynamic bidirectional guide mechanism are included. According to the system, firstly, feature information of different frequency bands is adaptively separated and modulated through a frequency sensing feature modulation model, and decoupling and compensation of composite unknown degradation are achieved; realizing cross-domain interaction and information fusion of visible light and infrared characteristics in a spatial domain and a channel domain by using a bimodal dual-domain transformation module; and finally, realizing collaborative enhancement of cross-modal degradation perception through a bidirectional dynamic guide mechanism, and generating an unmanned aerial vehicle visible light reconstruction image and an infrared super-resolution image with higher structural consistency and texture fidelity. According to the method, deep fusion and degeneration decoupling of multi-modal information can be realized in a complex degeneration environment, and the imaging quality and the environmental adaptability of an unmanned aerial vehicle full-time sensing system are remarkably improved.

Systems and methods for detecting phishing using image hashing include obtaining a plurality of images from different sources, generating a hash for each image, comparing at least one hash associated with a first image to one or more hashes associated with a second image, calculating a similarity score based on the comparing, and classifying the first image based on the similarity score.

The invention provides an efficient image super-resolution method and device based on dual guidance semantic enhancement, and relates to the technical field of image super-resolution, visual content generation and the like in computer image process.The method comprises the steps that an image super-resolution model containing a pre-training diffusion model backbone structure is constructed, and a pre-training diffusion model is constructed; a trainable LoRA module is inserted into a U-Net structure and a VAE encoder of the image super-resolution model; calculating the perception image loss of the generated image and the real image by adopting a dual-guidance quality enhancement training strategy; based on a semantic alignment method of score matching, performing semantic feature constraint on the generated image through a condition vector of a pre-training diffusion model to form semantic consistency loss aligned with real image distribution; and carrying out end-to-end training on the image super-resolution model under joint optimization of perceptual image loss and semantic consistency loss, and after the training is finished, realizing high-resolution generation of a low-resolution image only through one-time U-Net reasoning.

The invention discloses an edge extraction method and system for target segmentation, and belongs to the field of target segmentation, and the method comprises the steps: obtaining a vehicle collection image set, and constructing an edge detection model based on the vehicle collection image set; acquiring a real-time perception image based on a vehicle visual perception system; inputting the real-time perception image into an edge detection model to obtain a semantic edge probability graph; performing binarization processing on the semantic edge probability graph to obtain a binarized edge graph, and performing pixel clustering on the binarized edge graph to obtain a plurality of continuous edge communication domains; and carrying out edge distribution intensity calculation to obtain a plurality of edge intensities corresponding to the plurality of continuous edge connected domains, and obtaining an edge extraction result based on the plurality of edge intensities. The technical problem of low target segmentation precision caused by poor edge extraction accuracy for target segmentation in the prior art is solved, and the technical effects of improving the edge extraction accuracy for target segmentation and improving the target segmentation precision are achieved.

The disclosure relates to systems and methods automatically extracting an image and related image components, computationally determining an understanding of the image, and generating mathematical vector embeddings via sentence encoders based on the computationally determined understanding. The mathematical vector embeddings may be used for semantic image retrieval that enables image searching based on a semantic understanding of input images and / or input text. The mathematical vector embeddings may be used for training and executing generative Artificial Intelligence (AI) models to create new content that includes retrieved images and / or generate new images.

The invention discloses an image quality evaluation method, device and equipment and a computer program product, and the method comprises the steps: carrying out the brightness region decomposition of an HDR image, and determining a plurality of brightness regions of the HDR image; multi-level spatial feature extraction is performed on each brightness region, and a first feature map of each brightness region in M stages is determined, so that features of a dynamic range extreme region can be extracted, features of a standard dynamic range region can be extracted, and unique distortion of an HDR image can be accurately sensed; and further performing feature recombination on the first feature maps of the plurality of brightness regions in the same stage, determining second feature maps of M stages, the second feature maps being capable of effectively representing global spatial features of the HDR image in one stage, and performing image quality evaluation according to the second feature maps of the HDR image in M stages. The quality evaluation accuracy of the HDR image can be improved.

The invention discloses a progressive migration training decoding method based on GPT, and belongs to the field of optic nerve decoding. According to the method, a GPT-based category and text decoding model is constructed based on functional magnetic resonance signal data recorded when a subject watches a large number of visual perception images, and the method comprises an image encoder used for embedding image feature vectors extracted by adopting a pre-training model into a hidden feature space; prompt containing task information and position codes are embedded into a hidden feature space; the visual information encoder is used for mapping a brain visual region voxel signal to a hidden feature space by using a GRU; the whole-brain information encoder is used for guiding the selection of voxel signals of other regions of the brain by taking a voxel signal of a visual region of the brain as a context, and mapping global information to a hidden feature space; and the language generator fuses all the hidden features and selects the Token with the highest probability to generate the text of the corresponding content. According to the method, the decoding accuracy is improved, and the generalization ability of the model is enhanced.

The invention discloses a generative video coding method based on implicit inter-frame alignment, which comprises the following steps of: 1, intelligently modeling space-time characteristics between video frames through a self-adaptive space-time importance encoder, and dynamically allocating code rates; 2, extracting features of a previous frame through an implicit inter-frame alignment strategy, fusing the features with information of a current frame, and converting the information of the previous frame through the structure of the U-Net network to be aligned with the current frame; and step 3, combining the perception loss function with the perception image block similarity constraint. The method has the following three advantages: the generation quality of a key region is optimized through an adaptive space-time importance encoder, and meanwhile, the bit overhead of a secondary region is reduced; an implicit inter-frame alignment strategy is used for capturing an inter-frame potential time-space relationship, so that the calculation complexity of estimating explicit motion information is reduced, and the calculation overhead is reduced; and the perception quality of the video frame is effectively improved through the perception loss function.

The invention discloses a cleaning equipment control method and related equipment, and relates to the field of cleaning equipment control, the cleaning equipment comprises a grabbing mechanical arm, and the method comprises the steps that perception image information of a first target area is acquired; determining whether a to-be-grabbed object exists or not based on the perceived image information; under the condition that the to-be-grabbed object exists, obtaining to-be-analyzed image information of the to-be-grabbed object; determining grabbing parameter information of the to-be-grabbed object based on the to-be-analyzed image information; and based on the grabbing parameter information, a grabbing mechanical arm is controlled to execute grabbing operation on the to-be-grabbed object.

Systems and methods for view-conditioned diffusion for real-world vehicle gaussian splatting. A single perspective image can be transformed using image transformation techniques to generate a training dataset that addresses a domain gap between synthetic data and real-world data in a traffic scene. A pre-trained diffusion model can be finetuned with the training dataset to obtain a fine-tuned diffusion model. Perspective-aware images having different perspective views of an entity from the single perspective image can be generated using the fine-tuned diffusion model. A large generative model (LGM) can be trained using the perspective-aware images to generate a gaussian splatting model for the entity. View-conditioned simulations from the single perspective image can be generated by using the gaussian splatting model for downstream tasks.

The present application provides a method and device for processing rail network information based on real-time state perception images, relating to the field of information processing technology. The method comprises: determining an initial rail network image under normal conditions for all rail transit network information; the initial rail network image represents the attributes, state, and location of rail transit network equipment; and performing pixel-level updates on the initial rail network image based on the real-time state of the rail transit network to obtain a real-time rail network image. The method and device for processing rail network information based on real-time state perception images provided by the present application can provide a real-time and intuitive description of rail transit network information that is difficult to obtain.

The invention discloses a three-dimensional attribute detection method and device and a storage medium. The method comprises the steps that a sensor located at the top of a lift car is called in an elevator to detect and sense image data downwards; sensing that a pixel point in the image data has a depth value; performing self-correction on the depth values of the pixel points in the perception image data; if the self-correction is completed, detecting image data of a region where the target object is located in the sensing image data according to the lift car; and under the condition of filtering out the background of the lift car, carrying out statistics on the three-dimensional attribute of the target object in the regional image data. According to the embodiment, the data size is greatly reduced through operations such as target detection and background filtering, so that the calculation amount is effectively reduced, the method is suitable for hardware with low elevator calculation power, and the calculation efficiency is improved. Moreover, interference is gradually eliminated through operations such as target detection and background filtering, the target object is gradually converged, the three-dimensional attributes are counted for the target object, and the accuracy of the three-dimensional attributes can be improved.

The present disclosure relates to systems, methods, and non-transitory computer readable media for utilizing a diffusion neural network for mask aware image and typography editing. For example, in one or more embodiments the disclosed systems utilize a text-image encoder to generate a base image embedding from a base digital image. Moreover, the disclosed systems generate a mask-segmented image by combining a shape mask with the base digital image. In one or more implementations, the disclosed systems utilize noising steps of a diffusion noising model to generate a mask-segmented image noise map from the mask-segmented image. Furthermore, the disclosed systems utilize a diffusion neural network to create a stylized image corresponding to the shape mask from the base image embedding and the mask-segmented image noise map.

The invention provides a multi-task model training method, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as automatic driving and the like. The method comprises the following steps: respectively inputting a perception image sample into a feature extraction network in a pre-trained single-task model and a feature extraction network in a to-be-trained multi-task model to obtain a single-task feature and a multi-task feature; one subtask branch of the multi-task model is the same as the task of the single-task model; determining distillation loss between the single-task feature and the multi-task feature based on the task type of the single-task model; based on the distillation loss, performing parameter updating on a feature extraction network of the multi-task model to train the multi-task model; the feature extraction network is shared by each sub-task branch in the multi-task model. According to the invention, the problem of multi-task negative migration is solved, and the performance of the multi-task model is improved.

The invention discloses an equipment identification and quality grade joint evaluation method for a transformer substation inspection image, and the method comprises the steps: inputting a collected original transformer substation image into an image identification and quality grade joint evaluation model; the model comprises a detection perception image enhancement module, a target detection module, an ROI extraction module and an image quality evaluation module which are connected in sequence. Performing enhancement processing on the original transformer substation image by using a detection perception image enhancement module to obtain an enhanced image; performing target detection on the enhanced image by using a target detection module to obtain a power equipment category label and ROI frame coordinate information; an ROI extraction module is utilized to cut the original substation image according to the ROI frame coordinate information to obtain a corresponding equipment ROI image; performing no-reference image quality evaluation on the ROI image of the equipment by using an image quality evaluation module to obtain a quality grade label; according to the method, the equipment identification accuracy is improved, and the image quality grade is credibly output.

The invention relates to an insulator material defect detection method, system, device, medium and program, and the method comprises the steps: carrying out the denoising filtering and image standardization operation of an insulator image obtained in advance, and obtaining a standard insulator image; performing multi-channel convolution and residual connection on the standard insulator image to obtain an insulator image feature set; performing global attention coding on the insulator image feature set to obtain an attention image feature set; extracting global channel image features from the insulator image feature set, and performing channel fusion and spatial fusion on the global channel image features to obtain perception image features; and performing layer-by-layer feature decoding on the attention image feature set by using the perception image features to obtain a defect detection result, thereby improving the insulator defect detection efficiency.

The invention provides a semantic and structure perception image restoration method and device based on a diffusion model. The method comprises the steps of obtaining a to-be-processed image and a mask used for identifying a to-be-restored area; performing panoramic segmentation on the to-be-processed image to obtain a semantic feature map, and constructing a peripheral semantic candidate set for the mask boundary to constrain semantic distribution of the to-be-restored region; extracting a structural feature map of the to-be-processed image, wherein the structural feature map comprises at least one of edge information, line segment information and angular point information; and inputting the to-be-processed image, the mask, the semantic feature map and the structural feature map into a multi-conditional diffusion generation network, and generating repair content in the to-be-repaired area by the multi-conditional diffusion generation network under the constraint of the semantic feature map, the structural feature map and the peripheral semantic candidate set, and outputting the restored image.

The invention relates to the technical field of image detection, and provides an image forgery detection method based on a multi-modal large language model, which comprises the following steps: inputting a to-be-detected image into the multi-modal large language model; the multi-modal large language model outputs an authenticity identification result of the to-be-detected image; wherein the multi-modal large language model has three parallel expert branches of physical consistency, semantic consistency and underlying structure clue analysis. According to the method, semantic information in the image is perceived and the context logic and the physical logic of the image are understood by fully utilizing the powerful priori knowledge, the deep semantic understanding capability and the chained reasoning capability of the multi-modal large language model, and meanwhile, complex reasoning and explanation can be performed on a forged image through a natural language.

The invention discloses a ground penetrating radar high-quality imaging method based on an image recovery diffusion model, and the method comprises the steps: constructing a simulation image through an open source simulation tool gprMax based on a finite difference time domain method (FDTD), and constructing a contrast high-frequency clutter-free and low-frequency clutter data set through preprocessing and fusing actual measurement clutter simulation; a double-layer routing attention mechanism is introduced on a basic model to enhance association between features, conditional information is fully utilized, discrete wavelet transform is used to effectively separate low-frequency and high-frequency information, information loss is reduced, and the high-frequency information is enhanced through an adaptive wavelet transform high-frequency enhancement module. According to the method, good improvement is achieved on a synthetic data set, acceptable effects are achieved on multiple evaluation indexes including root mean square error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index (SSIM), perceptual image similarity (LPIPS) and depth image structure and texture similarity (DISTS), and a certain underground target structure can also be recovered in actual measurement data.

According to embodiments of this disclosure, a method and apparatus for data processing are provided. The method includes: determining a reference region corresponding to a target traffic light indicated by map data in an image coordinate system corresponding to a perceived image; determining a set of candidate objects from a plurality of candidate objects identified based on the perceived image, based on the positional relationship between the reference region and a plurality of candidate regions, wherein the plurality of candidate regions correspond to the plurality of candidate objects identified as traffic lights; determining a first correlation between a first attribute of the target traffic light and a second attribute of the set of candidate objects, wherein the first attribute is determined based on the map data and the second attribute is determined based on the perceived image; and matching the target traffic light with the target objects in the set of candidate objects based on the first correlation. Based on this approach, embodiments of this disclosure can more accurately identify and match traffic lights.

The application discloses a three-dimensional medical image visual language model pre-training method of reconstruction mixed strategy, belongs to the technical field of medical image computation, and comprises the following steps: constructing a medical image text pair data set, a language text mask reconstruction strategy, a visual image mask reconstruction strategy, a semantic perception fusion strategy and multi-task joint learning; the large language model is fine-tuned, the fine-tuned large language model is used to extract diagnosis and attribute information in a medical report and generate efficient prompts, the large language model has strong generalization ability, and the cost of manual labeling is greatly saved; the semantic perception fusion strategy is used for combining text features obtained by a text encoder and image features obtained by an image encoder to obtain new text features, so that the text perceives diagnosis and attribute information of the image in advance, the alignment of the image and the text in an embedding space is further optimized, and the pre-training efficiency is improved.

A display system for a motor vehicle includes a projection device having a display device which is configured to output image-bearing light for projecting a display image in an eye area of a user via a display surface. A combiner surface reflects the image-bearing light into the eye area of the user. A device configured to obtain an eye pose of the user transforms a display image with the aid of a post-transformation matrix so that local flaws in a surface section of the combiner surface which result in a distortion or a fuzziness of a perception image perceptible by the user are compensated. A display image is displayed on the display surface using brightness values so that a homogeneous and high contrast representation results based on the eye pose of the user and the detail of the combiner surface. The transformed display image is output via the display device.