773 results about "Image representation" patented technology

The invention discloses a multi-modal entity and relation extraction method and system based on cross-modal alignment and fusion, and the method comprises the steps: carrying out the processing and coding of an input text and an image, and obtaining a plurality of types of image and text features; performing feature alignment on the fine-grained and coarse-grained text features and the pixel-level image representation by taking the semantic representation of the image as an anchor point, and mapping the image and the text features to the same semantic space; performing multi-granularity feature fusion through text-guided dynamic gating aggregation, visual prefix cross-modal fusion and cross-modal image-text matching, modeling association between noun phrases and image objects in a text while increasing feature complementarity, and obtaining multi-granularity multi-modal feature representation; fusing multi-granularity multi-modal features through entity guidance attention gating, and gathering visual information related to a text entity to obtain final multi-modal fusion representation; according to the multi-modal fusion representation, task prediction of multi-modal named entity recognition and multi-modal relation extraction is carried out.

A method includes adding an outpaint mask to an image to generate a masked image. The method also includes processing the image using an encoder neural network to generate an image representation of the image in a latent space. The method further includes processing the masked image using a convolution neural network and adding the image representation to generate an image embedding. The method also includes processing the image representation and the image embedding using at least one of a diffusion model and an interpolation process to generate a noisy latent image representation. The method further includes using a large language model to contextualize an outpainting prompt. The method also includes denoising the noisy latent image representation based on the contextualized outpainting prompt to generate a denoised latent image representation. In addition, the method includes processing the denoised latent image representation using a decoder neural network to generate an outpainted image.

The invention discloses an on-load tap-changer fault diagnosis method based on lightweight YOLO11, and is used for solving the problem of low mechanical fault diagnosis efficiency of a traditional on-load tap-changer. Comprising the following steps of: converting vibration signals in a motor starting stage and a change-over switch action stage into high-resolution time-frequency images through continuous wavelet transform (CWT) so as to comprehensively reflect time and frequency characteristics of the signals; starNet is adopted to replace a YOLO model backbone network, and the feature extraction capability is enhanced while the network compactness is kept through star operation; and designing an efficient joint convolution detection head EJCD, and realizing model lightweight and multi-scale target detection through grouping normalization and convolution parameter sharing. According to the invention, through combination of the time-frequency image representation capability of continuous wavelet transform and an efficient target detection framework of deep learning, the limitation of a traditional method in non-stationary signal processing is overcome, and the diagnosis efficiency and the model performance are significantly improved.

The invention discloses a sea ice target detection method based on multi-dimensional feature extraction, and the method specifically comprises the following steps: S1, data preprocessing: carrying out the denoising of an original sea ice image through a Gaussian filter, carrying out the multi-scale analysis of the sea ice image through an image pyramid, and obtaining sea ice images under different scales; s2, local feature extraction: extracting texture features of each local region by using a gray-level co-occurrence matrix; s3, performing enhanced texture feature processing on the extracted texture features through a directional filter and an enhancement algorithm to obtain final texture feature data; and S4, performing feature fusion on the obtained multi-dimensional features and the texture features to form a comprehensive feature image, and obtaining a classification result of the sea ice target. According to the sea ice target detection method, through multi-scale analysis, image representation is obtained under different scales by using an image pyramid transformation method, changes and expressions of texture features under different scales are captured, and the stability of sea ice target detection is improved.

The subject technology provides vision-based hand gesture customization. An apparatus receives input data having image representations of a first hand gestures corresponding to different types of gestures. For each of the first hand gestures, the apparatus extracts features from one of the image representations of the first hand gestures to produce a skeleton-based feature representation of a corresponding hand gesture. The apparatus produces a first trained machine learning model by training a neural network with a first dataset having skeleton-based feature representations associated with the first hand gestures to predict a classification for the first hand gestures. The apparatus generates a second dataset having training samples representing at least a second hand gesture customized by a user. The apparatus produces a second trained machine learning model by training the first trained machine learning model based on the second dataset to predict a classification for the second hand gesture.

The invention relates to a remote sensing image domain generalization semantic segmentation method based on multi-scale instance decoupling, which relates to the technical field of remote sensing image interpretation and comprises the steps of multi-scale instance coding, domain invariant instance decoupling and domain generalization semantic decoding. Image features in a segmented backbone network are used as input, and a multi-scale instance coding module effectively senses feature responses of cross-domain instances through deep convolution, multi-scale cooperation and multi-path sensing and is not affected by scale differences, spatial deformation and noise; designing a domain invariant instance decoupling module to decouple the cross-domain style from the image representation; and finally, the decoupled features are input into a domain generalization semantic decoding module so as to predict a segmented image on a target domain which is not seen, and the segmentation method can generalize various remote sensing images which are not seen so as to obtain better segmentation capability.

A machine-learned system for aligning textual and image representations prior to input to a sequence processing model is described. The system includes a machine-learned image embedding model configured to receive image data and generate one or more image embeddings and a machine-learned text embedding model configured to receive text data and the one or more image embeddings and generate one or more text embeddings. The system includes a machine-learned cross-modal adapter configured to generate one or more text tokens aligned with one or more image tokens based at least in part on aligning data associated with the one or more text embeddings and the one or more image tokens. The system includes a machine-learned sequence processing model configured to generate an output based at least in part on the one or more text tokens and the one more image tokens.

The invention relates to the technical field of image recognition, in particular to a stem cell classification method and system based on image segmentation. The method comprises the following steps: acquiring original stem cell image data, and performing Wasm processing on the original stem cell image data to obtain stem cell image quality parameters; performing adaptive preprocessing on the original stem cell image data based on the stem cell image quality parameters to obtain standardized stem cell image data; and performing multi-resolution pyramid construction on the standardized stem cell image data to obtain multi-level stem cell image representation data. According to the method, through multi-resolution pyramid construction, hierarchical processing of cell regions, accurate morphological feature extraction and hierarchical classification and distributed storage processing, the accuracy, reliability, adaptability and expandability of stem cell image classification are effectively improved.

A system includes an input to receive a digital waveform signal, a memory, and one or more processors configured to execute code to cause the one or more processors to: generate a horizontal ramp sweep signal based on the digital waveform signal; receive a selection input to identify a segment of the digital waveform signal; gate the horizontal ramp sweep signal and the digital waveform signal based on the selection input to produce cyclic loop image data for the segment of the digital waveform; store the cyclic loop image data in the memory; and provide the cyclic loop image data as one or more inputs into a machine learning system. A method of waveform classification using a cyclic loop image includes receiving an input waveform, receiving a selection of a segment of the input waveform, transforming the segment of the input waveform into cyclic loop image data, the transforming comprising generating a horizontal ramp sweep signal based on edge transitions in the input waveform, and storing the cyclic loop image data in a memory; and sending the cyclic loop image data to a machine learning system to determine an attribute of the input waveform.

The present invention belongs to the technical field of chip defect detection. Disclosed is an intelligent detection system for a chip appearance defect. According to the present invention, chip appearance defects are categorized according to similarity of image representations, and on the basis of detection advantages of different detection algorithms, the defects are detected in corresponding categories, so that more than 39 types of common and uncommon defects can be detected, thereby achieving comprehensive detection of the chip appearance defects. In addition, during defect detection, filtering parameters are set to filter NG-type products determined on the basis of recognition confidence, and strict detection parameters are used to ensure that all defects are found. Moreover, user-defined filtering parameters are allowed on a software interface. A user can adjust the filtering parameters on the basis of an enterprise quality standard to control detection quality.

The invention discloses a cerebral apoplexy prediction method and device based on a cerebrovascular image representation group, a medium and a product, and relates to the field of image processing, and the method comprises the steps: constructing a clinical database; according to the multi-modal image feature group data, the clinical feature data and the clinical cause typing of the patient, constructing a clinical cause typing prediction model; according to TOF-MRA, a cerebrovascular tissue segmentation model and voxel-based morphological analysis are adopted to determine cerebrovascular image features, and positioning diagnosis of a responsible blood vessel area is carried out according to the cerebrovascular image features; according to the cerebrovascular image features in the TOF-MRA, the SWI, the T1WI, the T2-FLAIR and the corresponding clinical feature data, a cerebral apoplexy recurrence risk prediction model is constructed; according to the application, the pathogenesis classification of the acute cerebral apoplexy patient can be quickly and accurately realized in a clinical scene, the responsible blood vessel can be accurately identified, and the recurrence risk prediction can be realized in the early stage of the disease.

The invention discloses a fault identification method combining enhanced convolution and multi-scale attention, and relates to the technical field of seismic interpretation and reservoir prediction. The method comprises the following steps: firstly, on the basis of seismic data high-resolution enhancement processing, generating a contrast-invariant four-channel image representation by using a local intensity sequence transformation filter; then, constructing an enhanced convolution encoder in combination with differential convolution to extract fault edge information, and meanwhile, constructing a convolution Mama attention module by fusing multi-scale separable convolution and a Mama framework to extract space and global information; and finally, constructing a feature fusion module to fuse the shallow features of the jump connection with the deep features of the decoder, and outputting a fault recognition result through full connection layer mapping. Compared with the prior art, the method combines the advantages of the local intensity sequence transformation, the attention mechanism, the convolutional network and the Mama architecture, can more effectively learn the fault feature information in the seismic data, and improves the fault recognition precision.

A system comprises an encoder configured to compress attribute information and / or spatial for a point cloud and / or a decoder configured to decompress compressed attribute and / or spatial information for the point cloud. To compress the attribute and / or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. Additionally, an encoder is configured to signal and / or a decoder is configured to receive a supplementary message comprising volumetric tiling information that maps portions of 2D image representations to objects in the point. In some embodiments, characteristics of the object may additionally be signaled using the supplementary message or additional supplementary messages.

The invention discloses a human body posture estimation method based on radar point cloud imaging and multi-dimensional feature fusion, and relates to the cross technical field of computer vision and radar signal processing, and the method comprises the following three key technical links: firstly, improving the target resolution through spatial energy distribution estimation; reconstructing target three-dimensional space distribution by using the positive correlation between radar signal energy and a target reflection area and adopting a least square estimation algorithm; secondly, constructing a structured multi-dimensional point cloud matrix, and converting sparse radar point cloud into high-information-density imaging representation through a distance-speed hierarchical sorting strategy; and finally, designing a multi-dimensional feature fusion attitude estimation network, integrating three-dimensional convolution, a multi-head attention mechanism and a gating circulation unit, and realizing collaborative extraction of spatio-temporal features. According to the method, the problems of sparse target features, noise sensitivity and poor universality in traditional millimeter wave radar attitude estimation are solved.

Non-invasively determining a treatment strategy for a blood vessel by obtaining a prediction of a total pressure drop value in the blood vessel based on features generated from a plurality of images of the blood vessel, the images representing all anatomic parts of the blood vessel, and calculating a contribution of one or more portion(s) of the blood vessel to the total pressure drop value. Based on the calculated contribution, a new simulated total pressure drop value of the blood vessel is calculated by neutralizing the contribution of the one or more portion(s) to the total pressure drop value to determine if the new simulated pressure drop improves, thereby indicating portions which should be treated to restore healthy pressure drop values.

The invention discloses a multi-mode LIBS mineral oxygen element analysis method. The method comprises the following steps: firstly, preprocessing an LIBS spectrum, and then fusing the spectrum and a mineral microscopic image by using a deep learning model to carry out feature extraction and oxygen analysis. The core of the method is model training: shielding a spectrum, pairing the spectrum with a corresponding mineral image (positive samples and negative samples), and generating a fused spectrum-image representation by using mask reconstruction and comparative learning. The element content is predicted based on this representation, and the model is optimized according to the oxygen prediction results. According to the method, accurate, robust and interpretable quantitative analysis of the oxygen element in the complex mineral matrix can be realized.

Systems, methods and computer readable media comprising a virtual exercise board, which is represented by images on the screen of a pad device; wearable devices configured to attach to each shoe of a user and to collect and transmit touch data to the pad device; cameras for tracking movement and calibrating with the data collected by the wearable devices; and computer programs for collecting user data, processing user data, and generating outputs. In embodiments, features include augmented reality; ratings of performance; automated workouts / protocols; real-time progress bar; multi-location database capabilities; and reports.

A computer-implemented method for identifying brands utilized in website phishing campaigns may include (i) capturing a website screenshot including visual elements representing a potential phishing vulnerability, (ii) transforming, utilizing a deep learning model, the website screenshot into an image representation including embeddings, (iii) determining whether the transformed website screenshot matches a dataset including reference transformed website screenshots representing previously identified brands utilized in phishing campaigns, (iv) clustering, upon determining a mismatch between the transformed website screenshot and the dataset, the transformed website screenshot with other transformed website screenshots sharing the visual elements representing the potential phishing vulnerability and one or more visual similarities, and (v) performing, based on the clustering, a security action that protects against potential phishing attacks by extracting brand information for adding to the dataset. Various other methods, systems, and computer-readable media are also disclosed.

The present disclosure provides a method, a device, and a product for retrieval. The method includes acquiring context information related to an image and determining a representation of the image based on image data and the context information of the image, where the context information includes at least one of environment parameters, user behavior data, time elements, or field metadata. The method further includes encoding the representation as an image vector in a high-dimensional vector space and storing it into an image vector database. When retrieval is performed, a query that includes text information and that is for the image vector database is received, and an image associated with the text information is determined from the image vector database. The method according to the present disclosure can improve accuracy and efficiency for image retrieval.

The invention discloses a multi-level cross-modal alignment method based on comparative learning, which is used for improving the accuracy and efficiency of multi-modal sentiment analysis. According to the method, a RoBERTa model and a Vision Transform model are used for coding a text and an image respectively, and text representation and image representation are obtained. The global cross-modal alignment module aligns the representation of the text and the representation of the image by adopting a comparative learning technology to enhance the consistency between the text and the representation of the image. Furthermore, through a local cross-modal alignment module, a cross-attention mechanism is used to perform fine-grained alignment on the text and image representations to identify smaller, more specific semantic units in the associated image and text. According to the method, a multi-task learning framework is adopted to integrate cross-modal information from texts and images, sequence tag prediction is carried out through a conditional random field, and terms and emotions in the aspects are recognized and classified. Experimental results show that the performance of the method on a Twitter-2015 data set and a Twitter-2017 data set is superior to that of an existing single-mode model and an existing multi-mode model, and the performance of multi-mode sentiment analysis is effectively improved.

A system comprises an encoder configured to compress attribute information and / or spatial information for a point cloud and / or a decoder configured to decompress compressed attribute and / or spatial information for the point cloud. To compress the attribute and / or spatial information, the encoder is configured to convert a point cloud into an image based representation. Also, the decoder is configured to generate a decompressed point cloud based on an image based representation of a point cloud. In some embodiments, a bit stream structure may be used to communicate compressed point cloud data. The bit stream structure may include point cloud compression network abstraction layer (PCCNAL) units that enable use of groups of frames (GOFs), frame, and sub-frame signaling of patch information. Such a bit stream structure may permit low delay streaming and random access reconstruction of point clouds amongst other applications.

The invention discloses a 3D printing process defect monitoring method and system based on a multi-modal large model. The method comprises the steps that S1, before related operation is carried out, corresponding preposition work needs to be completed, wherein the preposition work comprises knowledge graph creation, retrieval enhancement generation, reasoning and action framework and fine adjustment of the large model; s2, collecting and preprocessing industrial camera cluster data; s3, according to the preprocessed image, segmenting the image, constructing a region adjacency graph, finding an optimal region combination by applying a random algorithm, and combining some regions to obtain image representation; s4, calling the multi-mode large model subjected to pre-training and specific field fine adjustment to realize real-time defect monitoring in the 3D printing process; s5, a detection report is given according to a defect detection result, user interaction intelligent consultation is opened, and the model is updated and adjusted according to printing data; according to the method, the problems of standard, efficiency, trust and supervision of defect monitoring in the 3D printing process can be effectively solved.

The invention belongs to the technical field of image processing, and provides an image processing method and an image processing terminal, and the method comprises the following steps: obtaining an original binary image from an image source, and carrying out the preprocessing of the original binary image; performing run-length coding on the preprocessed binary image to obtain an image data result after run-length coding; performing morphological processing on the image in the row direction by using the structural element matrix; re-integrating each row of data processed in the row direction into a complete image representation, and performing morphological processing in the column direction; taking a processing result in the column direction as a binary morphological calculation result of the original binary image; according to the method, the image is independently processed in the row direction and the column direction, specific shapes and structures can be more flexibly extracted and enhanced, proper structural elements are adopted according to specific application requirements, and the processing effect can be optimized; by using the run-length coding to represent the image, the overhead of data storage and transmission can be significantly reduced, and the processing speed is improved.

The invention relates to the field of soil substance detection, and discloses a soil plastic detection method and system based on multi-modal fusion and deep learning, and the method comprises the following steps: obtaining one-dimensional hyperspectral data of a to-be-detected soil sample; converting the one-dimensional hyperspectral data based on an image conversion algorithm to obtain two-dimensional image representation; performing feature extraction on the one-dimensional hyperspectral data based on a first preset algorithm to obtain spectral features; performing feature extraction on the two-dimensional image representation based on a second preset algorithm to obtain image features; performing multi-view probability fusion on the spectral features and the image features to obtain fusion features; and inputting the fusion features into a pre-trained double-path attention residual convolutional network model for classification to obtain a detection result of the micro-plastics in the to-be-detected soil sample. According to the method, the detection accuracy is improved, the model robustness is enhanced, and low-concentration detection is realized.

The invention relates to the technical field of computers, in particular to a general traffic image generation method for solving unbalanced network traffic classification, which comprises the following steps of: arranging and combining original network traffic into a session stream according to a time sequence based on a session stream mode, and converting the session stream into an image by taking a data packet as a unit. Converting the effective load of each data packet into a grayscale image; the SCGAN is used for training; eliminating noise by adopting a convolution noise reduction auto-encoder, and performing high-definition reconstruction on the generated flow sample; and combining a minority of types of traffic image samples generated through high-definition reconstruction with the original real traffic samples. According to the method, when the data packets are converted into the traffic images, the time sequence dependency relationship of the network traffic is reserved, the structural features among the data packets in image representation are also reserved, a balanced new network traffic data set is constructed, the authenticity and diversity of the data set are kept, and the generalization ability and effect of the model are improved.

The invention discloses an image retrieval method and device, computer equipment and a storage medium, belongs to the technical field of artificial intelligence, and is provided with a retrieval system applied to an insurance marketing scene image. According to the method and the device, the to-be-processed image is segmented and coded, the content features of the image are combined with the position information to form high-quality image representation, and the corresponding image index information is generated by utilizing the preset image index generator, so that efficient indexing and retrieval of the image are realized. In the encoding process, local features of the image are reserved, spatial position information is fused, discrimination and uniqueness of image indexing are improved, generated image indexing information and an original image are stored in an image retrieval database in an associated mode, it is ensured that the image retrieval process has high matching precision and response speed, and the image retrieval efficiency is improved. The image retrieval accuracy and processing efficiency are effectively improved, and the method is suitable for large-scale image library management and rapid retrieval.

Methods for multiple instance learning of tissue sample images are described. The methods may comprise, for example, receiving a whole slide image from a needle core biopsy sample from a subject; identifying a tissue region in the whole slide image; selecting a set of image patches from the identified tissue region; resampling the set of image patches at a plurality of image scales to generate a plurality of resampled image patches; generating image representations for the plurality of resampled image patches; extracting feature vectors based on the image representations; providing the feature vectors as input to a trained machine learning model configured to predict a gene alteration state; and outputting the predicted gene alteration state for the needle core biopsy sample for the subject.

The invention is suitable for the field of intelligent jet cleaning, and provides a dirt recognition cleaning method, device and equipment, and the dirt recognition cleaning method comprises the steps: obtaining an original image corresponding to target dirt and a sound reflection signal corresponding to the target dirt; converting the sound reflection signal corresponding to the target dirt into an image representation; obtaining a fused image corresponding to the target dirt according to the original image corresponding to the target dirt, the image representation and a preset multi-channel image fusion algorithm; inputting a fusion image corresponding to the target dirt into the improved deep neural network to obtain segmentation mask data and size data of the target dirt; adjusting the pose of the dirt cleaning equipment according to the segmentation mask data and a preset coordinate conversion algorithm; according to the size data of the target dirt, dirt cleaning parameters are adjusted, and dirt cleaning equipment is controlled to execute cleaning operation. According to the method, the dirt recognition and cleaning precision and efficiency are remarkably improved, and various cleaning requirements and operation environments can be better met.

Techniques for generating a novel image using tokenized image representations are disclosed. In some embodiments, a method of generating the novel image includes generating, via a first machine learning model, a first sequence of coded representations of a first image having one or more features; generating, via a second machine learning model, a second sequence of coded representations of a sketch image having one or more edge features associated with the one or more features; predicting, via a third machine learning model, one or more subsequent coded representations based on the first sequence of coded representations and the second sequence of coded representations; and based on the subsequent coded representations, generating, via the third machine learning model, a first portion of a reconstructed image having one or more image attributes of the first image, and a second portion of the reconstructed image associated with the one or more edge features.