37 results about "Cross modality" patented technology

Neuro-response data including Electroencephalography (EEG), Functional Magnetic Resonance Imaging (fMRI) is filtered, analyzed, and combined to evaluate the effectiveness of stimulus materials such as marketing and entertainment materials. A data collection mechanism including multiple modalities such as, Electroencephalography (EEG), Functional Magnetic Resonance Imaging (fMRI), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data and removes cross-modality interference.

Central nervous system, autonomic nervous system, and effector data is measured and analyzed to determine the effectiveness of marketing and entertainment stimuli. A data collection mechanism including multiple modalities such as Electroencephalography (EEG), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data. The response data is enhanced using intra-modality response synthesis and/or a cross-modality response synthesis.

Central nervous system, autonomic nervous system, and effector data is measured and analyzed to determine the effectiveness of marketing and entertainment stimuli. A data collection mechanism including multiple modalities such as Magnetoencephalography (MEG), Electrooculography (EOG), Galvanic Skin Response (GSR), etc., collects response data from subjects exposed to marketing and entertainment stimuli. A data cleanser mechanism filters the response data. The response data is enhanced using intra-modality response synthesis and/or a cross-modality response synthesis.

Characterization of carotid atherosclerosis and classification of plaque into symptomatic or asymptomatic along with the risk score estimation are key steps necessary for allowing the vascular surgeons to decide if the patient has to definitely undergo risky treatment procedures that are needed to unblock the stenosis. This application describes a statistical (a) Computer Aided Diagnostic (CAD) technique for symptomatic versus asymptomatic plaque automated classification of carotid ultrasound images and (b) presents a cardiovascular stroke risk score computation. We demonstrate this for longitudinal Ultrasound, CT, MR modalities and extendable to 3D carotid Ultrasound. The on-line system consists of Atherosclerotic Wall Region estimation using AtheroEdge™ for longitudinal Ultrasound or Athero-CTView™ for CT or Athero-MRView from MR. This greyscale Wall Region is then fed to a feature extraction processor which computes: (a) Higher Order Spectra; (b) Discrete Wavelet Transform (DWT); (c) Texture and (d) Wall Variability. The output of the Feature Processor is fed to the Classifier which is trained off-line from the Database of similar Atherosclerotic Wall Region images. The off-line Classifier is trained from the significant features from (a) Higher Order Spectra; (b) Discrete Wavelet Transform (DWT); (c) Texture and (d) Wall Variability, selected using t-test. Symptomatic ground truth information about the training patients is drawn from cross modality imaging such as CT or MR or 3D ultrasound in the form of 0 or 1. Support Vector Machine (SVM) supervised classifier of varying kernel functions is used off-line for training. The Atheromatic™ system is also demonstrated for Radial Basis Probabilistic Neural Network (RBPNN), or Nearest Neighbor (KNN) classifier or Decision Trees (DT) Classifier for symptomatic versus asymptomatic plaque automated classification. The obtained training parameters are then used to evaluate the test set. The system also yields the cardiovascular stroke risk score value on the basis of the four set of wall features.

The present invention encompasses a physical or virtual, computational, analysis, fusion and correlation system that can automatically, systematically and independently analyze collected sensor data (upstream) aboard or streaming from aerial vehicles and/or other fixed or mobile single or multi-sensor platforms. The resultant data is fused and presented locally, remotely or at ground stations in near real time, as it is collected from local and/or remote sensors. The invention improves detection and reduces false detections compared to existing systems using portable apparatus or cloud based computation and capabilities designed to reduce the role of the human operator in the review, fusion and analysis of cross modality sensor data collected from ISR (Intelligence, Surveillance and Reconnaissance) aerial vehicles or other fixed and mobile ISR platforms. The invention replaces human sensor data analysts with hardware and software providing two significant advantages over the current manual methods.

A cross-time and cross-modality inspection method for medical image diagnosis. A first set of medical images of a subject is accessed wherein the first set is captured at a first time period by a first modality. A second set of medical images of the subject is accessed, wherein the second set is captured at a second time period by a second modality. The first and second sets are each comprised of a plurality of medical image. Image registration is performed by mapping the plurality of medical images of the first and second sets to predetermined spatial coordinates. A cross-time image mapping is performed of the first and second sets. Means are provided for interactive cross-time medical image analysis.

The invention belongs to the technical field of cross-media relevance learning, and particularly relates to a social image oriented cross-modality image-label relevance learning method. The invention comprises three algorithms: multi-modal feature fusion, bidirectional relevancy measuring and cross-modality relevancy fusion; a whole social image set is described by taking a hyperimage as a basic model, the image and a label are respectively mapped into hyperimage nodes for treatment, relevancy aiming at the image and the relevancy aiming at the label are obtained, and the two different relevancies are combined according to a cross-modality fusion method to obtain a better relevancy. Compared with the traditional method, the method is high in accuracy and high in adaptivity. The method has important significance in performing efficient social image retrieval by considering multi-modal semantic information based on large-scale social images with weak labels, retrieval relevancy can be improved, user experience is enhanced, and the method has application value in the field of cross-media information retrieval.

The invention discloses a cross-modal retrieval algorithm based on mixed hypergraph learning in subspace, based on the cross-modal public subspace learning of canonical correlation analysis. The similarities inside and among modals are calculated by a mapping of the public subspace. A mixed relation matrix is calculated by the similarities inside and among the modals. A mixed hypergraph model is built by the extraction of the relation matrix. And the cross-modal retrieval and sample sequencing are conducted by adoption of hypergraph learning finally. With aiming at cross-modal heterogeneous variations and high order relation among samples, the algorithm instance applies the hypergraph model combined with the cross-modal public subspace learning to the cross-modal retrieval, so that the model is capable of considering the similarity among the modals and the similarity inside the modals simultaneously and giving consideration to the high order relation among a plurality of the samples meanwhile, improving the final precision ratio and the final recall ratio of the cross-modal retrieval. The algorithm is capable of effectively improving the performance of the cross-modal retrieval and greatly enhancing the precision ratio and the recall ratio of the cross-modal retrieval.

The present teaching relates to method, system, medium, and implementation of in-situ perception in an autonomous driving vehicle. A plurality of types of sensor data are acquired continuously via a plurality of types of sensors deployed on the vehicle, where the plurality of types of sensor data provide information about surrounding of the vehicle. One or more items surrounding the vehicle are tracked, based on some models, from a first of the plurality of types of sensor data from a first type of the plurality of types of sensors. A second of the plurality of types of sensor data are obtained from a second type of the plurality of sensors and are used to generate validation base data. Some of the one or more items are labeled, automatically, via validation base data to generate labeled at least some item, which is to be used to generate model updated information for updating the at least one model.

The invention discloses an image search method based on soft constraint non-supervision cross-modality hash. The method sequentially comprises the following steps of building a picture and a dataset of text data corresponding to the picture to obtain an image feature data matrix and a text feature data matrix; configuring an image similarity matrix and a text similarity matrix, distributing hash codes according to the image similarity matrix and the text similarity matrix, and thus obtaining guiding hash codes; adopting the guiding hash codes to optimize final hash codes and a corresponding projection matrix; calculating a Hamming distance between the hash codes and hash codes of a sample in a search base, and then descendingly outputting search results according to the size order of the Hamming distance. When the image search method is utilized, quantification loss of the hash codes can be lowered, semantic gaps can be shortened, discrete solutions can be obtained, and then the accuracy and efficiency of crossed searching of pictures and texts can be improved.

Techniques for generating cross-modality semantic classifiers and using those cross-modality semantic classifiers for ground level photo geo-location using digital elevation are provided. In one aspect, a method for generating cross-modality semantic classifiers is provided. The method includes the steps of: (a) using Geographic Information Service (GIS) data to label satellite images; (b) using the satellite images labeled with the GIS data as training data to generate semantic classifiers for a satellite modality; (c) using the GIS data to label Global Positioning System (GPS) tagged ground level photos; (d) using the GPS tagged ground level photos labeled with the GIS data as training data to generate semantic classifiers for a ground level photo modality, wherein the semantic classifiers for the satellite modality and the ground level photo modality are the cross-modality semantic classifiers.

In certain embodiments, the invention relates to systems and methods for in vivo tomographic imaging of fluorescent probes and/or bioluminescent reporters, wherein a fluorescent probe and a bioluminescent reporter are spatially co-localized (e.g., located at distances equivalent to or smaller than the scattering mean free path of light) in a diffusive medium (e.g., biological tissue). Measurements obtained from bioluminescent and fluorescent modalities are combined per methods described herein.

The present teaching relates to a method, system, medium, and implementation of processing image data in an autonomous driving vehicle. Sensor data acquired by one or more types of sensors deployed on the vehicle are continuously received to provide different types of information about surrounding of the vehicle. Based on a first data set acquired by a first sensor of a first type of sensors at a specific time, an object is detected, where the first data set provides a first type of information with a first perspective. Depth information of the object is estimated via object centric stereo based on the object and a second data set, acquired at the specific time by a second sensor of the first type of sensors with a second perspective. The estimated depth information is further enhanced based on a third data set acquired by a third sensor of a second type of sensors at the specific time, providing a second type of information about the surrounding of the vehicle.

A multi-modality electronic communication method for communication via end-user selected communication modalities, including (at a server, having access to memory table/s operative to store, for each end user from among a multiplicity thereof, destinations via which the individual end user receives communications via a corresponding plurality of communication modalities): receiving communication/s from Tx client processor/s serving a Tx end user, the communication including content the Tx end user wishes to send to an Rx end user; and an indication of a known destination of an Rx end user, when a first communication modality is employed; extracting the known destination from the communication and retrieving from the table, a retrieved destination other than the known destination via which the Rx end user receives communications via a second communication modality; and sending the content to the Rx end user via the second communication modality to the retrieved destination.

A cross-time and cross-modality inspection method for medical image diagnosis. A first set of medical images of a subject is accessed wherein the first set is captured at a first time period by a first modality. A second set of medical images of the subject is accessed, wherein the second set is captured at a second time period by a second modality. The first and second sets are each comprised of a plurality of medical image. Image registration is performed by mapping the plurality of medical images of the first and second sets to predetermined spatial coordinates. A cross-time image mapping is performed of the first and second sets. Means are provided for interactive cross-time medical image analysis.

The invention discloses a bridge modal parameter intelligent updating method based on a cross-modal confidence criterion (CMAC) matrix, and the method comprises the steps: taking a CMAC matrix as a basis, combining an adaptive convolution operation layer and a full-connection classification layer to form a modal spectrum response intelligent extraction neural network, and carrying out the classification reconstruction of the CMAC matrix; extracting a bridge structure physical modal spectrum response interval; furthermore, establishing an agent model of bridge vibration data power spectral density and modal information intensity based on the initial modal spectral response interval and a finite element model modal parameter theoretical value, so that a maximum modal information spectral response interval is determined, and bridge structure modal parameter identification is carried out accordingly. According to the method, the CMAC matrix and the adaptive convolutional neural network are combined to carry out intelligent analysis and identification of the structural modal parameters, the network training efficiency is high, the response of the weak excitation modal can be well extracted, and the method can be applied to bridge structure health state monitoring to carry out automatic updating of the modal parameters.