314 results about "Image mode" patented technology

Disclosed is a suppression method for a multipath signal of an image mode based on correlation peaks of a satellite baseband signal, including the steps of: constructing a real-world direct-multipath two-dimensional color image mode data set; and building a multipath suppression model of a deep learning network based on a long-short term memory (LSTM) and a self-attention mechanism module, and training the model. In the present disclosure, a satellite signal can be quickly captured without losing the sensitivity of the captured signal, and a Beidou satellite baseband signal can be captured from a signal recorded in the real world, which enables the model to learn complex signal patterns, improving the accuracy and robustness of multipath signal suppression in urban complex scenes.

The invention provides an investment decision-making system and method based on federated learning and multi-modal risk control, and the system comprises a data collection layer which is used for integrating satellite remote sensing data, supply chain Internet of Things data and financial market data; the federal processing layer is used for constructing a distributed training framework based on a FedAvg algorithm and realizing cross-mechanism data fusion and global model parameter updating through homomorphic encryption and differential privacy; the intelligent decision-making layer adopts a TD3 reinforcement learning engine and adopts a dynamic gating mechanism to adjust fusion weights of texts, time sequences and image modalities in real time; and the execution monitoring layer is used for detecting transaction execution deviation in real time through a digital twinning technology, realizing abnormal response by adopting a CUSUM algorithm, and dynamically adjusting a federal learning rate through Bayesian optimization. The key technical defects of data islands, tail risk response delay, insufficient cross-market linkage analysis and the like of a traditional investment system are overcome through an innovatively designed federated learning architecture and a time-varying risk control model.

The invention discloses a large model enhanced cross-modal retrieval method and system and related equipment. The cross-modal retrieval method comprises the following steps: acquiring an image-text pair; based on the image-text pair, obtaining large model enhancement information of the image-text pair, combining an original image, a text and the enhancement information, adopting a multi-branch encoder of a pre-training CLIP model to obtain a plurality of feature vectors, constructing a multi-stage collaborative alignment loss function, and performing collaborative alignment on an image mode and a text mode in a public semantic space; and training a model through the multi-stage collaborative alignment loss function and a pre-constructed training database, and performing retrieval through the trained model. According to the method, collaborative learning is carried out on the image-text features obtained by the encoder, auxiliary semantic enhancement information is added, and collaborative alignment is carried out on the image modality and the text modality in the public semantic space, so that a better retrieval network is trained, and the accuracy of image-text retrieval is improved.

Various embodiments of the present disclosure relate to operator assistance based on operator monitoring. For instance, during long drives, a driver may become drowsy or may not otherwise be alert. As such, particular embodiments have the capability of starting a conversation with the driver based on driver interests and / or detecting that the driver is getting drowsy. In an illustrative example, a Driver Monitoring System (DMS) camera of a vehicle may employ a component that derives pixel-level information showing head nodding, hands dropping, or the like. Based on image pattern characteristics in the image data, particular embodiments generate a score representing an alertness level. A representation of the alertness level can be provided as input to a machine learning model so that the model may generate a suitable natural language or other response, such as starting a conversation with personalized trivia, sending a control signal to honk a horn, or the like.

In some embodiments, an electronic device presents representations of items of content available for playback on the electronic device. In some embodiments, an electronic device presents selectable options for playing an item of content in accordance with the manners in which the item of content is available to be played on the electronic device. In some embodiments, an electronic device presents representations of episodes in a series of episodic content. In some embodiments, an electronic device presents an enhanced preview of content. In some embodiments, an electronic device presents a control panel. In some embodiments, an electronic device switches the active user profile of a device. In some embodiments, an electronic device enters into a picture-in-picture mode.

The invention is applicable to the technical field of artificial intelligence, and provides a cross-modal retrieval method and device, electronic equipment and a storage medium, and the method comprises the following steps: encoding an image modal through an image encoder, encoding a text modal through a text encoder, and generating a vector representation of a high-dimensional space; inputting the vector of the image mode and the vector of the text mode into the same shared self-attention layer for alignment, and extracting features through respective feedforward network layers; performing cross-modal cross fusion processing on the vector of the image modal and the vector of the text modal through a cross attention mechanism to obtain a cross-modal representation vector of the image modal and a cross-modal representation vector of the text modal; and based on the cross-modal representation vector of the image modality and the cross-modal representation vector of the text modality, outputting the similarity of the image modality and the text modality. According to the method, the cross-modal retrieval precision can be improved.

To improve social safety by functioning as a crime prevention buzzer, predicting crimes, and transmitting real-time alerts to nearby pedestrians.SOLUTION: A system comprises: means for functioning as a crime prevention buzzer; preemptive correlation means in which criminal psychology, behavioral psychology, aura, and past crime data (news images(patterns)) are combined; and means for transmitting alerts to mobile devices of nearby pedestrians using information from satellites and big data.SELECTED DRAWING: Figure 1

The invention provides a fluorescence imaging method and device based on light source intensity adjustment and a storage medium, and the method comprises the steps: in a PDD fluorescence mode, when a blue light source is turned on, after blue light is reflected by an observation tissue, filtering a spectrum of a green fluorescence wave band generated by excitation of riboflavin through a notch filter; in the photodynamic white light image mode, red light, green light and blue light emitted by the red light source, the green light source and the blue light source are adjusted through the light compensation module according to preset light power, so that the light is reflected by observation tissue and then passes through the notch filter to be white light, and the light compensation module controls the light power output by the red light source and the light power output by the blue light source to be consistent. The green light output power is higher than the light power of the red light or blue light. By combining the notch filter and the light compensation module, the problem that in the PDD fluorescence imaging process, green fluorescence generated by excitation of riboflavin affects the contrast ratio of PDD fluorescence imaging is solved, and meanwhile color rendition of traditional white light imaging can be guaranteed.

The invention discloses a multi-modal data glaucoma classification method and system, a terminal and a medium, and the method comprises the steps: obtaining a color fundus image and an OCT image in different image modes, and carrying out the advanced feature extraction of the color fundus image and the OCT image based on a first feature extraction module and a second feature extraction module, the mode specificity characteristics of the two modes are obtained; fusing the mode specific features of the two modes based on a shallow feature fusion module to obtain shallow fusion features; inputting the mode specific features of the two modes and the shallow fusion features into a deep feature fusion module for fusion to obtain final fusion features; and based on a multi-layer perceptron classifier, performing classification prediction on the final fusion feature to obtain a classification result, the classification result including no glaucoma, early glaucoma or mid-late glaucoma. According to the method, through two-stage feature extraction, two-stage feature fusion and the classifier, accurate glaucoma classification can be obtained.

The invention discloses a laser SLAM mapping positioning method and system in a dynamic environment. The method comprises the steps that point cloud information and IMU measurement data in the dynamic environment are collected; the data preprocessing module carries out pre-integration by using IMU data so as to remove point cloud distortion; the pose estimation module realizes positioning based on error Kalman filtering by using IMU measurement information and point cloud feature matching; the dynamic point filtering module stores the three-dimensional point cloud in a two-dimensional image mode, judges whether the three-dimensional point cloud is a dynamic point or not based on the change condition of the point cloud at the same pixel, and finally carries out clustering elimination to output the point cloud only containing a static scene; and the loopback detection module extracts descriptors based on the static scene point cloud, and corrects the pose deviation through ICP (Inductively Coupled Plasma) matching. According to the method, the problem that dynamic object interference occurs in the mapping positioning scene is effectively solved, the dynamic points do not interfere with positioning, the map does not contain ghosts generated by moving objects, help is provided for follow-up navigation and the like, and the robustness and accuracy of mapping positioning are improved.

The invention provides a construction state monitoring and risk assessment method and device based on BIM multi-mode conversion, and relates to the technical field of building information models. According to the method, a standardized image mode is generated by analyzing and extracting component information of a BIM model, and a BIM text mode is generated by using natural language description; constructing a graph structure mode based on space and construction logic, and realizing unified alignment and deep fusion of multi-modal data through multi-level modal alignment and a cross-modal attention mechanism to obtain a cross-modal fusion representation which is used for inputting a state recognition model and automatically detecting an execution deviation so as to monitor a construction state; and then introducing a deviation conduction mechanism to quantitatively calculate a comprehensive risk index of the component so as to carry out risk assessment. According to the method, the fusion representation which not only keeps semantic consistency but also conforms to construction logic can be obtained, the abstract cross-modal semantic features are converted into quantifiable and interpretable construction states and risk indexes, and powerful support is provided for intelligent analysis and application in a construction scene.

The invention provides a cross-modal steganography method and device, equipment and a storage medium, and the method comprises the steps: inputting a carrier image and a ciphertext image corresponding to ciphertext text data into a preset encoder, enabling a learnable wavelet transform network and a reversible neural network in the encoder to carry out the forward feature extraction of the carrier image and the ciphertext image, and obtaining a feature extraction result; obtaining a target carrier feature vector corresponding to the carrier image and a target ciphertext feature vector corresponding to the ciphertext image; and generating ciphertext cross-modal steganography result data corresponding to the ciphertext text data based on the target carrier feature vector and the target ciphertext feature vector for network transmission. According to the method and the device, the ciphertext text data of the text mode can be embedded into the carrier image of the image mode, a multi-mode steganography technology is realized, the steganography concealment and robustness of the ciphertext can be effectively improved, and the ciphertext network transmission security and reliability can be further improved.

The invention discloses a gas recognition method based on multi-source information fusion and environmental perception, and the method comprises the steps: constructing a deep feature learning framework of multi-source fusion through combining the spatial response features, time sequence features and external environmental information of gas; the method comprises the following specific steps: preprocessing collected gas data, and respectively extracting features of an image mode, a sequence mode and an environment mode; fusing the image features and the sequence features through a cross attention fusion module, and capturing the space-time correlation of the data; a cross-modal attention compensation module is introduced, so that main-modal gas data adaptively gathers key information in an auxiliary-modal environment, and effective compensation of environmental factors on gas recognition performance is realized; and finally, gas category prediction is performed through a classification decision head. According to the method, the problems that the detection is easily interfered by environmental factors, the stability is poor or the qualification is inaccurate due to the fact that modeling depends on single modal data in the existing gas identification technology are solved.

The invention belongs to the technical field of geological disaster risk assessment, and particularly relates to a slope stability grade identification method and system based on multi-modal deep learning. Comprising the following steps: acquiring a remote sensing image and corresponding parameter data, and respectively preprocessing the remote sensing image and the corresponding parameter data to obtain a high-dimensional image feature vector and a parameter feature vector; performing dynamic fusion of multi-modal information on the high-dimensional image feature vector and the parameter feature vector through a modal-level gating and fine-grained weighting mechanism to obtain a fusion feature; and inputting the fusion features into a preset classifier to obtain probability distribution of slope stability levels, and outputting the class with the maximum probability as a stability level identification result. According to the method, through combined modeling of the image modality and the parameter modality and introduction of an improved gating attention mechanism in a feature fusion stage, a reliable explanatory basis can be provided while high-precision prediction is ensured, so that the engineering applicability and the popularization value of the model are enhanced.

The invention belongs to the technical field of reservoir logging identification, and discloses a method and system for quantitatively extracting gravel parameters of a conglomerate reservoir, and the method comprises the steps: carrying out the image preprocessing, image enhancement and full-hole image processing of imaging logging information, and obtaining the full-hole logging image information; identifying full-borehole logging image information, and determining a conglomerate development depth section; and performing threshold segmentation on the conglomerate development depth section by using two-dimensional wavelet transform to obtain a binary image of conglomerate features. The method provided by the invention has the advantages that 1) the conglomerate development section is automatically determined through intelligent identification of an imaging logging image mode; 2) by adopting a two-dimensional wavelet transform method, imaging logging image segmentation thresholds in different windows can be determined in a self-adaptive manner, and automatic separation of gravels and a matrix background is realized; and 3) further processing the segmented gravel features through an image analysis method to realize effective separation of the bonded gravels.

The invention provides a fluorescence imaging method and device based on light source proportion adjustment and a storage medium, and the method comprises the steps: in a photodynamic diagnosis PDD fluorescence mode, when a blue light source is turned on, after blue light is reflected by tissues, filtering the spectrum of a green fluorescence wave band generated by excitation of riboflavin through a notch filter, the interference of green fluorescence on PDD fluorescence contrast is reduced; in the photodynamic white light image mode, red light, green light and blue light are controlled to be imaged in a stroboscopic mode through the light compensation module, and the proportion of the light power of the red light, the green light and the blue light is controlled, so that the red light, the green light and the blue light are reflected by observation tissue, penetrate through the notch filter and then are synthesized into white light. By combining the notch filter and the light compensation module, the problem that in the PDD fluorescence imaging process, green fluorescence generated by excitation of riboflavin affects the contrast ratio of PDD fluorescence imaging is solved, and meanwhile color rendition of traditional white light imaging can be guaranteed.

The invention provides a thermal barrier coating defect analysis method based on terahertz multi-domain imaging weighted fusion, which comprises the following steps: according to different positions of defects in a thermal barrier coating, respectively intercepting signals of imaging modes of the surface, the interior, the lower interface and the integral structure of the thermal barrier coating from thermal barrier coating signals; calculating a time domain index value and a statistical domain index value in each signal, arranging the index values into a two-dimensional matrix, and drawing a two-dimensional imaging graph; carrying out fusion imaging on the two-dimensional image by adopting a weighted fusion method to obtain a terahertz fusion image; segmenting and identifying defect features in the terahertz fusion image to obtain defect type and size information; according to the thermal barrier coating falling failure judgment criterion, the failure state of the thermal barrier coating is evaluated according to the defect area proportion; according to the method, the terahertz multi-domain imaging graph is adopted to effectively extract the early internal defect characteristics of the thermal barrier coating, the defect characteristics are enhanced through weighted fusion imaging, and finally early internal defect detection and failure evaluation are achieved.

Several embodiments of the present disclosure relate to providing assistance to the operator based on operator monitoring. For example, a driver may become drowsy or otherwise inattentive during long journeys. Therefore, certain embodiments are able to initiate a conversation with the driver based on the driver's interests and / or the detection of drowsiness. In an illustrative example, a camera of a vehicle's driver monitoring system (DMS) may employ a component that derives pixel-level information indicating head nodding, dropping hands, or the like. Based on image pattern properties in the image data, certain embodiments generate a numerical value representing a level of attention.A representation of the level of attention can be provided as input for a machine learning model, so that the model can generate a suitable natural language or other response, such as starting a conversation with personalized trivialities, sending a control signal to honk the horn, or the like.

The invention provides a knowledge graph construction method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a script text and a deductive video corresponding to the script text; constructing a plurality of first nodes of a text mode based on the script text, and constructing a plurality of second nodes of an image mode and a plurality of third nodes of an audio mode based on the deductive video; based on the content information corresponding to the fourth node and the fifth node, determining a semantic consistency score between the fourth node and the fifth node; determining a time sequence synchronization score between the fourth node and the fifth node based on the time sequence information corresponding to the fourth node and the fifth node; in response to the fact that the semantic consistency score is larger than a first threshold value and the time sequence synchronism score is larger than a second threshold value, constructing a first relationship between the fourth node and the fifth node; and constructing the knowledge graph based on the plurality of first nodes, the plurality of second nodes, the plurality of third nodes and the plurality of first relationships.

The invention relates to the technical field of small sample image classification, in particular to a small sample classification method for multi-granularity image-text prototype matching based on a task-driven structure. The method comprises the following steps: constructing a support set and a query set; inputting the two features into a multi-granularity feature set extraction network, and extracting low-level, middle-level and high-level image features by a multi-level image feature extraction network consisting of an image modal input improved ResNet12 and an embedded task-driven mapper selection module; the text mode calls a large language model (LLM) three times successively to progressively generate word-level, sentence-level and paragraph-level semantic descriptions, and a text editor is combined to form multi-view text features; the image and text feature set is input into a semantic graph interactive fusion network, and multi-level information fusion is realized through node and edge construction, cross-modal attention, graph convolution and a gating mechanism; the matching score of each layer is calculated through the fusion features through a multi-granularity prototype matching network, so that classification prediction is carried out, and the performance of the model is optimized by adopting a multi-granularity composite weighted loss function in a training stage.

An image forming apparatus includes a first image forming unit, a second image forming unit, and a control unit configured to control the first image forming unit and the second image forming unit. The control unit performs control so as to allow execution of a first mode and a second mode in which a color gamut of an image to be formed on a printing material is expanded from the first mode, as an image forming mode for forming an image on a printing material. The second mode includes a second single-color mode in which image formation is performed by only the first image forming unit and a second multicolor mode in which image formation is performed by the first image forming unit and the second image forming unit.

The invention discloses an emotion feature fused bimodal forged information detection method, which comprises the following steps of: extracting emotion features, respectively extracting emotion categories of a text and an image through a large language model, respectively giving emotion intensity of the categories, and scoring the emotion; analyzing the emotion consistency of the mixed modal features of the text and the image; the hybrid expert network calculates domain probabilities to which a text mode, an image mode and a hybrid mode belong respectively through a text domain encoder, an image domain encoder and an image text hybrid feature domain encoder based on domain class processing features; the hybrid expert network learns feature information of a text mode, an image mode and a hybrid mode according to the domain information; the AdaIN network adjusts modal distribution, the AdaIN network adjusts feature distribution according to contribution of each modal, the adjusted features are fused and input into the classifier, a final classification result is obtained, and the information detection accuracy is greatly improved.

The invention relates to a circuit board defect identification method and system based on multi-dimensional image data, and belongs to the technical field of data identification processing, and the method comprises the following steps: obtaining synchronous image data of a circuit board to be detected in a plurality of imaging modes; performing space-spectrum joint registration on each modal image to generate a multi-dimensional image cube with a unified coordinate system and pixel alignment; inputting the multi-dimensional image cube into a pre-trained multi-branch heterogeneous fusion neural network; generating a pixel-level defect probability graph by utilizing a defect sensing context decoder, and performing geometric constraint optimization on the probability graph by combining prior information of a circuit board design layout; outputting defect types, positions and confidence coefficients, and establishing an interpretable defect fingerprint database according to the multi-dimensional response characteristics of the defects; the method has the beneficial effects that false defect signals generated by image noise and circuit board surface texture interference can be effectively inhibited, the omission ratio and the false detection ratio are greatly reduced, and pixel-level accurate defect positioning is realized.

The invention belongs to the technical field of underwater sea condition data monitoring, and discloses an underwater online intelligent monitoring analysis system and method. According to the method, an acoustic Doppler current profiler (ADCP) carried by a seabed observation system is used for carrying out seabed flow measurement, the speed and direction of water flow are obtained, and a binocular camera carried by the seabed observation system is used for determining the size of underwater organisms; performing image mode recognition and classification on the underwater creatures by using an image recognition algorithm in the underwater observation system, and recognizing the types and the number of the underwater creatures; transmitting data to a communication module of the virtual anchor system unmanned platform by using the armored cable, performing error control on the data transmitted by the armored cable, and detecting and correcting error information of the data in the transmission process; and judging whether the returned information is successful or not through an information transmission control method in the underwater observation system. According to the invention, the method can achieve the image recognition of Branchiostoma under a complex sea condition, and improves the operation efficiency and safety.

The invention discloses a multi-mode well site video safety analysis method based on attention, and relates to the field of intelligent video monitoring analysis, and the method comprises the steps: obtaining a low-level fusion feature corresponding to a to-be-analyzed well site operation video; obtaining video modal global spatial-temporal features corresponding to the low-level fusion features; acquiring pixel-level fine-grained local features of a single-frame image mode; inputting the global spatial-temporal features of the video modality and the pixel-level fine-grained local features of the single-frame image modality into a multi-modal feature progressive fusion module to obtain multi-modal fusion features; and inputting the multi-modal fusion features into a multi-task decoder module based on the multi-modal features to obtain a safety analysis result of the well site operation video. According to the invention, the safety analysis of the well site video can be accurately and effectively realized, and the accuracy and real-time performance of safety monitoring are improved.

An LMMs-boosted LMC framework system includes a user interface, a multi-modal feature generation module, a multi-modal feature contrast module, and a prediction logic module. The user interface operates in support image mode to process support set images for training, transforming them into text and visual features within a multi-modal support feature pool. In test image mode, the user interface transmits test images to the contrast module, which compares their features with the support pool using visual and textual modalities. The prediction logic module integrates these comparisons to generate a classification index with the most likely classification and confidence score.

The invention discloses an integrated image transmission terminal and method special for electric power based on a high-flux broadband satellite, and the terminal comprises an image collection module which is used for converting image information in a complex environment into an image data set, carrying out the fusion of the information through the image modes of an RGB image, a depth image and the like, and improving the understanding and processing capability of the complex environment; and the quick transmission interface is used for quickly transmitting the processed data to be displayed, and the transmission speed is high. Through the installation mechanism, the purpose of rapidly installing the image transmission terminal equipment is achieved, so that the installation process is simple and convenient, the stability is higher, the installation time is shortened during emergency use, great convenience is brought to a user, images are clearer in the transmission process through a DPM diffusion probability model algorithm, and the image transmission efficiency is improved. Therefore, the terminal equipment can receive useful information conveniently, and the data transmission rate and quality are improved.

The invention discloses a welding seam defect X-ray image intelligent diagnosis and credible traceability method based on multi-modal heterogeneous information collaboration, which is characterized in that a defect analysis network fusing multi-domain feature modeling and graph structure expression is constructed on the basis of bimodal data formed by a welding seam X-ray image and an industry detection standard text. In the image mode, dividing the weld seam image into a plurality of local area units through superpixel segmentation, taking the areas as image nodes, respectively extracting spatial domain, frequency domain, wavelet domain and edge domain features, and constructing a weighted graph structure by combining the spatial adjacency relation and the feature similarity relation between the areas; realizing overall modeling and correlation analysis of weld defect structure information by using a graph convolutional network; in a text mode, feature coding is carried out on an industry detection standard text, and the feature coding is used as an important prior constraint for defect judgment. Collaborative modeling of an image detection result and standard semantic information is achieved through a gating fusion mechanism, a mapping relation between a detection conclusion and a standard term is established, and interpretable expression and result credible traceability of the weld defect diagnosis process are achieved. And a welding seam X-ray film automatic digital acquisition and observation device is adopted in a matched manner, so that stable transmission, positioning observation and high-resolution digital imaging of the industrial ray film are realized, and reliable and consistent image data input is provided for the intelligent diagnosis method. The method is suitable for intelligent defect detection under complex welding seam structures and multi-working-condition imaging conditions, and has high engineering application value and popularization prospect.

The invention provides a flood event multi-mode retrieval method and device based on a semantic intensifier, and relates to the technical field of data retrieval, and the method comprises the steps: carrying out multi-stage semantic enhancement on keywords through a global flood event knowledge base and internet news information included in the semantic intensifier, then quality priority retrieval and preliminary screening are carried out to obtain a to-be-matched multi-source remote sensing image, and corresponding optical and radar image features and context semantic features of a semantic enhancement text are extracted from an image mode and a text mode respectively; and then fusing the features of the image modality and the text modality to obtain a multi-modality fusion feature, mapping to obtain the similarity between the semantic enhancement text and the to-be-matched multi-source remote sensing image, and finally determining a retrieval result of the keyword. According to the method and the device, the problems of low retrieval matching degree and low retrieval efficiency caused by lack of semantics and difficulty in acquisition of event-related space-time ranges in flood event remote sensing data retrieval are solved.

In various examples, an image processing pipeline can switch between different operating or switching modes based on the speed of the ego's movement and / or the active gear (e.g., park vs. drive) of a vehicle or other ego machine using an RGB / IR camera. For example, a first operating or switching mode can be used that toggles between IR and RGB imaging modes at a fixed frame rate or interval when the vehicle is in motion, in a specific gear (e.g., drive), and / or exceeds a threshold speed. In another example, a second operating or switching mode can be used that switches between IR and RGB imaging modes based on detected light intensity when the vehicle is stationary, in park (or disengaged), and / or below a threshold speed.