174 results about "Image mode" patented technology

The invention provides a construction state monitoring and risk assessment method and device based on BIM multi-mode conversion, and relates to the technical field of building information models. According to the method, a standardized image mode is generated by analyzing and extracting component information of a BIM model, and a BIM text mode is generated by using natural language description; constructing a graph structure mode based on space and construction logic, and realizing unified alignment and deep fusion of multi-modal data through multi-level modal alignment and a cross-modal attention mechanism to obtain a cross-modal fusion representation which is used for inputting a state recognition model and automatically detecting an execution deviation so as to monitor a construction state; and then introducing a deviation conduction mechanism to quantitatively calculate a comprehensive risk index of the component so as to carry out risk assessment. According to the method, the fusion representation which not only keeps semantic consistency but also conforms to construction logic can be obtained, the abstract cross-modal semantic features are converted into quantifiable and interpretable construction states and risk indexes, and powerful support is provided for intelligent analysis and application in a construction scene.

The invention discloses a gas recognition method based on multi-source information fusion and environmental perception, and the method comprises the steps: constructing a deep feature learning framework of multi-source fusion through combining the spatial response features, time sequence features and external environmental information of gas; the method comprises the following specific steps: preprocessing collected gas data, and respectively extracting features of an image mode, a sequence mode and an environment mode; fusing the image features and the sequence features through a cross attention fusion module, and capturing the space-time correlation of the data; a cross-modal attention compensation module is introduced, so that main-modal gas data adaptively gathers key information in an auxiliary-modal environment, and effective compensation of environmental factors on gas recognition performance is realized; and finally, gas category prediction is performed through a classification decision head. According to the method, the problems that the detection is easily interfered by environmental factors, the stability is poor or the qualification is inaccurate due to the fact that modeling depends on single modal data in the existing gas identification technology are solved.

The invention belongs to the technical field of geological disaster risk assessment, and particularly relates to a slope stability grade identification method and system based on multi-modal deep learning. Comprising the following steps: acquiring a remote sensing image and corresponding parameter data, and respectively preprocessing the remote sensing image and the corresponding parameter data to obtain a high-dimensional image feature vector and a parameter feature vector; performing dynamic fusion of multi-modal information on the high-dimensional image feature vector and the parameter feature vector through a modal-level gating and fine-grained weighting mechanism to obtain a fusion feature; and inputting the fusion features into a preset classifier to obtain probability distribution of slope stability levels, and outputting the class with the maximum probability as a stability level identification result. According to the method, through combined modeling of the image modality and the parameter modality and introduction of an improved gating attention mechanism in a feature fusion stage, a reliable explanatory basis can be provided while high-precision prediction is ensured, so that the engineering applicability and the popularization value of the model are enhanced.

The invention provides a knowledge graph construction method and device, equipment, a storage medium and a computer program product. The method comprises the following steps: acquiring a script text and a deductive video corresponding to the script text; constructing a plurality of first nodes of a text mode based on the script text, and constructing a plurality of second nodes of an image mode and a plurality of third nodes of an audio mode based on the deductive video; based on the content information corresponding to the fourth node and the fifth node, determining a semantic consistency score between the fourth node and the fifth node; determining a time sequence synchronization score between the fourth node and the fifth node based on the time sequence information corresponding to the fourth node and the fifth node; in response to the fact that the semantic consistency score is larger than a first threshold value and the time sequence synchronism score is larger than a second threshold value, constructing a first relationship between the fourth node and the fifth node; and constructing the knowledge graph based on the plurality of first nodes, the plurality of second nodes, the plurality of third nodes and the plurality of first relationships.

The invention relates to the technical field of small sample image classification, in particular to a small sample classification method for multi-granularity image-text prototype matching based on a task-driven structure. The method comprises the following steps: constructing a support set and a query set; inputting the two features into a multi-granularity feature set extraction network, and extracting low-level, middle-level and high-level image features by a multi-level image feature extraction network consisting of an image modal input improved ResNet12 and an embedded task-driven mapper selection module; the text mode calls a large language model (LLM) three times successively to progressively generate word-level, sentence-level and paragraph-level semantic descriptions, and a text editor is combined to form multi-view text features; the image and text feature set is input into a semantic graph interactive fusion network, and multi-level information fusion is realized through node and edge construction, cross-modal attention, graph convolution and a gating mechanism; the matching score of each layer is calculated through the fusion features through a multi-granularity prototype matching network, so that classification prediction is carried out, and the performance of the model is optimized by adopting a multi-granularity composite weighted loss function in a training stage.

The invention relates to a circuit board defect identification method and system based on multi-dimensional image data, and belongs to the technical field of data identification processing, and the method comprises the following steps: obtaining synchronous image data of a circuit board to be detected in a plurality of imaging modes; performing space-spectrum joint registration on each modal image to generate a multi-dimensional image cube with a unified coordinate system and pixel alignment; inputting the multi-dimensional image cube into a pre-trained multi-branch heterogeneous fusion neural network; generating a pixel-level defect probability graph by utilizing a defect sensing context decoder, and performing geometric constraint optimization on the probability graph by combining prior information of a circuit board design layout; outputting defect types, positions and confidence coefficients, and establishing an interpretable defect fingerprint database according to the multi-dimensional response characteristics of the defects; the method has the beneficial effects that false defect signals generated by image noise and circuit board surface texture interference can be effectively inhibited, the omission ratio and the false detection ratio are greatly reduced, and pixel-level accurate defect positioning is realized.

The invention discloses a welding seam defect X-ray image intelligent diagnosis and credible traceability method based on multi-modal heterogeneous information collaboration, which is characterized in that a defect analysis network fusing multi-domain feature modeling and graph structure expression is constructed on the basis of bimodal data formed by a welding seam X-ray image and an industry detection standard text. In the image mode, dividing the weld seam image into a plurality of local area units through superpixel segmentation, taking the areas as image nodes, respectively extracting spatial domain, frequency domain, wavelet domain and edge domain features, and constructing a weighted graph structure by combining the spatial adjacency relation and the feature similarity relation between the areas; realizing overall modeling and correlation analysis of weld defect structure information by using a graph convolutional network; in a text mode, feature coding is carried out on an industry detection standard text, and the feature coding is used as an important prior constraint for defect judgment. Collaborative modeling of an image detection result and standard semantic information is achieved through a gating fusion mechanism, a mapping relation between a detection conclusion and a standard term is established, and interpretable expression and result credible traceability of the weld defect diagnosis process are achieved. And a welding seam X-ray film automatic digital acquisition and observation device is adopted in a matched manner, so that stable transmission, positioning observation and high-resolution digital imaging of the industrial ray film are realized, and reliable and consistent image data input is provided for the intelligent diagnosis method. The method is suitable for intelligent defect detection under complex welding seam structures and multi-working-condition imaging conditions, and has high engineering application value and popularization prospect.

According to the glaucoma multi-mode auxiliary diagnosis device and the electronic equipment, firstly, first feature extraction and second feature extraction are carried out on an eye fundus image and an OCT image respectively, then semantic spaces of an eye fundus image mode and an OCT image mode are aligned by utilizing comparison loss, and the first feature and the second feature are fused by utilizing a cross attention mechanism, so that an eye fundus image is obtained. According to the method, the two modal data are subjected to fusion to obtain fusion features, then the fusion features are subjected to feature extraction to obtain third features, and glaucoma classification judgment is performed based on the third features, so that global modeling of the two modal data can be realized, effective features are extracted to perform glaucoma classification judgment, and the accuracy and timeliness of diagnosis are ensured.

The invention discloses a large model image segmentation method and system based on multi-scale fusion and a medium, and relates to the technical field of image processing. Obtaining an image segmentation model; the structure construction mode of the image segmentation model is that a traditional SAM is used as a network basis, an Adapter module is inserted between a multi-head attention layer and a multi-layer perceptron in each VIT block of an image encoder of the SAM, and a four-layer CBR-Net module is arranged at the end of the image encoder in parallel. The output end of the image encoder and the back of an up-sampling layer of a mask decoder of the SAM are respectively connected with an Atte-FFB module in series, and residual connection is constructed between the first three-layer output of the CBR-Net module and the Atte-FFB module of the corresponding layer in the mask decoder of the SAM network; residual connection is constructed between the fourth layer output of the CBR-Net module and an Att-FFB module behind the output end of the image encoder; and performing image segmentation on the to-be-segmented image through the image segmentation model. According to the method, the segmentation precision is improved, and the model generalization ability of a cross-image mode is enhanced.

The invention discloses an infrared and visible light image fusion method and system based on text-guided semantic perception, and relates to the technical field of image fusion. A text prompt is obtained by using a reference image subjected to semantic segmentation, and the features of a source image are respectively obtained by using encoders with the same structure and independent parameters. A semantic correspondence alignment module is provided between an encoder and a decoder, and rich feature representations related to text features are obtained, so that the difference between a text mode and an image mode is reduced. In a decoder stage, a semantic modulation module is provided, interaction between a text and an image is fully realized, effective combination of texture details and advanced semantic information is realized in combination with shallow layer features, and finally a fused image with rich semantics and complete details is generated. According to the method, the overall quality of the fused image is remarkably improved in the aspects of semantic consistency and perception quality.

The invention discloses a tea quality evaluation method based on spectral image bimodal fusion. The tea quality evaluation method comprises the following steps: acquiring hyperspectral data of different grades of tea; preprocessing the tea hyperspectral image data; acquiring spectrum and image modal information of the tea hyperspectral image; a spectrum-image dual-mode fusion neural network (SIFNet) model is established, and the SIFNet adopts a dual-branch network architecture. A spectrum feature extraction branch is combined with a one-dimensional convolutional neural network and a bidirectional long-short-term memory network module to extract time sequence dependence features of a spectrum mode; an image feature extraction branch introduces a two-dimensional convolutional neural network and a fuzzy logic processing module to extract accurate features and fuzzy features of an image modal, and advantage complementation of multi-modal information is realized through a feature depth fusion technology; the SIFNet model is trained; and evaluating the quality of the tea to-be-detected sample. Through deep coupling and complementary enhancement of the spectrum-image bimodal features, the tea quality grading accuracy is improved.

The invention discloses a joint semantic segmentation method for a camera and a laser radar in an off-road environment, and relates to the technical field of computer vision. According to the method, the 2D image and the 3D point cloud data are respectively processed by constructing the double-branch basic segmentation network, so that efficient processing and fine segmentation of the point cloud are realized; a multi-scale one-way knowledge distillation framework is designed, feature fusion of a common-view area is realized through a channel self-attention mechanism, and knowledge migration from an image mode to a point cloud mode is realized by adopting a dynamic temperature adjustment and Logit standardization technology; in the training stage, multi-modal data is utilized, and high-precision segmentation can be completed only through point cloud input in the reasoning stage. According to the method, the semantic segmentation precision and robustness in the cross-country environment are remarkably improved, the segmentation performance of the occlusion region and the sparse feature region is effectively improved, and meanwhile, the computing resource consumption is greatly reduced.

The invention discloses a brain load assessment method and system based on hemodynamic information and electroencephalogram signals, and relates to the field of brain-computer interface classification systems. The technical problem that how to fuse electroencephalogram signals and hemodynamic information and comprehensively utilize complementary characteristics of the electroencephalogram signals and the hemodynamic information in time and space dimensions to achieve accurate recognition and dynamic evaluation of the mental load state in the prior art is urgently needed to be solved in the current brain-computer interface field is solved. The method comprises the following steps: acquiring EEG data and fNIRS data of a testee; performing preprocessing, selecting a suitable frequency band, removing interference such as high-frequency noise and low-frequency baseline drift, and constructing a corresponding training data set; respectively constructing an EEG single-mode network and an fNIRS single-mode network which are used for extracting related characteristics; and constructing an EEG-fNIRS multi-mode classification network for fusing data of brain imaging modes in the EEG single-mode network and the fNIRS single-mode network, and performing classification evaluation on the hemodynamic information of the subject and the brain power load data of the electroencephalogram by using the EEG-fNIRS multi-mode classification network.

A question and answer data construction method comprises the following steps: 1) reading samples containing image and text data in batches, constructing a unified instruction for each batch of samples, and calling a large language model to extract candidate questions which have discriminability and teaching value and can be answered from images to form multiple groups of candidate questions; (2) carrying out structured deduplication merging on the cross-batch candidate problems obtained in the step (1) to obtain a deduplication problem pool; 3) for each sample, identifying an available image mode of the sample, screening candidate problems matched with the mode from the problem pool, and randomly extracting candidates of which the number is greater than a target value q; 4) constructing a request only based on image answering, calling the multi-modal large language model by taking the sample image as input, and generating a question-answer pair; if reliable answers of the candidate questions cannot be obtained only through the images, replacing the candidate questions with new candidates and retrying until q question and answer pairs capable of being answered are accumulated and obtained; 5, the generated question and answer pairs are subjected to structured verification and safe disking.The method supports parallel processing and breakpoint continuous running so as to meet the large-scale data construction.The method has the advantages of being good in universality and expandability and standardized in result.

The invention provides an apparatus for providing guidance to a user of an ultrasound acquisition system to acquire standardized images of a target anatomical structure of a patient. A processing unit is configured to toggle a user interface between two modes: a guidance mode in which only guidance information is presented on the user interface, and an imaging mode in which a received ultrasound image is presented on the user interface. The processing unit toggles the user interface between these two modes based on a spatial relationship between a current probe position and a target view. When the ultrasound acquisition system is within a threshold distance of the target view then the processing unit toggles the user interface to display the imaging mode, otherwise the guidance mode is displayed.

The invention discloses a text and image multi-mode user interest identification method and system oriented to private domain communities. The system comprises a text feature extraction module, an image feature extraction module, a multi-modal fusion module, an interest probability aggregation module and a dynamic threshold judgment module. The multi-modal fusion module adopts a gating mechanism to realize self-adaptive fusion, a gating factor is automatically adjusted according to the characteristics of an input sample, and weighting is carried out between a text mode and an image mode, so that the robustness of a fusion result can still be ensured when the quality of information in different modes is unbalanced. And the interest probability aggregation module performs time sequence smoothing on the instant interest probability through an exponential weighted moving average method, and inhibits short-time noise interference. And the dynamic threshold value judgment module is used for calculating a dynamic threshold value according to the mean value and the standard deviation of the recent window, comparing the smoothed interest probability with the dynamic threshold value, and judging that the corresponding interest label is activated when the smoothed interest probability is greater than or equal to the dynamic threshold value. According to the method, the user interests can be accurately recognized in a complex and changeable private domain community environment, and the adaptability, stability and real-time performance of interest recognition are achieved.

The invention discloses a light field time domain super-resolution method based on an event camera, and belongs to the field of optical engineering. A light field camera and an event camera are used for collecting light field event data, decoding the light field data, denoising the event data and generating an event image, the light field data and an event grayscale image are subjected to joint calibration in a sub-aperture image mode, joint optimization is carried out through calibration results of multiple viewpoints, a checkerboard is shot, and a target object is obtained. And obtaining an affine transformation matrix and a perspective transformation matrix, and obtaining a super-resolution result of projecting the event image onto a light field image plane. Starting from the expression mode of the light field macro pixel, the method has applicability to light field data of any macro pixel size; compared with a traditional time domain super-resolution method, the time domain super-resolution of the light field camera at any frame rate can be realized by segmenting the event stream at different scales.

A charged particle detector includes a plurality of sensing elements, with each sensing element being further divided into sub-sensing elements. The sub-sensing elements may be individually addressed during high-resolution image acquisition in a picture mode, and may be grouped together during high speed detection in a beam mode. The arrangement allows a selectable tradeoff between speed and resolution without introducing significant parasitic parameters.

Embodiments of the present application provide a game data processing method and device, electronic equipment and readable storage medium, the method comprises: in the game process, the terminal can obtain game resources, wherein the game resources can include scene resources, model resources and obstacle information for scene resources, model display parameters for model resources, then the corresponding game scene of the scene resources can be displayed in the form of two-dimensional image mode, and the corresponding game model of the model resources can be displayed in the form of three-dimensional image mode in the game scene according to the model display parameters, then the game scene can be processed according to the obstacle information, the corresponding obstacle map is generated, and the moving information of the game model is obtained, and then the target moving path of the game model in the obstacle map is determined according to the moving information, so as to control the game model to move in the game scene.

An automobile includes an entertainment execution device, a display device and a mode select controller. The entertainment execution device executes entertainment software, communicates with radio control equipment, or plays back a video content. The display device displays an image output from the entertainment execution device on a windshield or on a flexible display capable of being raised or lowered between a front windshield and a steering wheel. The mode select controller enables selection between an entertainment mode for operating the entertainment execution device and a driving mode for driving the automobile and executes control according to each mode. The mode select controller controls the display device to display the image output in a first region of the windshield in the driving mode, and to display the image output in a second region of the windshield or on the flexible display in a raised state in the entertainment mode.

A Virtual Reality (VR)-based seat control apparatus for a vehicle includes: an input unit for receiving an image mode operation command from a user; and a control unit for playing a prepared timetable according to the image mode operation command, performing interlocking with a seat module of a vehicle and VR equipment, and controlling an operation module of the seat module according to the timetable.

The present disclosure relates to a panoramic image display system, method, vehicle, electronic device and storage medium, and relates to the field of vehicle design, and the system comprises: in the case of starting a panoramic image function and a UAV image auxiliary function, controlling a UAV to take off from a vehicle and fly to a first target position, and acquiring a UAV image from the UAV, the UAV image being acquired by the UAV from the vehicle at the first target position according to an image mode of the UAV image auxiliary function at present, and according to a display mode of the panoramic image function at present, controlling the display screen to display a panoramic image containing the UAV image, through the UAV image auxiliary function, the target position required to acquire the image is acquired by the UAV, and the UAV image is displayed on the display screen, which can reduce image distortion and image blind area, and can provide effective image assistance for users.

The invention relates to an air component analysis method and system in an airport environment, and the method comprises the steps: continuously collecting the wind field motion in the airport environment through a laser wind measurement radar in a preset time period, and obtaining an initial wind field motion point cloud data set; performing wind field displacement vector calculation on each wind field motion point cloud data in the set to obtain a wind-point-free cloud data set; constructing a corresponding triple for each piece of windless point cloud data in the set; and performing normalization and visualization processing on the triple to generate image data, and analyzing the image data through an air component classification model to obtain air components of the airport environment. By means of the air component analysis method and device, the wind field state which changes severely in the airport environment is converted into the windless state, the emission characteristics of pollution sources are reflected more truly, large-range air component analysis is conducted in a triple-image mode, and the problem of how to improve the applicability of air component analysis in the large-range environment is solved.

A multi-modal combined ultrasound imaging method and ultrasound imaging system, wherein: in response to a first modal imaging instruction, entering a single modal imaging mode of a first modal; in the single modal imaging mode of the first modal, performing ultrasound imaging of the first modal and displaying an ultrasound image of the first modal on a first ultrasound interface; displaying a multi-modal control on the first operation interface; in response to a first operation on the multi-modal control, entering a multi-modal combined imaging mode; in the multi-modal combined imaging mode, performing multi-modal combined ultrasound imaging and displaying one or more ultrasound images corresponding to the multi-modal on a second ultrasound interface. Since the multi-modal combined ultrasound imaging is directly entered through the multi-modal control, it is not necessary to combine different modalities in different levels of functions, greatly improving the efficiency of multi-modal combination and giving users a better use experience.

In order to confirm the activity state of the pelvic floor muscle in practice of the pelvic floor muscle exercise, a sensor sheet is attached to the abdominal region corresponding to the transversus abdominis muscle, and an electromyogram of the transversus abdominis muscle which contracts in cooperation with the pelvic floor muscle is acquired. At this time, even if the transversus abdominis muscle contracts, the pelvic floor muscle does not necessarily contract, and thus the activity state of the pelvic floor muscle is confirmed by echo. In more detail, an ultrasonic sensor is pressed against the abdomen in a manner that an ultrasonic probe is placed on the sensor sheet, and thus an image (M-mode image) of an ultrasonic echo representing the activity state of the bladder bottom which can be considered to be the same as the contraction of the pelvic floor muscle is acquired. Thereby, the implementer of the pelvic floor muscle exercise can experience the knack of contracting the pelvic floor muscle, that is, what kind of consciousness to focus on which part of the body can contract the pelvic floor muscle.

The application belongs to the field of optical storage, and specifically discloses an optical disc data reading method and device. The optical disc involved in the application comprises a plurality of data layers, each data layer comprising a plurality of data blocks. The optical disc data reading method comprises the following steps: focusing imaging light to a preset data layer of the optical disc; obtaining images of each data block in the preset data layer based on the imaging light when the optical disc rotates; controlling the imaging light to stably focus on the preset data layer according to the brightness and the definition of each data block image, and then reading corresponding data information based on the images of each data block. According to the application, the defocus distance and direction can be accurately predicted in combination with the definition evaluation function, and real-time focus control can be performed in the process of continuous rotation of the optical disc. Data is read in a pure image mode, and focus servo control can be completed without making a data recording layer with a track in the optical disc, so that reading of hundreds of layers of data in a multi-dimensional storage optical disc can be realized.

The invention provides an image processing method, electronic equipment, a computer program product and a storage medium, and relates to the technical field of terminals. And the electronic equipment displays a shooting preview interface, and the shooting preview interface displays the preview image frame by frame. The electronic equipment analyzes whether the preview image is matched with the multi-modal scene information of the target scene or not in the shooting preview process, a target image matched with the multi-modal scene information of the target scene is found out, and the target image obtains a snapshot image matched with the target scene. When the electronic equipment snapshots a target scene, the used multi-modal scene information of the target scene comprises scene reference information of an image modal and a scene prompt statement of a language modal. The electronic device can automatically capture the captured image of the corresponding target scene without depending on the pre-judgment of the user on the wonderful scene and the capture operation of clicking a shutter. Automatic snapshot is carried out based on the multi-modal scene information, and the scene matching degree of the snapshot image is higher.

A device for uniformly feeding and paving granular crops comprises: a uniform feeding silo for controlling the speed and flux of granular crops during feeding and conveying, and for automatically paving, dispersing, and evenly distributing the crop particles; a linear vibration module disposed at the bottom of the uniform feeding silo; a uniform speed perturbation module disposed within the uniform feeding silo; a visual monitoring module disposed at the outlet of the uniform feeding silo; a material conveying module; an image acquisition module; and a computer system. The present invention uniformly spreads and disperses large quantities of granular crops within the imaging field of view of a machine vision module, and adjusts the feeding speed and flux in real time according to changes in the density of the crop particles, so that the sample particles form an easily detectable image pattern and maintain the spatiotemporal uniformity of the feeding, thereby utilizing the detection and recognition algorithm combined with the device to perform efficient and intelligent appearance quality and quarantine testing on large quantities of granular crops. This is a key technical issue that urgently needs to be addressed in the field of food security.

The application discloses a multi-modal imaging system and method based on an optical low-coherence interference field. The application can realize efficient interference signal collection and imaging under various OCT imaging modes in one set of system and imaging mechanism, and can realize mapping of imaging coordinate systems of various modes through linear transformation by combining the OCT axial imaging characteristics and the scanning setting under the same galvanometer mirror, so that the fusion display of multi-dimensional information in tissues can be completed without complex offline registration. The application can provide wide-area positioning imaging in a large depth range, structure imaging with high resolution and high signal-to-noise ratio, blood flow velocity detection and angiography with a large dynamic range, fast elastic imaging, and can also provide a real-time structure imaging mode with a large lateral field of view, so that a user can observe and operate according to the above information in a dynamic scene. The application has great significance for biomedical application scenes with multi-tissue information requirements.

A stovetop camera application device is disclosed, which is a stovetop Wi-Fi enabled camera system and corresponding software application. The stovetop camera application device comprises a heat-resistant camera component with a wide-angle lens that operates in a normal image mode or a heat camera mode. The camera component is installed above a stovetop hood or is wall mounted via magnets and angled brackets. The camera component includes sensors to detect methane gas or smoke. The corresponding software application has customizable features, such as alarms for flame detection, steam alerts, boiling notifications, and timers to enhance functionality and safety.