Patents
Literature
Patsnap Eureka AI that helps you search prior art, draft patents, and assess FTO risks, powered by patent and scientific literature data.

1269 results about "Image code" patented technology

Machine and human vision-oriented image coding and decoding method and compression method

The invention provides an image coding and decoding method and a compression method for machine and human vision, and the method mainly comprises the steps: obtaining a first feature map and a second feature of a to-be-coded image, and obtaining the super-prior information of the first feature map; obtaining quantization step sizes of the to-be-coded image under different machine vision tasks, and performing quantization to obtain a corresponding feature map; modeling the feature map into Gaussian distribution, and predicting a corresponding mean value and variance through an entropy model; coding layer by layer according to the distribution and probability of each layer of feature map to obtain a feature map binary code stream; and combining the super-prior information binary code stream and the feature map binary code stream to obtain a compressed image binary code rate. According to the method, the quantization step size of feature map quantization is adaptively adjusted, transmission feature code streams for human vision and different machine vision tasks are formed, information in the feature map can be more efficiently utilized, the transmission code rate is reduced, meanwhile, the performance of the machine vision tasks is guaranteed, and meanwhile, the rate distortion performance oriented to human vision is balanced.
Owner:SHANGHAI JIAOTONG UNIV

Image optimization method based on CLIP model

The invention discloses an image optimization method based on a CLIP model, and the method comprises the steps: S1, carrying out the coding initial processing of characters and images, inputting a character description and an initial image code into a CLIP model, and obtaining a corresponding semantic vector and a visual vector; s2, cross-modal contrast learning: based on positive and negative sample contrast and an adaptive attention mechanism, aligning the semantic vector and the visual vector obtained in the S1; s3, self-supervised learning is carried out, and precision improvement of semantic-to-visual mapping is completed through self-supervised learning; s4, feature refinement generation: based on the optimized semantic and visual feature vectors, generating a preliminary concept conforming to description; and S5, image output: repeating the steps S1 to S4 until a preset matching value is met, and outputting a conception image matched with the text description. According to the image optimization method based on the CLIP model, semantic mapping from characters to images is improved, so that the semantic mapping is more in line with the expectation of natural language description, and a visual conception graph with rich details is generated.
Owner:TIANFU JIANGXI LAB

Mathematical problem solving method and device based on multi-modal large model and electronic equipment

The invention provides a mathematical problem solving method and device based on a multi-modal large model and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a mathematical element image of a mathematical problem; inputting the mathematical element image into an image coding model to obtain an image vector output by the image coding model; the image coding model is obtained by training based on a sample mathematical element image and a positive sample text description and a negative sample text description corresponding to the sample mathematical element image; inputting the image vector into a self-adaptive module to obtain an image conversion coding vector output by the self-adaptive module; the adaptive module is obtained based on training of a sample image vector and a sample text vector; determining question stem characters of the mathematical question, and inputting the question stem characters and the image conversion coding vector into the large language model to obtain a prediction answering process output by the large language model; the large language model is obtained based on sample question stem characters, sample image conversion coding vectors and sample answering process training, and the mathematical problem solving capability of the multi-modal large model can be improved.
Owner:TSINGHUA UNIVERSITY

Moving image decoding device and moving image coding device

To provide a moving image coding and decoding device capable of improving a coding efficiency.SOLUTION: A moving image decoding device comprises a parameter decoding part that decodes a syntax (mts_idx) of an object block from a coding data. The parameter decoding part is characterized that the mts_idx is decoded in accordance with a block size of the object block, a LAST position, and a non-zero coefficient position. The parameter decoding part determines whether or not the mts_idx is decoded or not by using a threshold value of the LAST position. The threshold value is characteristic that, in the case where the block size is a predetermined size or larger, the block size is larger as compared with the case where the block size is smaller than the predetermined size.SELECTED DRAWING: Figure 15
Owner:SHARP KK

Underwater image enhancement method and system based on multi-modal degradation feature learning

The invention relates to the technical field of image processing, and provides an underwater image enhancement method and system based on multi-modal degradation feature learning. The method comprises the following steps: collecting underwater image data of different modes to construct an underwater image data set, and dividing the underwater image data set into a training set and a test set; preprocessing the training set to obtain an image text degradation pair data set; constructing a multi-modal visual language MD-CLIP network framework, and training the MD-CLIP network framework by using the image text degradation pair data set; the MD-CLIP network framework comprises an image coding controller ContNet module, a pre-trained CLIP module and a comparative learning module; integrating the trained image coding controller ContNet module into an NAFNet network structure, generating an image enhancement network, and training by using the test set to obtain an underwater image enhancement model; and inputting a to-be-enhanced underwater degraded image into the underwater image enhancement model to generate an enhancement result graph of the to-be-enhanced underwater degraded image.
Owner:HENAN UNIVERSITY

Image coding method and device based on deep learning

The invention provides an image coding method and device based on deep learning, and belongs to the field of image coding, and the image coding method comprises the steps: a coding end uses nonlinear transformation to project an original image to a hidden space; the coding end performs probability distribution estimation by using a causal context; the encoding end encodes the hidden variables by using an entropy encoding technology; the decoding end performs probability distribution estimation by using a causal context; the decoding end decodes the hidden variables by using an entropy coding technology; the coding end reconstructs the hidden variables into an original image by using nonlinear transformation; and performing end-to-end reverse gradient propagation optimization on an image coding and decoding process by using a distortion loss function and a causal context adjustment loss function. By using a better causal context to adjust a loss function and a more efficient network structure, better image compression capability and higher rate distortion performance are obtained.
Owner:UNIV OF ELECTRONICS SCI & TECH OF CHINA

Virtual face generation method and system, electronic equipment and storage medium

The invention provides a virtual face generation method and system, electronic equipment and a storage medium, and the method comprises the steps: inputting a to-be-processed face image, and audio features and face key point features corresponding to the to-be-processed face image into a coding module in a face generation model, and obtaining image coding features and lip motion coding features; inputting the image coding features and the lip motion coding features into a feature fusion module in a face generation model to obtain fusion features; inputting the fusion features into a decoding module in a face generation model to obtain a virtual face image; wherein the face generation model is obtained by training an initial image generation model, the parameter scale value of which is smaller than a preset value, based on a sample real face image, a mask face image corresponding to the sample real face image, and sample audio features and sample face key point features corresponding to the mask face image. According to the invention, the virtual face generation quality is guaranteed, and the virtual face generation efficiency is improved.
Owner:IFLYTEK CO LTD

Remote sensing target detection method and device based on diffusion model

The invention relates to the technical field of target detection, and provides a remote sensing target detection method and device based on a diffusion model. The method comprises the following steps: step 1, acquiring a remote sensing image data set, and preprocessing the remote sensing image data set; 2, training an image coding module based on the preprocessed remote sensing image data set; the image coding module is used for extracting multi-scale features of a remote sensing image to obtain a multi-scale feature map; 3, constructing an initial noise frame through a variable variance scheduling method, and training a detection decoding module based on the initial noise frame and the multi-scale feature map extracted by the image coding module; the detection decoding module performs noise prediction on the initial noise frame and recovers a target frame to complete target detection; and 4, forming a remote sensing target detection model by using the trained image coding module and the detection decoding module, wherein the remote sensing target detection model is used for performing target detection on the remote sensing image.
Owner:HENAN UNIVERSITY

Crack image processing method based on neural network and plane geometry image transformation

The invention discloses a crack image processing method based on a neural network and plane geometry image transformation, and particularly relates to the field of general image data processing driven by plane geometry image transformation and neural network identification. Comprising the steps that an image collection platform projects a structured image coding pattern to the surface of a to-be-measured target through a preset image coding pattern template, and synchronously collects an image frame containing an image mark so as to construct a space anchor point used for image geometric reference in an image plane; view angle parameter change information in an image acquisition process is acquired through an image acquisition platform, and projection configuration parameters of an image coding pattern are combined. By constructing an image offset modeling mechanism jointly driven by an image coding pattern and a space anchor point, image in-plane correction of geometric distortion features in an image frame is realized, and structural consistency and texture continuity before subsequent image analysis are improved.
Owner:UNIV OF JINAN +1

Image encoding and decoding method, device and storage medium

The present application provides an image encoding and decoding method, device and storage medium, which relates to the field of video encoding and decoding. The encoding method includes: obtaining an encoding unit; the encoding unit is an image block in the image to be processed; the encoding unit includes encoding blocks of multiple channels; the multiple channels include a first channel; the first channel is any one of the multiple channels; the encoding block of the first channel is encoded according to a first encoding mode; the first encoding mode is a mode for encoding sample values ​​in the encoding block of the first channel according to a first fixed-length code; the code length of the first fixed-length code is less than or equal to the image bit width of the image to be processed; the image bit width is used to represent the number of bits required to store each sample in the image to be processed. The method is applicable to the image encoding and decoding process, and is used to solve the problem of too large a substream buffer setting.
Owner:HANGZHOU HIKVISION DIGITAL TECHNOLOGY CO LTD

Optical flow estimation method and system fusing Mama and visual basis model knowledge

The invention belongs to the technical field of computer vision and deep learning, and particularly relates to an optical flow estimation method and system fusing Mama and visual basic model knowledge. The method comprises the following steps: performing down-sampling feature extraction on two adjacent frames of input images by using a convolutional neural network to obtain local texture features; performing down-sampling on the first frame image to obtain context features; meanwhile, extracting global semantic features of two adjacent frames of images by using a pre-trained visual model, and performing adaptive fusion enhancement through an adaptive semantic texture feature fusion module to obtain an image coding feature pair after semantic enhancement; constructing a related volume through pixel-by-pixel dot product operation; and finally, based on the obtained related volume and context features, iteratively optimizing the output optical flow through a loop iteration updating module. The method solves the problems that in a low-texture, repeated-texture or sheltered area, feature expression is unstable, self-adaptive modeling capacity is lacked, different scenes are difficult to generalize, and model performance and efficiency cannot be balanced.
Owner:QILU UNIVERSITY OF TECHNOLOGY (SHANDONG ACADEMY OF SCIENCES) +1

Cross-layer image text interaction prompt learning method based on attention mechanism

The invention relates to the technical field of multi-mode prompt learning, and discloses a cross-layer image text interaction prompt learning method based on an attention mechanism, and the method comprises the steps: obtaining text data and image data; constructing a cross-layer image text interaction prompt learning network model; converting the text data into word vectors by utilizing a word embedding module; performing multi-level attention coding on the word vector and the text prompt vector by using a text encoder network to obtain a text code; a cross-layer interaction prompt learning module is used for generating an image prompt vector for the text prompt vector by adopting a multi-head attention mechanism; converting the image data into block feature maps by using a map segmentation module; performing multi-level attention coding on the block feature map and the image prompt vector by using an image encoder network to obtain an image code; and carrying out multi-modal contrast learning on the text codes and the image codes by using a multi-modal contrast learning module. According to the invention, the accuracy of multi-modal learning task execution can be further improved.
Owner:UESTC (SHENZHEN) ADVANCED RES INST

Pathological section image analysis method and device based on large language model

The invention provides a pathological section image analysis method and device based on a large language model, and belongs to the field of computer vision, deep learning and medical image analysis. The method comprises the following steps: constructing an image coding large model which is composed of a pre-training model and an additionally-added task head and aims at a pathological section; freezing the pre-training model, training a task head by using a preset training set, and checking by using a preset verification set to obtain a task head after preliminary training; unfreezing the pre-training model, performing fine tuning training on the large model by using the training set, and performing inspection by using the verification set so as to obtain a finally trained large model; and inputting a to-be-analyzed pathological section image into the large model, and calculating corresponding features of the image based on a result output by the large model to obtain a final analysis result. According to the method, the features in the pathological section image can be automatically extracted, and the diagnosis report is generated based on the features, so that the accuracy and efficiency of pathological section image diagnosis are improved.
Owner:TSINGHUA UNIVERSITY

Information processing method and system based on cellular pixel structure

The invention discloses an information processing method and system based on a honeycomb pixel structure, and the method comprises the steps: an encoding step: encoding a positioning graph and input information to form an encoding matrix based on regular hexagonal pixels, the encoding matrix comprising a positioning graph region and an effective information region; a masking step, at least performing XOR operation on the effective information area of the coding matrix and different mask matrixes to form different cellular codes, and comparing the different cellular codes to obtain an optimal cellular code; the mask matrix is also a matrix based on hexagonal pixels; and a decoding step: decoding the optimal cellular code to obtain effective information. According to the method, the cellular codes can be quickly, accurately and efficiently coded and decoded, and a new choice is provided for application of an image coding technology.
Owner:ZHEJIANG DAOMING OPTOELECTRONICS TECH

Visual image compression, decompression and transmission method and device for underwater robot

The invention discloses an underwater robot visual image compression, decompression and transmission method and device. The method comprises the steps of constructing an image compression model used for underwater robot visual image compression, training the image compression model, constructing a visual image coding end based on the trained image compression model, performing compression coding on the underwater robot visual image by using the visual image coding end, generating code stream data, and outputting the code stream data. And constructing a visual image compression end based on the trained image compression model so as to complete compression of the visual image of the underwater robot, and constructing a visual image decoding end based on the trained image compression model, the visual image decoding end being used for decoding the code stream data and reconstructing the image so as to generate a reconstructed visual image of the underwater robot. The underwater robot visual image compression and decompression method provided by the invention is high in compression ratio and good in image reconstruction performance, and can be applied to compression and transmission tasks of underwater robot visual images.
Owner:ZHEJIANG UNIV OF TECH

Image filling method and device, electronic equipment and storage medium

The invention relates to the technical field of image processing, and provides an image filling method and device, electronic equipment and a storage medium, and the method comprises the steps: encoding a text prompt word to obtain a text encoding vector, and encoding a masked background image to obtain an image encoding vector; splicing the image coding vector, the position mask and the random noise to obtain a fused noise vector; and step-by-step denoising is carried out on the fused noise vector, a text coding vector and a texture guide vector are added in the denoising process as guide conditions, a filling image is generated, and the texture guide vector is obtained by carrying out splicing, smooth transition and coding processing on texture features of the foreground image and texture features of the background image. According to the method, the filling image which is rich in details, natural in texture and highly fused with the background image can be generated through a step-by-step denoising process and combining the text coding vector and the texture guide vector as double guide conditions.
Owner:IFLYTEK CO LTD

Image enhancement method and system of spatial vision neural network model

ActiveCN119494789ASatisfy direction selectivityImprovements to difficult to filter outImage enhancementImage analysisPattern recognitionIdeal image
The invention relates to an image enhancement method for constructing a spatial vision neural network model. The method comprises the following steps: firstly, carrying out normalization processing on a binarized noise-containing two-dimensional image to enable the binarized noise-containing two-dimensional image to meet small parameter limitation brought by activation of an FHN neuron adiabatic approximation condition; secondly, the space signal is subjected to convolution through an FHN neuron template, and a space neuron pulse potential sequence two-dimensional image coding signal is obtained; constructing an anisotropic spread function based on improved bidirectional adaptive selection as a visual receptive field model with a direction selection mechanism, and carrying out convolution on a neuron pulse coded signal and the constructed receptive field model; and finally, an optimal binary image result is found. According to the method, the direction selectivity and the neuron pulse issuing mechanism of the biological vision system are effectively combined, so that the biological vision system conforms to the physiological characteristics of real vision neuron pulse triggering and signal selectivity, and the ideal image denoising effect is achieved.
Owner:WESTLAKE INSTITUTE FOR OPTOELECTRONICS

Voice interaction method, server and readable storage medium

The invention discloses a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program. The method comprises the following steps: receiving a current round voice request forwarded by a vehicle and current environment image coding information of a cabin; and determining target input information according to the current environment image coding information of the cockpit, the current round of voice request, the historical round of voice request and the natural language understanding result of the historical round of voice request. And according to the target input information, determining an intention label of the current round of voice request. And completing voice interaction according to the intention label. Therefore, the target input information is determined by combining the current environment image coding information of the cabin, the current round of voice request, the slot position identification result, the historical round of voice request and the understanding result thereof, intention identification is performed on the target input information, the weak intention voice request of the user is accurately identified, misunderstanding of the user request is reduced, and the user experience is improved. And the probability of error response is reduced, so that the user experience is improved.
Owner:GUANGZHOU XIAOPENG MOTORS TECH CO LTD

Image coding method and device

The embodiment of the invention provides an image coding method and device, and relates to the technical field of image coding and decoding. The method includes: acquiring potential features of a target image; performing quantization processing on the potential features based on a preset quantization parameter to obtain quantized potential features; processing the quantized potential features based on a channel attention mechanism to obtain channel attention masks of the potential features; dividing the potential features into basic layer potential features and enhancement layer potential features according to the channel attention masks of the potential features; and generating coded data of the target image according to the basic layer potential features and the enhancement layer potential features. Some embodiments of the invention are used for improving the coding performance and machine vision performance of the expandable image encoder based on machine learning.
Owner:HISENSE VISUAL TECH CO LTD

Model training method, electronic equipment and computer readable storage medium

The invention discloses a model training method, electronic equipment and a computer readable storage medium, and relates to the technical field of data processing and large models. The method comprises the steps of obtaining a training data set, wherein the training data set comprises image annotation data of various types of concerned objects and description text annotation data associated with the image annotation data; training the initial multi-modal retrieval model by adopting the training data set to generate a learnable prompt; carrying out image coding training on the initial multi-modal retrieval model by adopting the training data set and the learnable prompt to generate an intermediate multi-modal retrieval model; the training data set is adopted to conduct text and image alignment training on the middle multi-modal retrieval model, a target multi-modal retrieval model is generated, and the target multi-modal retrieval model is used for conducting multi-modal retrieval on the target retrieval request to obtain a target retrieval result. According to the method and the device, the technical problems that a multi-modal retrieval model in related technologies cannot adapt to variability and large-scale data requirements, and the model performance is relatively poor are solved.
Owner:ALIBABA CLOUD FEITIAN (HANGZHOU) CLOUD COMPUTING TECH CO LTD

Image coding method and system based on single-step diffusion model

The invention provides an image coding method and system based on a single-step diffusion model, and the method comprises the steps: coding a potential representation to a decoding end through employing an extremely low bit rate based on a stable diffusion model (SD-Turbo) and a depth potential representation compression model, and obtaining a reconstructed image through single-step denoising; a group of auxiliary encoders and decoders are introduced, rich and entropy-perceived pixel-level original image semantic information is extracted at an encoding end, and decoded image structure information is shared for a single-step denoising process at a decoding end; and the whole model is subjected to end-to-end optimization by adopting code rate and pixel level constraint so as to achieve the optimal subjective coding quality. According to the invention, the technical problems of high decoding complexity and poor reconstruction consistency are solved.
Owner:STATE GRID ANHUI ELECTRIC POWER CO LTD ELECTRIC POWER SCI RES INST

Triangulation-based adaptive subsampling of dense motion vector fields

The present disclosure relates to an apparatus and a method for providing a plurality of motion vectors related to an image coded in a bitstream. The method includes obtaining a set of sample positions within the image, obtaining respective motion vectors associated with the set of sample positions, deriving an additional motion vector based on information coded in the bitstream, determining an additional sample position located within a triangle, which is formed by three sample positions of the set of sample positions, based on the respective motion vectors associated with the three sample positions, the triangle not including the rest of the sample positions of the set of sample positions, adding the additional sample position to the set of sample positions, and associating the derived additional motion vector with the additional sample position.
Owner:HUAWEI TECH CO LTD +1

Anti-loss image coding method for unstable satellite network

The invention provides an anti-loss image coding method for an unstable satellite network. The anti-loss image coding method comprises the following steps: step 1, establishing an end-to-end image coding and decoding network; step 2, during encoding, performing spatial channel rearrangement on potential features extracted from an image by an encoder, and inputting the rearranged potential features into a super-prior encoder for distribution modeling; 3, inputting the combined features into a decoder for decoding to obtain a reconstructed image; 4, in the training process of the image coding and decoding network, adopting a plurality of double-tail discarding strategies and a channel random covering strategy, and introducing a Gilbert-Aglaite packet loss model; and step 5, the compressed hyper-priori and potential features are sent in a grouping and packaging manner. The method has good universality in actual network evaluation, and pixel errors or color deviation in the progressive transmission process can be prevented. In addition, the method introduces acceptable calculation complexity, so that the method is suitable for practical application.
Owner:NANJING UNIV

View processing method for multi-mode electronic information system

The invention relates to the technical field of image processing, in particular to a multi-modal electronic information system view processing method, which comprises the following steps of: acquiring image data, analyzing and determining data features and spatial distribution, and generating a feature positioning map; key content and background information in the image are identified and marked according to the feature positioning map, and a key content mapping map is generated; and adjusting image coding parameter setting based on the key content mapping graph. According to the method, compression of different levels is applied to key content and background information, so that the storage and transmission efficiency is optimized, and meanwhile, the most important image quality is reserved. Besides, the strategy of monitoring the network state in real time and adjusting the transmission rate and the compression rate ensures the optimization of data transmission under different network conditions, and reduces the risk of data transmission error or delay. By identifying repeated or redundant content, data stream processing is optimized, the burden of the system is reduced, and the efficiency and reliability of data processing are improved.
Owner:ZHEJIANG GONGSHANG UNIVERSITY

Automobile headlamp communication system and method based on image frame insertion

The invention relates to the technical field of optical communication, in particular to an automobile headlamp communication system and method based on image frame interpolation, comprising a transmitting end and a receiving end, a vehicle-mounted central control system of the transmitting end is used for receiving input to-be-transmitted information, and an ambient light detection module evaluates an external illumination condition in response to the input of the to-be-transmitted information; an image gray value and frame insertion frequency are set based on the illumination intensity, the image coding module codes information to be transmitted based on the set image gray value to generate a projection image frame, and the projection module inserts the projection image frame into a normal illumination frame sequence of a vehicle lamp based on the frame insertion frequency and projects the projection image frame to an external environment; the image acquisition module of the receiving end is used for acquiring a projection image frame, the image decoding module preprocesses the projection image frame and then decodes the projection image frame, and the information to be transmitted is restored and sent to the vehicle-mounted central control system. According to the invention, vehicle lamp communication at a certain distance in a traffic environment can be realized on the premise of ensuring normal vehicle lamp illumination.
Owner:SUZHOU UNIV

Graph-to-text model training method and device and text generation method and device

The invention provides a graph-to-text model training method and device, relates to the technical field of artificial intelligence, in particular to the technical fields of computer vision, deep learning, large models and the like, and can be applied to scenes such as content generation of artificial intelligence. According to the specific implementation scheme, an image sample set is obtained, wherein the image sample set comprises at least one image sample; obtaining an initial graph-to-text model, wherein the graph-to-text model comprises an image coding sub-network, a connection sub-network and a multi-modal large language model; training an image coding sub-network and a connection sub-network based on the image sample set and the image generator to obtain a trained image coding sub-network and a trained connection sub-network; and based on the image sample set, training the multi-modal large language model, the trained image coding sub-network and the trained connection sub-network to obtain a trained graph-to-text model.
Owner:BAIDU ONLINE NETWORK TECH (BEIJIBG) CO LTD

Digital human voice lip synchronization method

The invention provides a digital human voice lip shape synchronization method, and relates to the technical field of digital human generation, and the method comprises the steps: obtaining a source video, carrying out the preprocessing of the source video, and obtaining audio data and image data; performing synchronous processing on the audio coding features and the image coding features by adopting a lip shape synchronous model to obtain synchronous features; the audio coding features are output by the audio data through the lip shape synchronization model; the image coding features are output by the image data through the lip shape synchronization model; the synchronous processing comprises linear feature projection and feature similarity calculation; and generating a video according to the synchronization feature to obtain a target video. Through the method, the problems that the matching degree of the voice and the lip shape of the digital human generated in the prior art is not high and the digital human cannot adapt to a multi-language environment are solved.
Owner:NANJING SILICON INTELLIGENCE TECH CO LTD

Image coding prediction mode selection method

The invention discloses an image coding prediction mode selection method, which solves the problems that in the prior art, image redundant information is not efficiently utilized in conventional image coding, and complex video coding is relatively high in application delay for low hardware cost, and the like. The absolute reference sample value and the recovery sample value of the reference image are used as reference sample values, so that the image redundant information is effectively utilized, and the compression efficiency is improved; a fixed number of coding prediction modes are set in a selection set, different coding prediction mode selection sets are determined for different scenes, and coding code streams are reduced; when the optimal coding prediction mode is selected, resource consumption and delay are reduced by means of source image information, so that high coding efficiency and good subjective effect can be realized under the condition of low cost and low delay.
Owner:NANJING MAGEWELL ELECTRONICS CO LTD