760 results about "Image code" patented technology

The invention discloses an image optimization method based on a CLIP model, and the method comprises the steps: S1, carrying out the coding initial processing of characters and images, inputting a character description and an initial image code into a CLIP model, and obtaining a corresponding semantic vector and a visual vector; s2, cross-modal contrast learning: based on positive and negative sample contrast and an adaptive attention mechanism, aligning the semantic vector and the visual vector obtained in the S1; s3, self-supervised learning is carried out, and precision improvement of semantic-to-visual mapping is completed through self-supervised learning; s4, feature refinement generation: based on the optimized semantic and visual feature vectors, generating a preliminary concept conforming to description; and S5, image output: repeating the steps S1 to S4 until a preset matching value is met, and outputting a conception image matched with the text description. According to the image optimization method based on the CLIP model, semantic mapping from characters to images is improved, so that the semantic mapping is more in line with the expectation of natural language description, and a visual conception graph with rich details is generated.

The invention provides a mathematical problem solving method and device based on a multi-modal large model and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a mathematical element image of a mathematical problem; inputting the mathematical element image into an image coding model to obtain an image vector output by the image coding model; the image coding model is obtained by training based on a sample mathematical element image and a positive sample text description and a negative sample text description corresponding to the sample mathematical element image; inputting the image vector into a self-adaptive module to obtain an image conversion coding vector output by the self-adaptive module; the adaptive module is obtained based on training of a sample image vector and a sample text vector; determining question stem characters of the mathematical question, and inputting the question stem characters and the image conversion coding vector into the large language model to obtain a prediction answering process output by the large language model; the large language model is obtained based on sample question stem characters, sample image conversion coding vectors and sample answering process training, and the mathematical problem solving capability of the multi-modal large model can be improved.

The invention relates to the technical field of image processing, and provides an underwater image enhancement method and system based on multi-modal degradation feature learning. The method comprises the following steps: collecting underwater image data of different modes to construct an underwater image data set, and dividing the underwater image data set into a training set and a test set; preprocessing the training set to obtain an image text degradation pair data set; constructing a multi-modal visual language MD-CLIP network framework, and training the MD-CLIP network framework by using the image text degradation pair data set; the MD-CLIP network framework comprises an image coding controller ContNet module, a pre-trained CLIP module and a comparative learning module; integrating the trained image coding controller ContNet module into an NAFNet network structure, generating an image enhancement network, and training by using the test set to obtain an underwater image enhancement model; and inputting a to-be-enhanced underwater degraded image into the underwater image enhancement model to generate an enhancement result graph of the to-be-enhanced underwater degraded image.

The invention relates to the technical field of target detection, and provides a remote sensing target detection method and device based on a diffusion model. The method comprises the following steps: step 1, acquiring a remote sensing image data set, and preprocessing the remote sensing image data set; 2, training an image coding module based on the preprocessed remote sensing image data set; the image coding module is used for extracting multi-scale features of a remote sensing image to obtain a multi-scale feature map; 3, constructing an initial noise frame through a variable variance scheduling method, and training a detection decoding module based on the initial noise frame and the multi-scale feature map extracted by the image coding module; the detection decoding module performs noise prediction on the initial noise frame and recovers a target frame to complete target detection; and 4, forming a remote sensing target detection model by using the trained image coding module and the detection decoding module, wherein the remote sensing target detection model is used for performing target detection on the remote sensing image.

The invention discloses a crack image processing method based on a neural network and plane geometry image transformation, and particularly relates to the field of general image data processing driven by plane geometry image transformation and neural network identification. Comprising the steps that an image collection platform projects a structured image coding pattern to the surface of a to-be-measured target through a preset image coding pattern template, and synchronously collects an image frame containing an image mark so as to construct a space anchor point used for image geometric reference in an image plane; view angle parameter change information in an image acquisition process is acquired through an image acquisition platform, and projection configuration parameters of an image coding pattern are combined. By constructing an image offset modeling mechanism jointly driven by an image coding pattern and a space anchor point, image in-plane correction of geometric distortion features in an image frame is realized, and structural consistency and texture continuity before subsequent image analysis are improved.

The invention belongs to the technical field of computer vision and deep learning, and particularly relates to an optical flow estimation method and system fusing Mama and visual basic model knowledge. The method comprises the following steps: performing down-sampling feature extraction on two adjacent frames of input images by using a convolutional neural network to obtain local texture features; performing down-sampling on the first frame image to obtain context features; meanwhile, extracting global semantic features of two adjacent frames of images by using a pre-trained visual model, and performing adaptive fusion enhancement through an adaptive semantic texture feature fusion module to obtain an image coding feature pair after semantic enhancement; constructing a related volume through pixel-by-pixel dot product operation; and finally, based on the obtained related volume and context features, iteratively optimizing the output optical flow through a loop iteration updating module. The method solves the problems that in a low-texture, repeated-texture or sheltered area, feature expression is unstable, self-adaptive modeling capacity is lacked, different scenes are difficult to generalize, and model performance and efficiency cannot be balanced.

The invention relates to the technical field of multi-mode prompt learning, and discloses a cross-layer image text interaction prompt learning method based on an attention mechanism, and the method comprises the steps: obtaining text data and image data; constructing a cross-layer image text interaction prompt learning network model; converting the text data into word vectors by utilizing a word embedding module; performing multi-level attention coding on the word vector and the text prompt vector by using a text encoder network to obtain a text code; a cross-layer interaction prompt learning module is used for generating an image prompt vector for the text prompt vector by adopting a multi-head attention mechanism; converting the image data into block feature maps by using a map segmentation module; performing multi-level attention coding on the block feature map and the image prompt vector by using an image encoder network to obtain an image code; and carrying out multi-modal contrast learning on the text codes and the image codes by using a multi-modal contrast learning module. According to the invention, the accuracy of multi-modal learning task execution can be further improved.

The invention provides a pathological section image analysis method and device based on a large language model, and belongs to the field of computer vision, deep learning and medical image analysis. The method comprises the following steps: constructing an image coding large model which is composed of a pre-training model and an additionally-added task head and aims at a pathological section; freezing the pre-training model, training a task head by using a preset training set, and checking by using a preset verification set to obtain a task head after preliminary training; unfreezing the pre-training model, performing fine tuning training on the large model by using the training set, and performing inspection by using the verification set so as to obtain a finally trained large model; and inputting a to-be-analyzed pathological section image into the large model, and calculating corresponding features of the image based on a result output by the large model to obtain a final analysis result. According to the method, the features in the pathological section image can be automatically extracted, and the diagnosis report is generated based on the features, so that the accuracy and efficiency of pathological section image diagnosis are improved.

The invention discloses an information processing method and system based on a honeycomb pixel structure, and the method comprises the steps: an encoding step: encoding a positioning graph and input information to form an encoding matrix based on regular hexagonal pixels, the encoding matrix comprising a positioning graph region and an effective information region; a masking step, at least performing XOR operation on the effective information area of the coding matrix and different mask matrixes to form different cellular codes, and comparing the different cellular codes to obtain an optimal cellular code; the mask matrix is also a matrix based on hexagonal pixels; and a decoding step: decoding the optimal cellular code to obtain effective information. According to the method, the cellular codes can be quickly, accurately and efficiently coded and decoded, and a new choice is provided for application of an image coding technology.

The invention discloses an underwater robot visual image compression, decompression and transmission method and device. The method comprises the steps of constructing an image compression model used for underwater robot visual image compression, training the image compression model, constructing a visual image coding end based on the trained image compression model, performing compression coding on the underwater robot visual image by using the visual image coding end, generating code stream data, and outputting the code stream data. And constructing a visual image compression end based on the trained image compression model so as to complete compression of the visual image of the underwater robot, and constructing a visual image decoding end based on the trained image compression model, the visual image decoding end being used for decoding the code stream data and reconstructing the image so as to generate a reconstructed visual image of the underwater robot. The underwater robot visual image compression and decompression method provided by the invention is high in compression ratio and good in image reconstruction performance, and can be applied to compression and transmission tasks of underwater robot visual images.

The invention relates to the technical field of image processing, and provides an image filling method and device, electronic equipment and a storage medium, and the method comprises the steps: encoding a text prompt word to obtain a text encoding vector, and encoding a masked background image to obtain an image encoding vector; splicing the image coding vector, the position mask and the random noise to obtain a fused noise vector; and step-by-step denoising is carried out on the fused noise vector, a text coding vector and a texture guide vector are added in the denoising process as guide conditions, a filling image is generated, and the texture guide vector is obtained by carrying out splicing, smooth transition and coding processing on texture features of the foreground image and texture features of the background image. According to the method, the filling image which is rich in details, natural in texture and highly fused with the background image can be generated through a step-by-step denoising process and combining the text coding vector and the texture guide vector as double guide conditions.

The invention relates to an image enhancement method for constructing a spatial vision neural network model. The method comprises the following steps: firstly, carrying out normalization processing on a binarized noise-containing two-dimensional image to enable the binarized noise-containing two-dimensional image to meet small parameter limitation brought by activation of an FHN neuron adiabatic approximation condition; secondly, the space signal is subjected to convolution through an FHN neuron template, and a space neuron pulse potential sequence two-dimensional image coding signal is obtained; constructing an anisotropic spread function based on improved bidirectional adaptive selection as a visual receptive field model with a direction selection mechanism, and carrying out convolution on a neuron pulse coded signal and the constructed receptive field model; and finally, an optimal binary image result is found. According to the method, the direction selectivity and the neuron pulse issuing mechanism of the biological vision system are effectively combined, so that the biological vision system conforms to the physiological characteristics of real vision neuron pulse triggering and signal selectivity, and the ideal image denoising effect is achieved.

The invention discloses a voice interaction method, a server and a nonvolatile computer readable storage medium containing a computer program. The method comprises the following steps: receiving a current round voice request forwarded by a vehicle and current environment image coding information of a cabin; and determining target input information according to the current environment image coding information of the cockpit, the current round of voice request, the historical round of voice request and the natural language understanding result of the historical round of voice request. And according to the target input information, determining an intention label of the current round of voice request. And completing voice interaction according to the intention label. Therefore, the target input information is determined by combining the current environment image coding information of the cabin, the current round of voice request, the slot position identification result, the historical round of voice request and the understanding result thereof, intention identification is performed on the target input information, the weak intention voice request of the user is accurately identified, misunderstanding of the user request is reduced, and the user experience is improved. And the probability of error response is reduced, so that the user experience is improved.

The invention discloses a model training method, electronic equipment and a computer readable storage medium, and relates to the technical field of data processing and large models. The method comprises the steps of obtaining a training data set, wherein the training data set comprises image annotation data of various types of concerned objects and description text annotation data associated with the image annotation data; training the initial multi-modal retrieval model by adopting the training data set to generate a learnable prompt; carrying out image coding training on the initial multi-modal retrieval model by adopting the training data set and the learnable prompt to generate an intermediate multi-modal retrieval model; the training data set is adopted to conduct text and image alignment training on the middle multi-modal retrieval model, a target multi-modal retrieval model is generated, and the target multi-modal retrieval model is used for conducting multi-modal retrieval on the target retrieval request to obtain a target retrieval result. According to the method and the device, the technical problems that a multi-modal retrieval model in related technologies cannot adapt to variability and large-scale data requirements, and the model performance is relatively poor are solved.

The invention provides an image coding method and system based on a single-step diffusion model, and the method comprises the steps: coding a potential representation to a decoding end through employing an extremely low bit rate based on a stable diffusion model (SD-Turbo) and a depth potential representation compression model, and obtaining a reconstructed image through single-step denoising; a group of auxiliary encoders and decoders are introduced, rich and entropy-perceived pixel-level original image semantic information is extracted at an encoding end, and decoded image structure information is shared for a single-step denoising process at a decoding end; and the whole model is subjected to end-to-end optimization by adopting code rate and pixel level constraint so as to achieve the optimal subjective coding quality. According to the invention, the technical problems of high decoding complexity and poor reconstruction consistency are solved.

The present disclosure relates to an apparatus and a method for providing a plurality of motion vectors related to an image coded in a bitstream. The method includes obtaining a set of sample positions within the image, obtaining respective motion vectors associated with the set of sample positions, deriving an additional motion vector based on information coded in the bitstream, determining an additional sample position located within a triangle, which is formed by three sample positions of the set of sample positions, based on the respective motion vectors associated with the three sample positions, the triangle not including the rest of the sample positions of the set of sample positions, adding the additional sample position to the set of sample positions, and associating the derived additional motion vector with the additional sample position.

The invention relates to the technical field of image processing, in particular to a multi-modal electronic information system view processing method, which comprises the following steps of: acquiring image data, analyzing and determining data features and spatial distribution, and generating a feature positioning map; key content and background information in the image are identified and marked according to the feature positioning map, and a key content mapping map is generated; and adjusting image coding parameter setting based on the key content mapping graph. According to the method, compression of different levels is applied to key content and background information, so that the storage and transmission efficiency is optimized, and meanwhile, the most important image quality is reserved. Besides, the strategy of monitoring the network state in real time and adjusting the transmission rate and the compression rate ensures the optimization of data transmission under different network conditions, and reduces the risk of data transmission error or delay. By identifying repeated or redundant content, data stream processing is optimized, the burden of the system is reduced, and the efficiency and reliability of data processing are improved.

The invention relates to the technical field of optical communication, in particular to an automobile headlamp communication system and method based on image frame interpolation, comprising a transmitting end and a receiving end, a vehicle-mounted central control system of the transmitting end is used for receiving input to-be-transmitted information, and an ambient light detection module evaluates an external illumination condition in response to the input of the to-be-transmitted information; an image gray value and frame insertion frequency are set based on the illumination intensity, the image coding module codes information to be transmitted based on the set image gray value to generate a projection image frame, and the projection module inserts the projection image frame into a normal illumination frame sequence of a vehicle lamp based on the frame insertion frequency and projects the projection image frame to an external environment; the image acquisition module of the receiving end is used for acquiring a projection image frame, the image decoding module preprocesses the projection image frame and then decodes the projection image frame, and the information to be transmitted is restored and sent to the vehicle-mounted central control system. According to the invention, vehicle lamp communication at a certain distance in a traffic environment can be realized on the premise of ensuring normal vehicle lamp illumination.

The invention provides a digital human voice lip shape synchronization method, and relates to the technical field of digital human generation, and the method comprises the steps: obtaining a source video, carrying out the preprocessing of the source video, and obtaining audio data and image data; performing synchronous processing on the audio coding features and the image coding features by adopting a lip shape synchronous model to obtain synchronous features; the audio coding features are output by the audio data through the lip shape synchronization model; the image coding features are output by the image data through the lip shape synchronization model; the synchronous processing comprises linear feature projection and feature similarity calculation; and generating a video according to the synchronization feature to obtain a target video. Through the method, the problems that the matching degree of the voice and the lip shape of the digital human generated in the prior art is not high and the digital human cannot adapt to a multi-language environment are solved.

The invention discloses an image coding prediction mode selection method, which solves the problems that in the prior art, image redundant information is not efficiently utilized in conventional image coding, and complex video coding is relatively high in application delay for low hardware cost, and the like. The absolute reference sample value and the recovery sample value of the reference image are used as reference sample values, so that the image redundant information is effectively utilized, and the compression efficiency is improved; a fixed number of coding prediction modes are set in a selection set, different coding prediction mode selection sets are determined for different scenes, and coding code streams are reduced; when the optimal coding prediction mode is selected, resource consumption and delay are reduced by means of source image information, so that high coding efficiency and good subjective effect can be realized under the condition of low cost and low delay.

The invention discloses an image coding method and device, a product, equipment and a medium, and is applied to the technical field of coding, and the method comprises the steps: calculating a transverse gradient and a longitudinal gradient of a target coding block, the target coding block being an image block in a to-be-coded image; calculating texture direction information of the target coding block based on the transverse gradient and the longitudinal gradient; comparing the texture direction information with a preset threshold value, and based on a comparison result, rejecting a target division mode from a division mode set corresponding to the target coding block to obtain residual division modes; determining an optimal division mode of the target coding block from the residual division modes; and dividing the target coding block based on the optimal division mode, if a division end condition is not met, determining the coding sub-block obtained by division as a new target coding block, and triggering the step of calculating the transverse gradient and the longitudinal gradient of the target coding block. Therefore, the time consumption for selecting the optimal division mode can be reduced, so that the coding efficiency is improved.

The invention relates to the field of fluid pipeline system safety and intelligent control, and provides a water hammer pressure prediction and active suppression method and system. The method comprises the following steps: acquiring a multi-dimensional digital signal flow, and generating a window data matrix sequence through sliding window segmentation and dimensionless; image coding is carried out to form a spatiotemporal feature sequence, the spatiotemporal feature sequence is processed by a double-branch spatiotemporal prediction network, and a prediction feature spectrum sequence is output and decoded to form a prediction waveform matrix; future risk indexes and morphological characteristics are extracted and spliced with the multi-dimensional digital signal flow to form a prediction enhancement state vector, and the prediction enhancement state vector is input into a strategy network to generate valve action control parameters; and a target valve opening curve is decoded, a composite control signal is obtained through feedforward-feedback composite control, actual response data flow and historical data packaging are collected, and a prediction network and a strategy network are updated online through mixed priority screening. According to the method, prospective and self-adaptive water hammer prediction and active suppression are realized, the pressure evolution trend of a pipeline system is informed, and decision support is provided for safe operation of a fluid pipe network.

The embodiment of the invention provides an image coding division decision method and device, equipment, a storage medium and a product. According to the technical scheme provided by the embodiment of the invention, the quadtree division decision result is obtained by performing the quadtree division decision on the current coding block, and the optimal mode information of the four sub-coding blocks obtained by performing the quadtree division on the current coding block is determined according to the quadtree division decision result; and determining skipping decision information in the preset direction according to the consistency of the optimal mode information in the preset direction, and skipping a division decision of the current coding block in the preset direction when the skipping decision information is a division decision of skipping the preset direction. According to the embodiment of the invention, the target division decision result of the current coding block can be determined according to the quadtree division decision result and the multi-type division result, the division decision quality is ensured, the data processing amount of the division decision can be effectively reduced, and the division decision efficiency is improved.

The invention discloses a video generation method based on a body mask, belongs to the technical field of artificial intelligence, and can solve the problems that an existing body world model is inconsistent in an action space and a pixel space, is sensitive to the change of a visual angle of a camera, and is not uniform in architecture among different body structures. The method comprises the following steps: S1, determining a mask sequence with a body according to a target video; s2, encoding the body mask sequence and the initial frame image of the target video respectively to correspondingly obtain body mask features and image encoding features; s3, inputting the body mask features into a control network module to obtain injection features, and inputting the image coding features into a backbone network of a video generation model to obtain backbone layer features; and S4, fusing the injection feature and the trunk layer feature to obtain a fused feature, and generating a prediction video according to the fused feature. The method is used for generating the predictive video with the body.

An image coding mechanism is disclosed. The mechanism includes determining a region of interest in an image. The image is partitioned into a plurality of blocks. A transform is applied to the blocks to obtain a plurality of coefficients. The coefficients are assigned into quality layers of decreasing priority. Each subsequent layer includes data to incrementally increase quality of a reconstructed image. Region of interest coefficients are positioned in one or more image data packets by quality layer in order of decreasing priority. Remaining coefficients are positioned in the image data packets with a lower priority than the region of interest coefficients. The remaining coefficients are positioned by quality layer in order of decreasing priority. The image data packets are stored and transmitted over a network.

Embodiments of the present application provide an image recognition method, apparatus and device, a storage medium, and a product. According to the technical solution provided in the embodiments of the present application, an image content recognition model uses an image-text interaction unit to perform analysis processing on a target text feature and a target image feature of an image to be recognized, to obtain a content recognition result of said image; moreover, the target text feature is generated on the basis of a set target review rule by means of a text coding unit in the image content recognition model, and the target image feature is generated on the basis of said image by means of an image coding unit in the image content recognition model. Image-text interaction can be effectively performed on the basis of the target text feature of the target review rule and the target image feature of said image, thereby accurately detecting metaphorical information in the image, and effectively improving the image recognition effect and efficiency.

The invention discloses an industrial design-oriented drawing semantic analysis and structured conversion method and system. The method comprises the following steps: acquiring an input image, carrying out different-scale coding on image features through an image coding backbone network, selecting a specific layer to extract a multi-scale feature map, and fusing to generate multi-scale features; inputting the multi-scale features into a geometric primitive detection network and an other element detection network, respectively identifying geometric primitives and non-geometric primitives, and carrying out positioning and classification; analyzing mutual relations between geometric elements and other elements contained in the graph through an element relation graph network, and generating formalized language description based on a matching rule; a Prompt template is constructed, geometric information is supplemented by using a multi-modal large model, the rationality of the supplemented information is verified through a geometric attribute relationship verifier, and complete formalized language description is obtained. According to the method, the industrial drawing image containing the complex constraint relation can be efficiently and accurately converted into machine-readable structured data.

The invention discloses a method for reconstructing an image from brain activity based on a multi-modal diffusion model. The method comprises the following steps: A1, preprocessing collected electroencephalogram data; a2, inputting the preprocessed electroencephalogram data into an electroencephalogram feature extraction network to obtain electroencephalogram data features; a3, respectively inputting the electroencephalogram data features into an image coding regression model, an image embedding regression model and a text embedding regression model to obtain an image coding result, a CLIP image embedding vector and a CLIP text embedding vector; and A4, taking the image coding result as image input of the multi-modal diffusion model, taking the CLIP image embedding vector as an image hidden variable, taking the CLIP text embedding vector as a cue word hidden variable, and carrying out de-noising diffusion decoding to obtain a reconstructed image. According to the method, the pictures which are relatively high in recognizability and relatively clear can be generated from the electroencephalogram signals.

An event-guided lightweight attention network for efficiently removing image motion blur belongs to the technical field of image processing and comprises an image feature extraction module, an event feature extraction module, a cross-modal attention feature fusion module and an image reconstruction module. Firstly, image coding features and event coding features are extracted from a motion blurred image and an event voxel grid through an image feature extraction module and an event feature extraction module respectively, then, the image coding features and the event coding features are fused through a cross-modal attention feature fusion module to obtain fusion features, and finally, the motion blurred image and the event voxel grid are subjected to cross-modal attention feature fusion. The fused features are processed by an image reconstruction module to obtain a clear image. According to the invention, the motion blur removal of the image is realized, the obtained image is higher in definition, the image processing time is saved, and the image processing efficiency is higher; the attention network is a single-stage network and has the characteristic of light weight.