879 results about "Sample graph" patented technology

The invention relates to the technical field of small sample image classification, in particular to a multi-modal small sample image classification method and system based on multi-scale dynamic feature fusion. The method comprises the following steps: acquiring a small sample image data set, and dividing the small sample image data set into a support set and a query set; and inputting the two features into a feature extraction network to extract global features, and inputting the features into a regional proposal network (RPN) to obtain corresponding local features. Extracting semantic information according to the category label of the support set data, and inputting the semantic information into a text encoder to obtain text features; inputting the global feature, the local feature and the text feature of the input image into an adaptive weight network, and performing dynamic feature fusion to effectively integrate multi-modal information; and inputting the fused multi-modal features into a classification model which takes a prototype network as a learning device and takes an MAML algorithm as a meta-learning device for classification prediction, finally training a network model by using a composite loss function, and performing image classification by using the trained classifier.

A method and apparatus for training a text-graph model, a device, and a storage medium, relating to the artificial intelligence. The method comprises: on the basis of a training set of graph-text sample pairs, performing iterative training on a model to be trained, to obtain a text-graph model; in a training process: selecting a graph-text sample pair from the training set of graph-text sample pairs, the graph-text sample pair comprising a sample image and a description text, and the sample image comprising at least two objects; obtaining mask images corresponding to the at least two objects and associated object class names, the mask images being used for distinguishing position regions of the objects in the sample image; inputting the sample image and the description text to a model to be trained, to obtain an image prediction noise of the sample image, and inputting at least two mask images and the associated object class names to the model to be trained to obtain object prediction noises respectively associated with the at least two mask images; and constructing a loss function on the basis of the image prediction noise and the object prediction noises, and by using the loss function, adjusting a parameter for the model to be trained.

The invention relates to the technical field of industrial defect detection, and discloses a memory matching industrial defect detection method based on adaptive feature fusion, and the method comprises the steps: generating a plurality of unknown defect samples through an enhanced image-level defect simulation strategy, and helping a model to effectively distinguish a normal mode from an abnormal mode; the method comprises the following steps: extracting multi-scale features of a to-be-detected sample image, a normal sample image and a defect sample image, designing an adaptive hierarchical memory bank architecture, embedding the multi-scale features into a grid memory bank in a layered manner, further improving the reasoning speed through an adaptive core set sampling strategy, and combining multi-scale feature fusion and a defect positioning optimization technology to obtain a defect positioning algorithm. And the difference between the multi-scale features and the normal mode is fully utilized, so that the accuracy of defect detection and positioning is remarkably improved. Compared with the prior art, the defect detection precision and the positioning accuracy can be improved, and meanwhile, the real-time detection requirement can be met.

The invention belongs to the technical field of computer vision and artificial intelligence, and discloses a combined zero sample image classification method based on hierarchical feature fusion. According to the method, a solution based on hierarchical feature fusion is innovatively provided for the technical difficulty of combined zero sample image classification. According to the method, features of different depths in a CLIP model visual encoder are selected and subjected to hierarchical fusion, multi-level and multi-scale visual information is effectively extracted, the fused features are further used for visual and semantic cross-modal interaction, and the understanding ability of the model for complex combinations is remarkably improved. In addition, by introducing a specific loss item, attributes and object features are effectively separated, and the feature aliasing problem is avoided. In a combined zero sample image classification task, the method shows excellent generalization ability and recognition effect, and can significantly improve the classification performance of the model for unknown complex combined images.

The invention discloses a lithium battery health state evaluation method based on deep learning, and relates to the technical field of battery evaluation. Comprising the following steps: acquiring a Nyquist curve graph of a lithium battery through an electrochemical impedance spectroscopy test, and mapping the Nyquist curve graph with internal impedance characteristic parameters one by one to generate a sample image set; and establishing a deep learning network model based on the sample image set, and training and generating a prediction model by taking the Nyquist curve graph as input and the internal impedance characteristic parameters as labels. And testing the to-be-evaluated battery, and inputting the target Nyquist curve graph to predict internal impedance characteristic parameters. And calculating a capacity retention ratio index, a cycle efficiency degradation index and a shape expansion influence index in combination with the electric power characteristic parameters. The indexes are comprehensively analyzed to obtain a health state comprehensive coefficient, and the health state comprehensive coefficient is compared with a health threshold value to judge the health state of the battery. According to the method, the evaluation precision is improved through deep learning, and efficient and accurate health state evaluation is realized.

The invention provides an automatic driving obstacle recognition method and system based on deep learning, and relates to the technical field of automatic driving, and the method comprises the steps: collecting sample image data and sample point cloud data; performing feature extraction on the marked point cloud and the marked image by using a 3DCNN network, and performing training according to an extraction result to obtain a classifier and a prediction head; enhancing the unmarked image and the unmarked point cloud; performing feature extraction on the enhanced image and the enhanced point cloud to obtain a multi-scale image feature and a multi-scale point cloud feature; dynamically adjusting the weights of the multi-scale image features and the multi-scale point cloud features based on a feature fusion method of an attention mechanism to obtain fusion features; and inputting the fused features into the trained classifier and the prediction head to obtain an obstacle recognition result. According to the method, the accuracy, robustness and generalization ability of obstacle recognition are remarkably improved, the marking cost is reduced, and the high requirements of an automatic driving scene for real-time performance and reliability are met.

The invention provides a mathematical problem solving method and device based on a multi-modal large model and electronic equipment, and relates to the technical field of artificial intelligence. The method comprises the following steps: determining a mathematical element image of a mathematical problem; inputting the mathematical element image into an image coding model to obtain an image vector output by the image coding model; the image coding model is obtained by training based on a sample mathematical element image and a positive sample text description and a negative sample text description corresponding to the sample mathematical element image; inputting the image vector into a self-adaptive module to obtain an image conversion coding vector output by the self-adaptive module; the adaptive module is obtained based on training of a sample image vector and a sample text vector; determining question stem characters of the mathematical question, and inputting the question stem characters and the image conversion coding vector into the large language model to obtain a prediction answering process output by the large language model; the large language model is obtained based on sample question stem characters, sample image conversion coding vectors and sample answering process training, and the mathematical problem solving capability of the multi-modal large model can be improved.

The embodiment of the invention relates to the technical field of data processing and AI vision, and provides a warehouse inventory intelligent management method and device based on AI vision monitoring and a terminal, and the method comprises the steps: obtaining inventory management basic data and external dynamic influence data of a to-be-monitored warehouse; based on an AI visual technology, according to the sample image data set and the label image data set, performing dynamic optimization processing on current inventory data in the inventory management basic data to obtain dynamically optimized inventory data; according to the inventory management basic data and the dynamic optimization inventory data, carrying out inventory structure evaluation to obtain inventory structure evaluation data; performing real-time dynamic adjustment processing on the inventory structure evaluation data according to the external dynamic influence data to obtain inventory dynamic adjustment data; according to the dynamic optimization inventory data, the inventory structure evaluation data and the inventory dynamic adjustment data, an inventory health evaluation report is generated, and the purpose of more accurate and more comprehensive warehouse inventory intelligent management is achieved.

The invention provides an intelligent defect detection method and an intelligent defect detection network based on S-shaped dynamic convolution and a cross-weighted feature pyramid, and the method comprises the following steps: collecting an OK sample image and a plurality of NG sample images, and employing a co-occurrence matrix matching method based on Laplacian gradient features, extracting defect labeling area images of the plurality of NG sample images; constructing a feature extraction network based on S-type dynamic convolution for extracting defect features; constructing a pyramid module based on cross-weighted features, wherein the pyramid module is used for multi-level fusion calculation; building an intelligent defect detection network based on S-type dynamic convolution and a cross-weighted feature pyramid, and training the intelligent defect detection network; performing product detection by using the defect intelligent detection network based on the S-shaped dynamic convolution and the cross-weighted feature pyramid; through the arrangement, high-precision segmentation can be better carried out on the defect area and the background area, the precision and accuracy of industrial AOI defect detection are improved, and the quality safety of industrial AOI products is guaranteed.

The invention relates to the technical field of defect detection, and particularly discloses an electronic component defect detection method, system and device, and the method comprises the steps: collecting a multiband original image set based on multispectral imaging, inhibiting batch material difference and surface reflection through orthogonal complementary space projection, and obtaining a de-noised image set; performing feature extraction on the de-noised image set, and identifying and marking all fragmented defect areas; based on a graph neural network space attention mechanism, establishing an association relationship between fragmented defect areas to form a complete defect feature chain; gray frequency value sequences of the to-be-detected image and the standard sample image are constructed respectively, and a first fitting curve and a second fitting curve are obtained through fitting of a skewed distribution function; and finally, comprehensively judging whether the electronic component has defects or not by combining the defect characteristic chain and the difference characteristics of the two fitting curves. According to the method, interference can be effectively suppressed, fragmented defects can be accurately identified, and the detection accuracy and reliability are remarkably improved through multi-dimensional feature fusion.

Disclosed in embodiments of the present application are a multi-modal retrieval model generation method and apparatus, a device, and a storage medium. The embodiments of the present application can be applied to various scenarios such as artificial intelligence, intelligent transportation, and assisted driving. The multi-modal retrieval model generation method comprises: by means of a prefix vector module, performing feature recognition on first sample image modal data to obtain a first image prefix vector; by means of a modal identification module and on the basis of the first image prefix vector, the first sample image modal data, and first sample text modal data, generating a first retrieval character; by means of a constrained decoding module, acquiring from a pre-generated database a first sample retrieval result associated with the first retrieval character; and on the basis of a first reference retrieval result and the first sample retrieval result, adjusting the model parameters corresponding to the prefix vector module and the constrained decoding module to obtain a retrieval model after the adjustment. The present application can improve the training efficiency of multi-modal retrieval models.

The present application relates to the field of autonomous driving, and in particular relates to an occupancy grid prediction method, an occupancy grid prediction apparatus for implementing the method, a computer storage medium for implementing the method, and a smart device having the occupancy grid prediction apparatus. The method comprises: acquiring a pure visual image of the surroundings of a vehicle body; inputting the pure visual image into an occupancy grid prediction model; and using the occupancy grid prediction model to determine an occupancy result of an object in the pure visual image as a model output. The model output is an occupancy result in a cylindrical form, and the occupancy grid prediction model is constructed on the basis of a training dataset comprising sample images and truth value information of the sample images. The truth value information is directly generated from point cloud data or converted from occupancy information in a voxel form generated on the basis of the point cloud data.

The invention provides a key point prediction model training method and device, electronic equipment and a computer readable storage medium. The method comprises the following steps: acquiring a sample key point in a sample image and first sample position information of the sample key point; extracting a feature map of the sample image through a to-be-trained key point prediction model; determining first prediction position information of the sample key point and first prediction offset information of a target pixel area where the sample key point is located in the feature map; determining a model loss value based on the first sample position information, the first prediction position information, and first sample offset information and first prediction offset information of the sample key point in the target pixel region; and performing model parameter updating on the to-be-trained key point prediction model based on the model loss value to obtain a trained key point prediction model. According to the invention, the precision and stability of the key point prediction model can be improved.

The invention provides a text graph model training method and device, an image generation method and device and a medium, relates to the technical field of artificial intelligence such as computer vision, deep learning and the like, can be applied to scenes such as AIGC content generation based on artificial intelligence, and comprises the following steps: obtaining a candidate text graph model and a sample high-quality image set; obtaining a candidate user preference data set corresponding to the user set, and obtaining a sample high-quality preference image set corresponding to the sample high-quality image set based on the candidate user preference data set; constructing a sample image-text pair set corresponding to the sample high-quality preference image set, and based on the sample preference image-text pair set, obtaining a target fine tuning sample required by the candidate text graph model; and based on the target fine adjustment sample, performing model fine adjustment on the candidate text graph model to obtain a target text graph model.

The invention discloses a few-sample classification method research based on feature cache optimization, and the method comprises the steps: (1), employing an improved ResNet-based SwAV model as a backbone, and enabling the SwAV model to be used for generating auxiliary features; (2) taking the image data and the text description as original input, and generating text input with rich downstream language semantics by utilizing GPT-3 to serve as text prompt of a CLIP model; (3) through a feature selection method based on feature similarity and difference, the problems of feature redundancy and inaccurate selection in the feature selection process are solved, feature dimensions with high selection value are identified, and normalization and enhancement zooming processing are performed on the features; and (4) utilizing visual contrast knowledge of SwAV, introducing a learnable cache model, and adaptively mixing prediction results from CLIP and SwAV. According to the method, under the CLIP framework without extra training, the accuracy of few-sample image classification is effectively improved through feature cache optimization and a self-adaptive hybrid strategy.

The embodiment of the invention provides a model training method and device based on visual reinforcement learning, equipment and a medium. Comprising the following steps: acquiring a sample image frame and semantic category information, inputting the semantic category information into a visual large language model to obtain a first convolution kernel parameter, and inputting the sample image frame into a first feature convolution kernel to obtain a first feature thermodynamic diagram; obtaining a second convolution kernel parameter and a second feature thermodynamic diagram of the sample image frame through a preset visual reinforcement learning model; constructing a first distillation loss based on the first convolution kernel parameter and the second convolution kernel parameter, and constructing a second distillation loss based on the first characteristic thermodynamic diagram and the second characteristic thermodynamic diagram; through prediction and calculation of the sample action data and the sample state data, self-supervision loss and target strategy loss are constructed; and based on the first distillation loss, the second distillation loss, the self-supervision loss and the target strategy loss, performing parameter adjustment on a preset visual reinforcement learning model to obtain a target visual reinforcement learning model.

The invention discloses a multi-model fusion PCB through hole defect detection method, device and equipment and a medium, and the method comprises the steps: obtaining a to-be-processed sample image set, and processing the to-be-processed sample image set to obtain a target sample image set; performing pruning training on each to-be-trained deep learning model based on the target sample image set to obtain a target deep learning model corresponding to each to-be-trained deep learning model, the target deep learning model including a target key point model, a target detection model and a target segmentation model; the model structures of the to-be-trained deep learning models are different from each other; and establishing a target fusion model based on the target deep learning model, inputting the to-be-detected image into the target fusion model, and determining a through hole defect detection result corresponding to the to-be-detected image. Through multi-model fusion and pruning optimization, the advantages of each model in defect detection can be fully played, and the reasoning speed is remarkably improved on the basis of ensuring high precision.

The present disclosure relates to the technical field of computers, and relates to a network training method and apparatus for pose estimation, a pose estimation method and apparatus, and an electronic device, a storage medium and a computer program product. The network training method for pose estimation comprises: inputting a sample image, which is acquired from a training data set, into a preset feature extraction network; and performing training on an initial pose network according to a feature tensor, which is outputted by the feature extraction network, of a target object in the sample image, so as to obtain a target pose network. By means of the embodiments of the present disclosure, the training efficiency can be improved, thereby further reducing the consumption of computing power resources a processor and memory resources.

Provided in the present application are an image processing method and apparatus, and a computer device, a storage medium and a program product. The method relates to computer vision technology of artificial intelligence. The method comprises: acquiring sample data, wherein the sample data comprises a sample image, a sample category to which the sample image belongs, and a sample identifier, which is set for the sample image; calling an image generation model to extract training representation information of the sample identifier; calling the image generation model to predict, on the basis of the training representation information of the sample identifier, noise added to a noise image corresponding to the sample image, so as to obtain predicted noise, wherein the noise image is obtained by adding label noise to the sample image; and on the basis of the difference between the label noise and the predicted noise, and the difference between training semantics of the sample identifier and semantics of the sample category, updating model parameters of the image generation model, so as to train the image generation model. Using the present application can reduce the degree of overfitting of the image generation model and improve the training effect of the image generation model.

The invention provides a parameter efficient fine tuning method based on a hybrid expert system, and an image segmentation method and device. The method comprises the following steps: based on an improved sensitivity constraint function and a low-rank matrix constraint function in a self-adaptive low-rank adaptation module; obtaining a target low-rank matrix corresponding to a to-be-trained model parameter of a to-be-trained network layer in the pre-trained segmentation model; obtaining an output result of the to-be-trained network layer according to the target low-rank matrix, a routing decision value of the hybrid expert system and the sample image; according to a segmentation loss value between an image segmentation result corresponding to an output result of the to-be-trained network layer and an image segmentation label of the sample image, and an equilibrium loss value corresponding to each expert module in the hybrid expert system, obtaining a target loss value of the pre-trained segmentation model; and according to the target loss value, performing parameter efficient fine tuning on the to-be-trained model parameter iteration to obtain an image segmentation model. According to the method, the robustness is improved, the inter-task interference is reduced, and the task processing performance is improved.

The invention relates to the technical field of target detection, and particularly provides a semi-supervised learning-based target detection method, system and device and a storage medium, and the method comprises the steps: obtaining a data set which comprises a marked image and an unmarked image; generating a pseudo label for the unmarked image based on the marked image by using a dense detector; dividing the pseudo labels into reliable labels and soft labels according to the scores of the pseudo labels by using a pseudo label distributor; the pseudo label is updated to the data set, the updated data set is utilized to train a target detection model, and the target detection model identifies a target object in the sample image after training; the soft label is set to only participate in the preset partial loss calculation in the training process so as to reduce the training error caused by the soft label. According to the invention, an efficient semi-supervised learning target detection framework is constructed, and the accuracy of a target detection model is improved.

The embodiment of the invention provides a model training method and device, a target detection method and device and electronic equipment, and relates to the technical field of computer vision. The method comprises the following steps: acquiring a sample image and a label; for each sample image, utilizing an encoder in the target detection model to obtain a first sample feature vector; the model further comprises a decoder; the decoder comprises at least one partial cross attention module; for each part of the cross attention module, splitting input data in a channel dimension, and processing a first input part and a second input part by utilizing a self-attention layer and a cross attention layer of the module; processing results of the self-attention layer and the cross attention layer are combined to obtain output data of the module; obtaining a detection result by using output data of the last module; and calculating a loss value based on the detection result and the label, and adjusting model parameters to obtain a trained target detection model. Therefore, the time delay of target detection can be reduced.

A semantic segmentation model training method and apparatus, an electronic device and a storage medium are provided. The semantic segmentation model training method includes: acquiring a sample image, and extracting visual image features corresponding to the sample image by a semantic segmentation model to be trained; processing the sample image to obtain a text image feature corresponding to the sample image, the text image feature being an image feature generated from language description text for the sample image; fusing the visual image features with the text image feature to obtain multimodal features, and performing image segmentation prediction based on the multimodal features to obtain a target loss; and training the semantic segmentation model to be trained based on the target loss to obtain a target semantic segmentation model.

The invention relates to the technical field of image generation, and discloses a country style and appearance design image generation method and system based on a knowledge graph and a large model, and the method comprises the steps: constructing a core element covering country style and appearance design and a knowledge graph; constructing a multi-level label system based on the knowledge graph; pre-processed rural style and appearance sample images are obtained, the rural style and appearance sample images are labeled based on a multi-level label system, and a special training set in the field of rural style and appearance design is obtained; constructing a basic diffusion large model architecture, and training the basic diffusion large model architecture by using the special training set for the country style and appearance design field to obtain a trained country style and appearance design image generation model; and generating a rural style and appearance design image by using the trained rural style and appearance design image generation model. According to the method, the problems that a general large model is insufficient in domain knowledge support and insufficient in data labeling specialty in rural style and appearance design image generation are solved, and the rural style and appearance design image generation efficiency and quality are improved.

The embodiment of the invention provides a training method and device of an open vocabulary detection model and electronic equipment. The open vocabulary detection model training method comprises the steps of obtaining sample data; the sample data comprises a sample image, a sample text and label information, inputting the sample data into a neural network model to be trained, and extracting text feature information of the sample text and image feature information of the sample image; generating dynamic text prompt information according to the image feature information; performing fusion processing on the text feature information and the dynamic text prompt information to obtain semantic fusion feature information; predicting an object in the sample image according to the semantic fusion feature information; and performing iterative training on the neural network model according to the object prediction result, the label information, the text feature information, the dynamic text prompt information and the semantic fusion feature information to obtain an open vocabulary detection model. According to the invention, the context sensing, semantic generalization and fine-grained understanding capabilities of the open vocabulary detection model can be improved.

The embodiment of the invention provides a multi-modal large model training method and device, a target detection method and device and electronic equipment, and relates to the technical field of artificial intelligence, and the multi-modal large model training method comprises the steps: obtaining a plurality of sample images and a first sample description text of an object labeling box in the image; utilizing the first large language model and the first text prompt to extract a category name and an attribute of an object described by the first sample description text corresponding to each sample image, and combining at least one of the obtained category name and the attribute to obtain a second sample description text; determining whether each object labeling box in the sample image is matched with each attribute of a plurality of corresponding second sample description texts or not; constructing a sample question containing the second sample description text and a corresponding sample answer in a thinking chain form to obtain a question and answer sample pair; and training the multi-modal large model by using the constructed question and answer sample pair. The complexity of descriptive target detection can be reduced, and the detection efficiency is improved.

A semantic segmentation model training method and apparatus, an electronic device and a storage medium are provided. The semantic segmentation model training method includes: acquiring a teacher semantic segmentation model that is pre-trained, the teacher semantic segmentation model including a first teacher network and a second teacher network, the first teacher network having structural characteristics of low depth and high width, and the second teacher network having structural characteristics of high depth and low width; processing a sample image based on the teacher semantic segmentation model to obtain a first segmentation map and a second segmentation map; and training a student semantic segmentation model that is lightweight according to the sample image, the first segmentation map and the second segmentation map, so as to obtain a target semantic segmentation model.

The invention relates to the technical field of target recognition, in particular to a target recognition and model training method and device based on global perception graph convolution. The training method comprises the following steps: acquiring sample images and category labels, counting the co-occurrence probability of one or two same category labels in all the sample images, and generating a label adjacency matrix; inputting the sample image into a feature extraction network to obtain a first image feature, and generating a corresponding first adjacent matrix based on each local feature of the first image feature; inputting the first image feature and the first adjacent matrix into a first image convolutional network to obtain a second image feature; fusing the first image feature and the second image feature, and inputting the fused first image feature and second image feature and the label adjacent matrix into a second image convolutional network to obtain a prediction category of the target object; and calculating the difference degree between the prediction category and the category label, and adjusting the parameters of the model to obtain a trained target recognition model. The problem that a traditional classification network method is difficult to consider classification accuracy and efficiency in a complex scene is solved.

The invention relates to a generation method and device of a defective image, electronic equipment and a storage medium, and the method comprises the steps: obtaining a defective sample image, a defect-free sample image, a sample control condition and a sample instruction, the sample control condition is used for indicating a defect background, a defect position and a defect form in the defective sample image, and the sample control condition is used for indicating the defect background, the defect position and the defect form in the defect-free sample image; the sample instruction is used for describing a main category and a sub-category of the defect; sequentially performing coarse training and fine training on the initial image generation model based on the defective sample graph, the defect-free sample graph, the sample control condition and the sample instruction to obtain a trained image generation model; and inputting the defect-free image, the target control condition and the target instruction into the trained image generation model to obtain an output image with defects. According to the invention, the generation efficiency of the defect graph is improved.

The invention relates to a part defect detection method and system, and relates to the technical field of part defect detection. Comprising the steps of classifying and sorting a part processing library by using a part information sorting module, and constructing a generative adversarial network by using a sample image expansion module, generating a defect sample image to expand a defect sample database, constructing an image defect recognition model by utilizing an image recognition module to perform defect recognition, and performing defect state scoring and grade division operation on the part by utilizing a defect grade determination module. The defect sample database is expanded by constructing the generative adversarial network, data guarantee is provided for subsequent recognition, and recognition efficiency and accuracy are improved; besides, based on the score of the related defect of each feature surface, the state of the part is evaluated on the whole, and a data basis is provided for the subsequent defect grade and degree of the part, so that the part is reasonably defined, the treatment pertinence of the defective part is improved, the invalid loss is reduced, and the cost is reduced.