99 results about "Image query" patented technology

The invention provides a cross-modal retrieval method for semantic and vector fusion in a data space, which belongs to the field of cross-modal information retrieval, and comprises the following steps: firstly, collecting and preprocessing multi-modal data; generating modal embedding and storing by utilizing the pre-training model; a shared semantic space is constructed, cross-modal vector alignment is optimized through comparative learning, and a modal mapping network is designed to enhance the embedding projection effect; storing the aligned embedding by using a Milvus database, and constructing an HNSW index; user text or image query is processed, text query analyzes limiting conditions to generate enhanced embedding, and image query extracts characters through OCR and fuses the characters with image features to generate embedding; in a database, through condition screening and semantic similarity calculation, a Top-K candidate item is retrieved; performing multi-modal correlation sorting on the candidate results and returning the results; according to the method, the shared semantic space is constructed, the alignment effect of different modal embedding is optimized, efficient storage and index management of multi-modal embedding are carried out, and real-time retrieval of large-scale cross-modal data is achieved.

The invention discloses an image cross-modal retrieval method and device based on a large language model and a medium, and the method comprises the steps: generating a text description corresponding to an original image through a BLIP model, and converting the text description into a corresponding image vector; establishing a mapping relationship between the image vector and the original image, and storing the mapping relationship and the image vector to a preset vector database; obtaining an image query description submitted by a user, optimizing the image query description through a preset language large model, and converting the optimized image query description into a corresponding query vector; determining a plurality of to-be-selected images corresponding to the image query description according to a first similarity between each image vector and the query vector in a vector database; and calculating a matching degree between the image query description and the plurality of to-be-selected images, and screening out a target image meeting the image query description from the plurality of to-be-selected images according to the matching degree.

The present disclosure describes methods, systems, apparatus, and media for object identification and classification, utilizing multi-feature and multi-modal data. This includes shape, material, brand, price, odor, taste, tactility, and sound. The system integrates a server space for data processing, a querying device for iterative searches, and a data interface module for refining results. It features AI-driven image optimization, feature extraction, and pattern recognition, employing novel techniques for fusing multi-feature and multi-modal embeddings utilizing multi-head attention. Additionally, a linker module powered by two active learning with feedback loops AI models consolidates scattered data into a unified object information database. The system also employs novel AI algorithms for isolating the object of interest through a saliency map and semantic analysis, as well as for enhancing raw images with a GAN-autoencoder.

The disclosure describes utilizing an image query system to improve response accuracy and reduce computational steps resources in responding to natural language queries of input images. In various implementations, the image query system utilizes grounding information from one or more sources to determine accurate information for an input image. For example, the image query system uses a single comprehensive image prompt to obtain extensive visual image grounding information for the input image from a visual-based large generative model. Additionally, or in alternative implementations, the image query system obtains reverse image search grounding information for the input image. The image query system then cleverly utilizes the grounding information with a large generative language model to generate text query responses to image-based queries of the input image more accurately and efficiently.

Implementations utilize an LLM to respond to queries comprising image data, such as multimodal queries that include both text and image data. A natural language processing system is extended such that when an image is provided, the natural language processing system invokes one or more auxiliary image processing models (e.g., visual query) and / or image search engines. The results, of invoking such model(s) and / or search engine(s), are collected into structured data signals related to the image. These signals form part of the conversation context and are used to extend the text prompt that is sent to the LLM. This allows the LLM to take the context into account when being used to process the user query, thereby enabling generation of an LLM reply that addresses relevant feature(s) of the image.

Systems, methods, and non-transitory computer-readable media implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, one or more embodiments involve receiving an input digital image and search input and further modifying the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and one or more embodiments involve retrieving the image search results utilizing a weighted combination of the queries. Some implementations involve generating an input embedding for the search input (e.g., the multi-modal search input) and retrieving the image search results using the input embedding.

The invention discloses an aerial view target detection method and system, a storage medium and equipment, and belongs to the field of BEV visual detection map reconstruction.The method comprises the steps that S1, images from different camera visual angles are input, and features of the different camera visual angles are extracted through a backbone network; s2, redesigning a time encoder through deformation and splicing by using long-period time information, and processing the characteristics of different camera visual angles through the designed time encoder; s3, based on a processing result of the step S2, further utilizing dynamic deformable attention to extract spatial features of the image; s4, processing the features obtained in the step S3 by using a feedforward network, and generating a final BEV feature map; and S5, performing target detection by using the final BEV feature map. According to the method, dynamic deformable attention is fused in aerial view angle detection, so that image query is more comprehensive, the calculation overhead is low, meanwhile, time sequence coding is improved, long-time historical information can be better integrated, and the robustness and accuracy of detection are improved.

The present disclosure relates to systems, methods, and non-transitory computer-readable media that implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, in one or more embodiments, the disclosed systems receive an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and the disclosed systems retrieve the image search results utilizing a weighted combination of the queries. In some implementations, the disclosed systems generate an input embedding for the search input (e.g., the multi-modal search input) and retrieve the image search results using the input embedding.

The present disclosure describes methods, systems, apparatus, and media for object identification and classification, utilizing multi-feature and multi-modal data. This includes shape, material, brand, price, odor, taste, tactility, and sound. The system integrates a server space for data processing, a querying device for iterative searches, and a data interface module for refining results. It features AI-driven image optimization, feature extraction, and pattern recognition, employing novel techniques for fusing multi-feature and multi-modal embeddings utilizing multi-head attention. Additionally, a linker module powered by two active learning with feedback loops AI models consolidates scattered data into a unified object information database. The system also employs novel AI algorithms for isolating the object of interest through a saliency map and semantic analysis, as well as for enhancing raw images with a GAN-autoencoder.

This disclosure describes a lightweight image query system that utilizes multiple lightweight models to quickly and efficiently generate responses to multimodal input queries. For example, in response to a multimodal input query, the lightweight image query system utilizes multiple lightweight context models to first obtain different types of context information based on the different input modes. The lightweight image query system then utilizes a lightweight large generative model (LGM) to quickly generate a query response using the different types of context information. By using lightweight models, including multiple lightweight context models and a lightweight LGM, the lightweight image query system can efficiently provide query responses to multimodal input queries in about half the time it takes conventional systems to return a query response.

The embodiment of the invention provides an information processing method and device and a commodity query method and device, and the information processing method comprises the steps: obtaining image-text query information, and determining an information attribute type corresponding to the image-text query information; determining image query information and text query information in the image-text query information, and constructing image-text fusion information according to the image query information and the text query information; carrying out object recall on the image query information, the text query information and the image-text fusion information to obtain an image recall object, a text recall object and an image-text recall object; and sorting the image recall object, the text recall object and the image-text recall object according to the information attribute type, and determining a target object corresponding to the image-text query information according to a sorting result.

The invention belongs to the technical field of surveillance video retrieval, and particularly relates to a pedestrian re-identification method and device and a medium. The method takes balance performance and efficiency as a core, a two-stage dynamic retrieval mechanism is provided, the two-stage dynamic retrieval mechanism comprises a coarse reasoning stage and a fine reasoning stage, and the difficulty degree of image query is judged through an early leaving strategy to realize on-demand distribution of computing resources: for simple query, global features are extracted only in the coarse reasoning stage for quick retrieval, and the calculation resources are extracted only in the fine reasoning stage; terminating subsequent reasoning; and for the difficult query, sending to a fine reasoning stage, and further extracting the features of the fine-grained part so as to carry out fine retrieval. In addition, in order to further reduce the calculation cost, the method also introduces a hybrid expert model to dynamically activate the network module to participate in calculation. By applying the method provided by the invention, the calculation cost can be effectively reduced and the retrieval efficiency can be improved while the retrieval accuracy is ensured.

The embodiment of the invention provides a nursing service monitoring method and device, equipment, a storage medium and a program product, and is applied to computer equipment, and the method comprises the steps: obtaining monitoring data of nursing service, employing a cue word engineering technology, carrying out the frame extraction processing of a video through a video analysis assembly, obtaining a plurality of groups of pictures, querying the video analysis rule configuration, and obtaining a monitoring result. Acquiring questioning points of the pictures, configuring and assembling cue words according to a video cue word rule, inputting the cue words into a large model to ask a question to the large model to obtain an analysis result of multiple groups of pictures, correcting, integrating, analyzing and judging according to a data processing rule to obtain a video analysis result, analyzing the audio through a voice recognition model to obtain an audio conversion text, and displaying the audio conversion text. And querying audio prompt word rule configuration to obtain a prompt word corresponding to an audio conversion text question point, inputting the prompt word into the large model to obtain an audio analysis result, and generating a nursing service monitoring report according to the video analysis result and the audio analysis result, thereby improving the monitoring efficiency of monitoring the nursing service quality.

The invention provides a long-distance crack detection method and system based on a sparse query mechanism, and belongs to the technical field of computer vision. The method is used for solving many technical problems existing in an existing crack detection method. The method comprises the steps of obtaining a two-dimensional image mode and a three-dimensional point cloud mode of a detection target; performing long-distance crack target preliminary detection on the two-dimensional image mode and the three-dimensional point cloud mode by adopting parallel mode independent detection branches to generate a two-dimensional long-distance crack detection result and a three-dimensional long-distance crack detection result; respectively carrying out sparse query on the two-dimensional and three-dimensional long-distance crack detection results to generate image query data and point cloud query data, and carrying out unified coding on the image query data and the point cloud query data; and performing long-distance crack target accurate detection on the uniformly coded image query data and point cloud query data to generate a long-distance crack detection result. The method is suitable for target detection research in the field of computer vision.

The invention relates to the technical field of image retrieval, and discloses a Hash image retrieval method and system based on improved triple loss, and the method comprises the following steps: inputting a query set image into a trained Hash network model based on improved triple loss, and obtaining a Hash code of the query set image; comparing the difference degree between the Hash code of the query set image and each Hash code in the Hash code library; sorting the difference degrees, and outputting a retrieval result according to the sorting; and calculating the average accuracy of the image query set according to the retrieval result. According to the invention, the retrieval precision can be improved.

The invention discloses an image query method and device, a storage medium and a computer program product. The method comprises the following steps: determining a first query vector corresponding to an image feature of a to-be-queried image; querying in an image retrieval database according to the first query vector to obtain a first image set; according to the first image set, a second query vector is determined, query is carried out in an image retrieval database according to the second query vector, a second image set is obtained, and the second query vector comprises a target class query vector and a confusion class query vector; and determining an image corresponding to the target class query vector in the second image set by adopting the classification model to obtain an image query result corresponding to the to-be-queried image. According to the method and the device, the technical problem that the risk of personal privacy disclosure possibly exists due to the fact that the image provided by a querier needs to be stored on a cloud server in a plaintext form when the image similar to the query image is retrieved in an image database in the related technology is solved.

The invention discloses a multi-target image query method and device and a storage medium, and relates to the technical field of computer vision, and the method comprises the steps: taking a plurality of associated first targets as a whole (i.e., a composite target), segmenting a single first target in an alternative image, and segmenting the composite target, so as to obtain a plurality of candidate images; and obtaining a single image and a first composite image. And then feature extraction is carried out on the single image and the first composite image, so that the obtained features comprise information of the first targets and information of the composite targets, and the extracted features can describe the alternative images more accurately. The alternative images are converted into more accurate feature descriptions, so that the detection precision is higher and the false detection rate is lower during subsequent image query. Therefore, according to the embodiment of the invention, the technical problem of false detection in a manner of returning the query result only by matching the features of the single or multiple first targets when the multiple targets are in contact with one another is solved.

The invention discloses a grain production-oriented cross-modal representation learning and retrieval method and system, and relates to the field of agricultural informationization, and the method comprises the steps: carrying out the multi-granularity semantic alignment of an image text pair in a grain production process based on an image-text bidirectional guidance fusion network, and obtaining a semantic segmentation image; performing image space decoupling and time sequence enhancement on a video text pair in a grain production process based on global semantic guidance to obtain a structured semantic image feature; constructing a text feature library and an image feature library; and determining a transmission plan matrix according to the modality of the to-be-retrieved data, generating query features of the to-be-retrieved data based on the transmission plan matrix, and outputting a text query result or an image query result by adopting a similarity measurement method according to the query features of the to-be-retrieved data, the text feature library and the image feature library. According to the method, deep fusion of cross-modal features can be realized, the semantic matching accuracy of the image and the text is improved, and rapid and accurate matching and retrieval between the image and the text are realized.

Systems, methods, and non-transitory computer-readable media implements related image search and image modification processes using various search engines and a consolidated graphical user interface. For instance, one or more embodiments involve receiving an input digital image and search input and further modify the input digital image using the image search results retrieved in response to the search input. In some cases, the search input includes a multi-modal search input having multiple queries (e.g., an image query and a text query), and one or more embodiments involve retrieving the image search results utilizing a weighted combination of the queries. Some implementations involve generating an input embedding for the search input (e.g., the multi-modal search input) and retrieving the image search results using the input embedding.

The invention belongs to the field of intelligent rail transit and computer vision, and provides a three-dimensional target detection and rail line identification method based on multi-modal fusion. According to the method, camera images, laser radar point cloud voxel features and point cloud features are extracted through deep learning, image query and point cloud query are generated according to an image query generator and a point cloud query generator, a unified query input detection head is formed through clustering fusion, and a target detection result is output. Meanwhile, a unified query generator is designed to generate a track line sensing query, and after fusion, a track line detection head is input to output a track line detection result. According to the method, multi-mode complementary information is effectively utilized, a coordinate system is unified to avoid conversion errors, and target detection and track line identification in a long-distance range in front of the track locomotive are realized by adopting a sparse query method.

This invention discloses a method and system for real-time monitoring and prediction of carbon intensity, belonging to the field of carbon emission monitoring technology. The method acquires and processes electricity-related data such as installed capacity and real-time load of each province hourly through an ETL data channel; it draws a map of China based on ECharts, dividing it into five color-level intervals according to carbon intensity values: 0-300, 301-500, 501-700, 701-900, and 901-1200 gCO2 / kWh; responding to user clicks on provinces, it displays detailed data for each province through indicator cards, circular progress charts, and one-sided bar charts; it calls a pre-trained STFCI hybrid model to predict carbon intensity hourly for the next 24 hours. This model integrates a 2-layer 128-neuron LSTM unit with an XGBoost unit with 100 trees and a learning rate of 0.1, outputting predicted values ​​after feature concatenation and a fully connected layer; the prediction results are displayed as a bar chart, marking customizable low-carbon threshold lines, highlighting low-carbon periods, and generating electricity consumption suggestions. The corresponding system includes a data interface, database, visualization, prediction, and WeChat service modules. The database adopts an InfluxDB+MySQL combined architecture, and the WeChat service module supports mobile text and image queries. The STFCI model of this invention has an average absolute error of only 12.3 gCO2 / kWh, significantly improving prediction accuracy, realizing hourly real-time monitoring and multi-dimensional visualization of carbon intensity, and providing precise decision support for power dispatching and energy conservation and emission reduction when combined with mobile services.

An object detection method and device based on an adaptive decoder, which constructs an object detection model AdaMixer, including a network configuration stage, a training stage, and a testing stage. Different-sized feature maps obtained in cooperation with a backbone network are combined into a 3D feature space, where efficient feature sampling is performed, and the sampled features are enhanced by adaptively coordinating the spatial information and position information of the query amount, thereby realizing the object detection task. The present invention effectively utilizes the information in the query amount through an adaptive module for different picture query amounts, avoids redundant network structures, saves computational amounts, and enables the network to converge quickly and stably. The sampling of the 3D feature space is introduced to efficiently encode the position information and semantic information, which can better cooperate with the adaptive module to flexibly, efficiently, quickly, and accurately complete the object detection task.

This application discloses a cross-modal representation learning and retrieval method and system for grain production, relating to the field of agricultural informatization. The method includes: performing multi-granularity semantic alignment on image-text pairs in the grain production process based on a bidirectional guided image-text fusion network to obtain semantically segmented images; performing image spatial decoupling and temporal enhancement on video-text pairs in the grain production process based on global semantic guidance to obtain structured semantic image features; constructing a text feature library and an image feature library; determining a transmission plan matrix based on the modality of the data to be retrieved; generating query features for the data to be retrieved based on the transmission plan matrix; and outputting text query results or image query results using a similarity measurement method based on the query features, text feature library, and image feature library. This application enables deep fusion of cross-modal features, improves the accuracy of image-text semantic matching, and achieves fast and accurate matching and retrieval between images and text.

The embodiment of the invention provides an image query method and device, equipment and a storage medium. The method comprises the following steps: determining a first group of images matched with a text query item from a first image set; determining a fused image feature by fusing a group of image features corresponding to the first group of images; and based on the fused image features, determining a second group of images from the second image set as a query result for the text query item. According to the embodiment of the invention, image query is carried out based on richer and more accurate fusion image features of the feature information, and the accuracy of image query can be effectively improved.

The present application provides an image query method, apparatus, and device, the method comprising: acquiring an initial image, the initial image comprising a first region of interest and a first noise region; generating a candidate image based on the initial image, the candidate image comprising a second region of interest and a second noise region; the image features of the second region of interest matching the image features of the first region of interest, and the image features of the second noise region not matching the image features of the first noise region; based on the initial image and the candidate image, querying a target image corresponding to the initial image from an image database; the image database being used to record images with identity information. Through the technical solution of the present application, an accurate target image can be queried based on the features of multiple frames of images to query a target image that matches the initial image. Even if only one frame of the initial image is provided, the target image can be queried based on multiple frames of images, thereby improving the performance of image queries.

An electronic device is provided. The electronic device includes a camera, a communication circuit, memory storing one or more computer programs, and one or more processors communicatively coupled to the camera, the communication circuit, and the memory, wherein the one or more computer programs include computer-executable instructions that, when executed by the one or more processors individually or collectively, cause the electronic device to extract, from an image obtained from the camera, a portion of the image including objects, determine a primary object to be searched for from among the objects in the image, determine an object associated with the primary object as a secondary object, from among other objects in the image, generate an image query including a portion of the image including at least one of the primary object or the secondary object, and including a request for information related to the primary object, and transmit the image query to a server through the communication circuit.

The application discloses a specific picture recognition method based on multi-modal, relates to the field of specific picture recognition, and comprises two parts of feature information extraction of pictures and texts and a specific picture matching strategy of FH-LANet, adopts an image matching idea to extract features of pictures through a multi-modal feature extraction network FH-LANet model, selects a specific picture as a query library picture, the query feature is not a single modal picture feature, but a text feature, similarity calculation is carried out by using the features of the input picture and the query feature, threshold judgment is then carried out, the matched picture and filtered text are matched again, noise data is filtered out, the picture that does not match the query picture is secondarily matched with the query text, and finally, the recognition result of the picture is obtained, and the specific picture in the mass pictures can be accurately found out.

The utility model discloses a wearable nursing worker housekeeping law enforcement instrument. The instrument comprises a camera shooting mechanism; the camera assembly is rotatably or fixedly arranged on the front surface of the camera mechanism; the control module is arranged in the camera mechanism and associated with the camera assembly, and at least one charging connector is arranged on the back face of the control module; the power supply mechanism and the camera shooting mechanism are attached to the inner side and the outer side of the clothes of the nursing and housekeeping personnel respectively and are magnetically attracted and positioned; the power supply assemblies are arranged in the power supply mechanism, power supply connectors are arranged on the front faces of the power supply assemblies, and the power supply connectors and the charging connectors conduct charging in a one-to-one or many-to-one mode in sequence. The nursing worker housekeeping law enforcement equipment can solve the problems that existing nursing worker housekeeping law enforcement equipment is low in pertinence and intuition of recorded content, short in endurance time and incapable of meeting the requirements of high endurance of the law enforcement equipment corresponding to nursing worker housekeeping long-time work, follow-up work image query and reward and punishment dispute handling convenience.

The invention discloses an image encoder-text encoder joint training method, an image query method, a health state evaluation method and equipment based on comparative learning, and relates to the field of medical image understanding. The method comprises the following steps: respectively inputting each 3D multi-modal brain MRI image and each Chinese medical report into an image encoder model and a text encoder model to obtain a one-dimensional image vector and a one-dimensional text vector; according to the image-text similarity matrix, the text-image similarity matrix and the semantic similarity matrix, using a fusion loss function to calculate the loss of the current iteration, and optimizing the parameters of the image encoder model and the text encoder model; and obtaining a trained image encoder model and a trained text encoder model when an iteration ending condition is met. Downstream tasks such as zero sample classification and cross-modal mutual retrieval of images and texts are realized.

The embodiment of the invention relates to a medical image query and comparison system. The system comprises an image database, a query server and a plurality of first clients, and the query server is respectively connected with the image database and each first client. The image database is used for storing a plurality of image data records. The query server is used for performing multi-dimensional query on the image database based on the multi-dimensional query configuration sent by the first client; the first client is used for generating multi-dimensional query configuration, sending the multi-dimensional query configuration to the query server, displaying a query list according to server feedback data, generating a multi-selection target set according to a selection result of a user on the query list, and registering and aligning all images of the multi-selection target set; and the synchronous operation module is used for carrying out synchronous operation on loading images in other image areas according to operation actions of a user in any image area. On the basis of the method, the query operation can be simplified, the query efficiency is improved, the big data value is improved, and the operation convenience of multi-image comparison is improved.