105 results about "Visual search tasks" patented technology

Visual search in an operating system of a computing device can process and provide additional information on the content being provided for display. The computing device can include an operating system that includes a visual search interface that obtains and processes display data associated with content currently being provided for display. The visual search interface can generate display data based on the current content provided for display, process the display data with one or more on-device machine-learned models, and provide additional information to the user. The visual search interface may transmit data associated with the display data to perform additional data processing tasks. Application suggestions may be determined and provided based on the visual search data.

The invention discloses a cascade architecture multi-modal large model retrieval question-answering method, and aims to improve the accuracy and efficiency of a question-answering system. The method comprises the following steps: firstly, searching multi-modal information related to query images and problem description from a knowledge base by utilizing a vector search technology through a visual search step; secondly, in the rough rearrangement step, paragraphs highly related to question description are screened out, and irrelevant content is eliminated; then, in the fine rearrangement step, the paragraphs screened out are deeply analyzed through a multi-modal large model, and the paragraphs most likely to contain correct answers are further sorted and selected; and finally, generating a final answer according to the selected paragraph content and the question description in the answer generation step. In addition, through the hybrid multi-modal retrieval strategy and the cascaded coarse-to-fine reordering method, the long-tail problem and the contradiction between efficiency and performance in the reordering stage are solved, and the system performance is remarkably improved. Experimental results show that the method is obviously superior to the existing most advanced method on multiple data sets.

Embodiments of the present invention provide systems, methods, and computer storage media for a visual and text search interface used to navigate a video transcript. In an example embodiment, a freeform text query triggers a visual search for frames of a loaded video that match the freeform text query (e.g., frame embeddings that match a corresponding embedding of the freeform query), and triggers a text search for matching words from a corresponding transcript or from tags of detected features from the loaded video. Visual search results are displayed (e.g., in a row of tiles that can be scrolled to the left and right), and textual search results are displayed (e.g., in a row of tiles that can be scrolled up and down). Selecting (e.g., clicking or tapping on) a search result tile navigates a transcript interface to a corresponding portion of the transcript.

Visual search in an operating system of a computing device can process and provide additional information on the content being provided for display. The computing device can include an operating system that includes a visual search interface that obtains and processes display data associated with content currently being provided for display. The visual search interface can generate display data based on the current content provided for display, process the display data with one or more on-device machine-learned models, and provide additional information to the user. The visual search interface may transmit data associated with the display data to perform additional data processing tasks. Application suggestions may be determined and provided based on the visual search data.

The utility model discloses a multifunctional mobile platform for detecting, interfering, trapping, striking and picking up an unmanned aerial vehicle. The multifunctional mobile platform comprises a mobile platform and portable reconnaissance and striking equipment which is arranged on the mobile platform and is used for striking the unmanned aerial vehicle, the mobile platform is provided with a positioning navigation antenna for autonomous navigation; the mobile platform is provided with a navigation laser radar for ground environment detection. An unmanned aerial vehicle countering antenna and an unmanned aerial vehicle countering controller are arranged on the mobile platform, and the unmanned aerial vehicle countering controller is in communication connection with the unmanned aerial vehicle countering antenna; a pickup unit is arranged on the moving platform, the pickup unit comprises a mechanical arm mounted on the moving platform, a pickup clamping jaw assembled at the end of the mechanical arm and a 3D camera, and the pickup clamping jaw, the 3D camera and the mechanical arm are in communication connection with the unmanned aerial vehicle countering controller. According to the utility model, the unmanned aerial vehicle countering controller sends a control instruction to the 3D camera and carries out visual search on the forced landing unmanned aerial vehicle; and a control instruction is sent to the mechanical arm and the picking clamping jaw to pick the unmanned aerial vehicle.

The invention relates to the technical field of visual search, in particular to a visual search experiment method for head fixing brain imaging. Comprising the following steps: fixing a target object on an experiment table, and training the target object through a display screen to obtain a trained target object; obtaining a target stimulus for an experiment and a target display strategy of the target stimulus in a display screen, the target stimulus being an icon for identification; and executing the target display strategy in the display screen, and obtaining behavioral statistical data and brain region microscopic imaging data of the trained target object based on the execution action of the trained target object. According to the method, the brain region microscopic imaging data of the trained target object can be obtained by executing the target display strategy corresponding to the target stimulation in the display screen, the display strategy of the target stimulation in the display screen is optimized, the motion noise interference of the target object is reduced, the brain region microscopic imaging data with higher quality is obtained, and the accuracy of the brain region microscopic imaging data is improved. And the comprehensiveness and the reliability of an experimental result are improved.

A head-mounted device may be used to perform a visual search on a physical environment around the head-mounted device. A user may wish to visually search one out of multiple physical objects in the physical environment. To clearly show the user which physical object was the target of a visual search, the head-mounted device may present a thumbnail of the candidate physical object on a display. The thumbnail may be cropped and / or zoomed using an image from a camera on the head-mounted device. By displaying a thumbnail that is taken by a camera on the head-mounted device, the thumbnail will directly match the physical object in the user's physical environment, eliminating ambiguity regarding which physical object is the subject of the visual search.

The invention relates to the technical field of robot visual guidance, and discloses a visual guidance robot material box grabbing module, which comprises a mechanical coarse positioning and state sensing unit for physically clamping a material box and collecting physical state data; a prediction type pre-positioning and dynamic reference generation unit receives the data, calls a historical deviation database, and generates a prediction target position and a temporary standard reference in parallel; the nested visual search and calibration unit moves to a prediction position and searches to obtain a complete feature image; the accurate deviation calculation and fine adjustment unit compares the image with a reference, calculates a final residual deviation and generates an accurate grabbing position; and the closed-loop learning and data feedback unit executes capturing and feeds back the final residual deviation to the historical deviation database. According to the method, a predicted target position and a dynamic temporary reference are established by using a physical state data vector and a historical deviation database, and parallel compensation of system errors and mechanical errors is realized.

Visual search query intent extraction and search refinement is described. In one or more implementations, a visual search query system receives a search query for items listed on an online marketplace, the search query including an image. Using one or more machine learning models, the visual search query system analyzes the image to determine characteristics of an object in the image. Based on the characteristics of the object in the image, the visual search query system automatically generates one or more search terms and searches the online marketplace to locate items matching the one or more search terms. The visual search query system then displays visual indications of the located items matching the one or more search terms in a user interface of the online marketplace.

The invention relates to the technical field of computer vision, discloses a visual retrieval enhancement prompt-based open vocabulary target detection method and system, and aims to solve the problem of poor rare category detection performance caused by training data long tail distribution in open vocabulary target detection. The method comprises two stages of off-line construction of a visual concept knowledge base and on-line retrieval enhanced reasoning, wherein a large language model is utilized to generate discriminative visual descriptors for each category, and the discriminative visual descriptors are encoded by a CLIP text encoder and then stored in an FAISS vector database; during reasoning, global visual features of an input image are extracted through a CLIP image encoder, the most relevant descriptor in the knowledge base is retrieved and added to an original category name prompt, and an enhanced prompt is formed and input to an open vocabulary detector to complete detection. Experiments show that the method obviously improves the detection performance in COCO and LVIS benchmark tests, and especially obviously improves the detection effect on rare categories.

The invention discloses a photovoltaic power station fire early warning method fusing visual search and a multi-mode large model, and relates to the field of fire early warning. According to the method, a smoke target recognition model built based on a multi-modal large model is utilized to perform preliminary recognition and positioning on a to-be-monitored image frame to obtain a smoke target positioning frame, a confidence coefficient and a smoke target search prompt heat map, and when the confidence coefficient of the smoke target positioning frame does not reach a confidence coefficient threshold value, the smoke target search prompt heat map is searched; furthermore, a visual search strategy is utilized to perform active search based on the smoke target search prompt heat map, an active search mechanism, a visual work memory and a dynamic adjustment strategy are introduced, and world knowledge and context understanding ability of a smoke target identification model can be fully utilized by simulating a smoke target identification mechanism of human beings; according to the method, reasoning can be directly carried out in an environment without training, high identification and positioning accuracy of a smoke target is achieved, early fire warning of the photovoltaic power station is effectively achieved, and the implementation difficulty is reduced.

The invention discloses a display method and system of an electronic rearview mirror based on multi-mode risk field coupling. The display method comprises the steps that driving state data of a current vehicle and one or more target vehicles and observation point data of a current vehicle driver are obtained; constructing a coupling dynamic risk field model based on the driving state data, and generating a comprehensive risk value; generating an attention entropy and a visual search efficiency value based on the observation point data; generating a display control signal by the comprehensive risk value, the attention entropy and the visual search efficiency value through a visual information flow optimizer based on an optimal control theory; and according to the display control signal, simplifying the background region by adopting an anisotropic diffusion equation, enhancing the foreground target by utilizing a nonlinear rendering pipeline, generating an electronic rearview mirror synthetic picture, and displaying the electronic rearview mirror synthetic picture on an in-vehicle display screen. According to the invention, the prominent presentation of key risk information and the effective compression of redundant information are realized, the cognitive load of a driver is reduced, and the driving decision-making efficiency and safety are improved.

The present disclosure relates to systems, methods, and non-transitory computer readable media that utilize deep learning to identify regions of an image that have been editorially modified. For example, the image comparison system includes a deep image comparator model that compares a pair of images and localizes regions that have been editorially manipulated relative to an original or trusted image. More specifically, the deep image comparator model generates and surfaces visual indications of the location of such editorial changes on the modified image. The deep image comparator model is robust and ignores discrepancies due to benign image transformations that commonly occur during electronic image distribution. The image comparison system optionally includes an image retrieval model utilizes a visual search embedding that is robust to minor manipulations or benign modifications of images. The image retrieval model utilizes a visual search embedding for an image to robustly identify near duplicate images.

Systems and methods for performing trick play functionality using trick play streams during adaptive bitrate streaming in accordance with embodiments of the invention are disclosed. One embodiment includes requesting a video container index from a video container file containing a video stream from a plurality of alternative streams of video; requesting at least one portion of the video stream using at least one entry from the video container index; decoding the at least one portion of the video stream; receiving at least one user instruction to perform a visual search of the media; requesting a trick play container index from a trick play container file containing a trick play stream; requesting at least one frame of video from the at least one trick play stream; and decoding and displaying the at least one frame of video from the trick play stream.

The invention discloses an external human-computer interface position determination method based on pedestrian visual search features, and particularly relates to the technical field of traffic safety, comprising the following steps: dividing a plurality of sight focusing parts according to a vehicle profile, and performing a pedestrian visual search test to collect test data; performing difference analysis on the data, and determining factors which have significant influence on the watching behavior of the testee; using a Markov chain to analyze the visual transfer characteristics of the pedestrian under different influence factor conditions, and quantifying the transfer probability of the pedestrian between different sight focusing parts under different influence factor conditions; through quantification of influence of factors on visual search, powerful support is provided for arrangement of the external human-computer interface position of the vehicle, and the arranged external human-computer interface position can be quickly noticed by pedestrians.

Systems and methods can be implemented to conduct a visual search as a service in a variety of applications. In various embodiments, a system is configured to provide searching capabilities of content provided by a first entity in response to a search request by a second entity. An image provided by the second entity can be used by the system as a query image to search the content of the first entity. In an embodiment, the first entity can be a commercial entity providing such a system with image related content regarding its products and services such that any number of individual consumers can search for particular products and services of the commercial entity via their communication enabled devices. In addition, such systems can be arranged for other embodiments to provide customized searches of a single source by many individual devices. Additional systems and methods are disclosed.

Described is a system and method for enabling visual search for information. With each selection of a search term, additional search terms are dynamically selected and presented to the user in conjunction with results matching the currently selected search terms. Likewise, a selected search term may be tokenized and a graphical token presented to the user to represent the selected search term.

Systems, methods, and computer program products for identifying a candidate product in an electronic marketplace based on a visual comparison between candidate product image visual text content and input query image visual text content. Unlike conventional optical character recognition (OCR) based systems, embodiments automatically localize and isolate portions of a candidate product image and an input query image that each contain visual text content, and calculate a visual similarity measure between the respective portions. A trained neural network may be re-trained to more effectively find visual text content by using the localized and isolated visual text content portions as additional ground truths. The visual similarity measure serves as a visual search result score for the candidate product. Any number of images of any number of candidate products may be compared to an input query image to enable text-in-image based product searching without resorting to conventional OCR techniques.

Systems and methods for performing trick play functionality using trick play streams during adaptive bitrate streaming in accordance with embodiments of the invention are disclosed. One embodiment includes requesting a video container index from a video container file containing a video stream from a plurality of alternative streams of video; requesting at least one portion of the video stream using at least one entry from the video container index; decoding the at least one portion of the video stream; receiving at least one user instruction to perform a visual search of the media; requesting a trick play container index from a trick play container file containing a trick play stream; requesting at least one frame of video from the at least one trick play stream; and decoding and displaying the at least one frame of video from the trick play stream.

The invention discloses a sketch retrieval method based on a modal fusion network, belongs to the field of computer vision, and develops a sketch retrieval visual search framework based on modal fusion by taking a user interface sketch as an efficient information representation in a software development process in mobile application program development. Making a user interface sketch data set by using a data synthesis technology, and preprocessing the data set; a network of a feature extraction encoder and a modal fusion operator is utilized to perform feature extraction on a user interface sketch, and a feature encoder module is responsible for extracting meaningful visual feature representation from an original image and a sketch of a UI example so as to accurately capture information such as design elements and layout. After the modal fusion network extracts visual feature representations from natural images and sketches, the visual feature representations are mapped to a unified modal fusion embedding space. By learning a representation space fusing modal data points and learning and calculating cross-modal attention mapping between the modal data points, the research can reserve semantically related and modal specific features so as to unify information of two modals. Through the proposed multi-modal embedding framework, not only can the structure of the original image and the sketch of the UI example and the joint feature of the associated content be learned, but also the multi-modal embedding framework can be applied to guide the retrieval process of the UI example sketch.

Systems and methods for identification and retrieval of content for visual search are provided. An example method includes receiving data specifying a digital supplement. The data may identify a digital supplement and a supplement anchor for associating the digital supplement with visual content. The method may also include generating a data structure instance that specifies the digital supplement and the supplement anchor and, after generating the data structure instance, enabling triggering of the digital supplement by an image based at least on storing the data structure instance in a database that includes a plurality of other data structure instances. The other data structure instances may each specify a digital supplement and one or more supplement anchors.

Described is a system and method for enabling visual search for information. With each selection of a search term, additional search terms are dynamically selected and presented to the user in conjunction with results matching the currently selected search terms. Likewise, a selected search term may be tokenized and a graphical token presented to the user to represent the selected search term.

The invention discloses a driver target attention method, system and device, a medium and a product, and relates to the field of target detection, and the method comprises the steps: constructing a driver target attention prediction model; the driver target attention prediction model comprises a target recognition model, a driving event classification model and an attention prediction model; the attention prediction model comprises an environment model fusing multi-layer visual memory and a target-level visual attention prediction model; obtaining visual behavior data and driving data of a driver in real time; and inputting the driver visual behavior data and the driving data into the driver target attention prediction model to obtain a visual attention prediction target. According to the method, the adaptive capacity of the driver visual target attention prediction model can be improved, the accuracy and robustness of model prediction are improved, the prediction result better fits the visual search mechanism of the driver, and the practicability of the driver visual target attention prediction model is improved.

A visual search based real time e-commerce system and method with computer vision is disclosed. A visual search engine (150) of the said system draws item detection and classification (110) sub module to retrieve information from a remote database (118). The said item detection sub module (110) on the server having instructions, that when executed by a processor, cause operations to be performed, wherein the operations includes receiving at least one image (102) and / or a video uploaded (104) by a user using a mobile application to the said visual search engine (150). The visual search (150) engine requests vendors registered in the system to quote availability and a cost associated, payment method, and shipping options (124) to a preferred location of the said user. The real-time e-commerce system trained using deep learning models accurately identifies and classifies the objects, maps the object label set with its respective product category which in turn, creates product attributes to search vendors in the specific category in real-time.

The invention relates to the technical field of cognitive deviation correction training, in particular to an old people subthreshold depression cognitive deviation correction training system which comprises an image processing module, an image storage module, a point probe output module, a visual search output module, a response data statistics module, an analysis calibration module, a detection implantation module and a unified processing module. Determining an attention change value for each user based on the response delay time length, dividing an attention category of each user based on the attention change value, and determining whether to correct the doping degree of the probe stimulation identifier output by the identifier generation unit based on the attention category; when it is determined that the attention category of the single user is the weak attention category, determining whether the emotional picture output for the single user is qualified or not based on the attention change representation value; personalized adjustment is carried out according to the specific condition of the user, the emotion picture is adjusted according to the response condition of the user, and the processing efficiency of the correction training data is improved.

The invention provides an online re-judgment method and system based on a large model and a visual retrieval screening method, and belongs to the related technical field of artificial intelligence and industrial quality inspection.The online re-judgment method comprises the steps that an efficient visual model is constructed through pre-training and generation technologies, and the efficient visual model is deployed to a production line for trial operation; carrying out high-value data collection and labeling by utilizing an active learning and automatic labeling technology, carrying out efficient visual model iteration, and starting mass production operation; the difficult cases detected by the model are re-judged through an industrial quality inspection large model and a retrieval enhancement technology, and data which are not confidence in re-judgment are handled manually. According to the method and the system, more than 80% of investment of manual re-judgment can be reduced, and misjudgment caused by subjectivity of manual re-judgment can be remarkably reduced, so that remarkable cost reduction and benefit increase are brought to enterprises; and popularization and application in the fields of quality inspection, screening and the like are facilitated.

The application discloses an online fiber visual searching system, which comprises luminous electronic tags, a first tag reader and a second tag reader. The luminous electronic tags are provided with inductive coils and luminous LED lamps. The luminous electronic tags are arranged at two ends of optical fibers, and the luminous electronic tags at the two ends of the same optical fiber have corresponding marks. The first tag reader is used for reading a single luminous electronic tag at a time. The second tag reader is used for reading a single or multiple luminous electronic tags at a time. The first tag reader and the second tag reader are in communication connection, and the first tag reader transmits tag information to the second tag reader after reading the tag information. The application has the advantages of realizing online optical fiber identification and being beneficial to fiber information management.

The invention discloses an attention state detection device, and relates to the technical field of computer vision. In the device, eye movement characteristics are acquired through a first acquisition module, then behavior characteristics are determined based on the eye movement characteristics in a first determination module, and finally, the attention state is determined based on a pre-established attention stage, a pre-established mapping relation between the behavior characteristics and a threshold value and the current behavior characteristics in a second determination module. In the attention detection device, the attention is divided into a plurality of stages in advance, such as a visual search characterization stage, a sight orientation characterization stage, an attention maintenance characterization stage and an attention concentration characterization stage, and a mapping relationship among the attention stages, behavior characteristics and threshold values is pre-established, so that the attention detection accuracy is improved. According to the invention, the existing attention stage and the missing attention stage can be determined based on the mapping relation and the current behavior characteristics, so that the user can know the specific stage where the attention is missing, and the accurate judgment of the attention state is realized.