230 results about "Object description" patented technology

The invention relates to the technical field of audio and video control, and discloses an audio and video player control method based on a voice instruction. The method comprises the steps that an original voice instruction stream of a user is collected, the instruction stream comprises a time domain audio signal sequence, an environment noise spectrum and user pronunciation characteristic parameters, and voice information can be comprehensively captured; multi-modal instruction analysis processing is carried out on the original voice instruction stream, a structured control instruction set containing acoustic control intention identification, semantic operation object description and context correlation parameters is generated, and the analysis precision is improved; then executing player state adaptation based on the set, generating a dynamic control response sequence containing an equipment state adjustment command, a media content positioning parameter and an interface interaction logic identifier, driving a player to execute a multi-dimensional control operation and generating real-time play control effect feedback data; and finally, multi-modal analysis parameters are optimized according to feedback data, a self-adaptive instruction analysis strategy is generated, and the control experience of a user on the audio and video player is optimized.

The invention provides an image privacy positioning identification method and device based on a multi-modal large model, and the method comprises the steps: obtaining an initial query feature based on a semantic token, and generating a condition query feature based on the initial query feature, a local visual token sequence and a global visual token; determining a target query feature based on the fusion feature and the condition query feature; inputting the target query feature into a multi-modal large model to obtain a prediction position and a prediction category; if the similarity between the prediction category and the privacy object description is greater than a semantic similarity threshold, determining that the privacy object concerned by the user exists in the to-be-detected image, and desensitizing the content matched with the prediction position; and if the similarity is not greater than the semantic similarity threshold, determining that a privacy object which the user does not pay attention to exists in the to-be-detected image, and not desensitizing the content matched with the predicted position. Through the scheme of the invention, the privacy information in the image can be effectively identified.

The invention discloses an intelligent SQL (Structured Query Language) generation method and system based on multistage intention recognition. The method comprises the following steps: receiving a natural language query of a user; vector matching: carrying out vector matching based on a knowledge base to obtain Top-K candidate query objects, the knowledge base containing multilayer structure description information; a large language model intention understanding step: determining a final query object in combination with the candidate query object description; loading a branch workflow, analyzing a field and a table structure, and dynamically configuring a database table and field information; sQL generation: generating SQL statements conforming to grammatical rules, including automatic field addition, time condition conversion and field priority matching operation; and executing the SQL statement and returning a result, and if the result fails, recording a log to prompt correction. By combining a natural language processing technology with a database knowledge base, an efficient, accurate and user-friendly database query solution is provided.

Techniques for image data processing and image detected are described herein. Techniques may include obtaining a source image that includes a target object, and obtaining a local feature sequence of the target object from the source image; performing location encoding processing on the local feature sequence to obtain location encoding information of the local feature sequence, and combining the local feature sequence and the location encoding information into an object description feature associated with the target object; obtaining an attention output feature of the object description feature; the attention output feature being configured for representing an information transfer relationship between global features of the target object; and determining an object encoding feature of the source image according to the object description feature and the attention output feature; and determining keypoint location information of the target object in the source image based on the object encoding feature.

Embodiments of the present description provide an image object detection method. The method comprises: on the basis of an image to be retrieved, an object description text, and an object retrieval condition, determining, by means of an object detection model, a target position of an object to be retrieved in said image, wherein the object description text is used for describing said object, and the object retrieval condition comprises at least one of a mask image, a pose, and a texture corresponding to said object.

The invention discloses a mandatory access control method and device based on a process function context, and the method comprises the steps: collecting a security context associated with a system call initiated by a target process, so as to generate a standardized object description; mapping the object description into a target function classification identifier, so as to obtain a process function context view of the target process according to the target function classification identifier; constructing a target decision key for access decision based on the current policy era, the qualifier, the function classification identifier, the view identifier of the process function context view and the isolation domain abstract; and querying the multi-level cache according to the target decision key to determine a matched target access decision. Therefore, context-sensitive judgment and cross-component consistency taking the functional context as the center are realized.

A method of generating a virtual avatar based on a large model, an agent, an electronic device and a storage medium, which relate to a field of artificial intelligence technology, and to fields of computer vision technology, deep learning technology, large model technology, etc., and may be applied to scenarios such as AIGC, digital character, intelligent e-commerce, etc. The method includes: processing a target image including a target object by using a large model to obtain object description information, the target object having texture information; processing the target image and a to-be-processed image representing an object morphology of a three-dimensional object by using a texture-generative large model to obtain a target three-dimensional object with target texture information, the three-dimensional object being determined based on the object description information, the target texture information being matched with the texture information; and generating the virtual avatar based on the target three-dimensional object.

The invention discloses an image generation model construction method and a three-dimensional digital model determination method. The method comprises the following steps: firstly, obtaining a first object description image, a first description text corresponding to the first object description image, and a first attitude image corresponding to the first object description image; and constructing an image generation model by using the first object description image, the first description text and the first attitude image, so that the constructed image generation model can generate image data with relatively high quality according to the first description text and the first attitude image. Therefore, the image generation model can achieve the purpose of generating the image data according to the text information and the posture information, the image generation effect of the image generation model is improved, and a three-dimensional digital model determined by means of the image generation model can also conform to natural posture constraints. And the determination effect of the three-dimensional digital model can be improved.

Object description information that is transmitted by a biometric recognition apparatus is received, the object description information includes a biometric feature of a target object and location information of the target object. Identity information of the target object is obtained according to the biometric feature. A service information set in association with the identity information is obtained. From the service information set, one or more pieces of candidate service information are selected. The one or more pieces of candidate service information are transmitted to a terminal device associated with the identity information. At least a first piece of target service information returned by the terminal device is received. At least a first service corresponding to the first piece of target service information is processed. Apparatus and non-transitory computer-readable storage medium counterpart embodiments are also contemplated.

The embodiment of the invention provides a training method and device, an index construction method and device and a searching method and device based on multi-modal information. The model training method comprises the steps that a pre-trained generative model is acquired, and the generative model comprises an image-text processing unit and a language model unit; obtaining an instruction word generated based on the cue word and object association information of the target object, wherein the object association information comprises object description information and image information; constructing input data based on the instruction word, the object description information and the image information, inputting the input data into a generative model, and outputting a corresponding keyword; calculating loss information through the keyword; and adjusting an image-text processing unit and a language model unit of the generative model based on the loss information. The precision of the generated model can be improved, so that the accuracy of keywords is improved, the accuracy of search results is improved, and the user experience is improved.

Disclosed are apparatuses, systems, and techniques for automated iterative content detection and annotation of objects in media items. The techniques include performing a plurality of iterations to identify objects represented in a media item and referenced in a plurality of object descriptions of a prompt. An individual iteration includes identifying, using a content detection model, a subset of the objects represented in the media item and referenced in the plurality of object descriptions, or no objects represented in the media item and referenced in the plurality of object descriptions. Using the content detection model includes applying the content detection model to the media item and to the prompt or to an iteration prompt obtained from the prompt by eliminating descriptions of the subsets of the objects identified during previous iterations. The techniques further include generating, using the identified objects, a characterization of the media item.

The invention provides a data processing method and device based on artificial intelligence, electronic equipment, a computer readable storage medium and a computer program product. The method comprises the following steps: acquiring an object grid model and a first texture parameter corresponding to the object grid model, and performing deformation processing on the object grid model based on a first weight parameter to obtain a first deformed object grid model; performing rendering processing based on the first deformation object grid model and the first texture parameter to obtain a first rendered object image, and performing image generation processing based on the object description text to obtain a generated object image conforming to the object description text; determining image semantic loss based on the first rendered object image and the generated object image, and updating the first weight parameter based on the image semantic loss to obtain a second weight parameter; and performing deformation processing on the object grid model based on the second weight parameter to obtain a second deformation object grid model. According to the invention, the modeling accuracy of the second deformation object grid model is improved.

A method, apparatus, non-transitory computer readable medium, and system for image processing include obtaining an input prompt including an image quality level and a description of an object, generating an image embedding based on the input prompt, where the image embedding represents the object and the image quality level in a vector space, and generating a synthetic image based on the image embedding, where the synthetic image depicts the object and has the image quality level.

The invention belongs to the technical field of image processing, and provides a text-guided zero sample target counting method, a program product and electronic equipment, and the counting method comprises the steps: inputting a query image and a target description text into a target counting model to obtain a density estimation graph of a target; the target counting model comprises a visual feature extraction module; a text feature extraction module; the multi-level polarity dual attention module comprises a plurality of cascaded feature fusion modules; each feature fusion module performs visual polarity cross attention processing on the visual-to-text fusion features or the visual features to obtain visual-to-text fusion features of the hierarchy, and performs text polarity cross attention processing on the text-to-visual fusion features or the text features to obtain text-to-visual fusion features of the hierarchy; the decoder is used for decoding the visual-to-text fusion features to obtain a density estimation graph of the target; according to the method, the information of the negative polarity part is reserved, and the accuracy of the density estimation graph of the target is improved.

The invention provides a live broadcast processing method and device, equipment and a storage medium, and the method comprises the steps: responding to a live broadcast plan generation operation, displaying to-be-recommended object information and live broadcast verbal skill information corresponding to a target live broadcast account number on a live broadcast plan page, the to-be-recommended object information comprising object attribute information of a to-be-recommended object, and the to-be-recommended object attribute information corresponding to the target live broadcast account number; the to-be-recommended object is determined according to historical interaction data corresponding to the target live broadcast account and object attribute information of the candidate to-be-recommended object, and the live broadcast verbal skill information at least comprises live broadcast explanation information corresponding to the to-be-recommended object; the live broadcast explanation information corresponding to the to-be-recommended object is determined according to the object description information of the to-be-recommended object. According to the embodiment of the invention, the to-be-recommended object information corresponding to the target live broadcast account and the live broadcast verbal skill information are displayed on the live broadcast plan page, and intelligent generation of the live broadcast plan is supported, so that the preparation work before live broadcast is started is simplified, and the efficiency of the preparation work before live broadcast is started is improved.

The invention belongs to the technical field of computer images, and provides an object counting method and device for enhancing text guidance by utilizing frequency characteristics. The method comprises the following steps: inputting a query image to a visual encoder to obtain visual features; inputting the object description text to a text encoder to obtain text embedding; fusing the visual features and the text features by using a fusion network to obtain fusion features; the fusion features are input into a decoder to obtain a density estimation graph of the object, the decoder comprises K cascaded adaptive frequency selection modules and one output layer, and the kth adaptive frequency selection module comprises a first convolution unit, an adaptive frequency selector and a first up-sampling unit; the kth self-adaptive frequency selector carries out filtering processing on the amplitude spectrum and the phase spectrum of the output characteristic of the first convolution unit in the frequency domain, and converts a filtering processing result into a spatial domain; according to the method, related frequency components are dynamically emphasized in decoding, the space domain and frequency domain features are combined, and accurate object positioning and accurate counting are achieved.

The invention relates to an information pushing method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining candidate product features and object description features; the candidate product features and the object description features are combined to obtain shared features, at least two autocorrelation degrees of the shared features are calculated, the shared features are transformed based on the at least two autocorrelation degrees, and autocorrelation features corresponding to the at least two autocorrelation degrees are obtained; taking the candidate product features and the object description features as input features, and fusing the input features with the self-correlation features corresponding to the at least two self-correlation degrees in sequence to obtain target fusion features; and calculating an interaction degree of the to-be-pushed object to the product information based on the target fusion feature, and pushing the product information of the candidate product to a terminal of the to-be-pushed object when the interaction degree meets a preset pushing condition. By adopting the method, the information pushing accuracy can be improved.

The invention provides an industrial robot self-adaptive grabbing method based on multi-modal sensing, which comprises the following steps of: 1, acquiring multi-modal data of an object and an environment through a multi-modal sensing module; the multi-mode sensing module comprises at least two sensing modules of a visual sensing unit, a touch sensing unit, a force sensing unit and an auditory sensing unit; 2, preprocessing and feature extraction are carried out on the multi-modal data, feature information of different modals is fused through a multi-modal fusion algorithm, and comprehensive object description information is generated; 3, generating an optimal grabbing strategy based on the comprehensive object description information; wherein the grabbing strategy comprises the position and posture of a grabbing point, a grabbing path and grabbing force; and fourthly, the industrial robot executes the optimal grabbing strategy, the first step to the third step are repeated, and the grabbing strategy is adjusted in a self-adaptive mode till grabbing is completed. Stable and efficient grabbing can be achieved.

The present invention provides a method, device and electronic device for intelligent control of a robotic arm, which are applied to the fields of artificial intelligence and computer vision. The method comprises: obtaining an environmental observation image and an environmental depth image; inputting a target language token sequence into a large language model to obtain a target object description text and a target task description text; inputting the target object description text and the environmental observation image into an open world environmental perception model to obtain a first mask image of the target object; obtaining a point cloud image of the target object based on the first mask image and the environmental depth image; inputting the target object description text, the target task description text and the point cloud image into the large language model to obtain the pose estimation information of the robotic arm; and controlling the robotic arm to perform corresponding operations corresponding to the target task description text on the target object based on the pose estimation information.

The embodiment of the invention provides a task processing method, a dialogue task processing method, a task planning model training method, an information processing method based on a task planning model and a model training platform. The task processing method comprises the steps of obtaining task data of a target task; the task data and the object description information of the candidate processing objects are input into a task planning model, multiple processing objects corresponding to the target task are determined, and the task planning model is used for planning the multiple processing objects of the target task; processing the task data by using the plurality of processing objects to obtain processing results respectively output by the plurality of processing objects; and determining a task processing result of the target task according to the processing results output by the plurality of processing objects. The task planning model is utilized to determine the plurality of processing objects for processing the target task, so that seamless integration between the model and the processing objects with different processing capabilities is realized, the task processing capability is expanded, and the task processing is more efficient and comprehensive.

This disclosure addresses the unresolved problems of tackling object disambiguation task for an embodied agent. The embodiments of present disclosure provide a method and system for disambiguation of referred objects for embodied agents. With a phrase-to-graph network disclosed in the system of the present disclosure, any natural language object description indicating the object disambiguation task can be converted into a semantic graph representation. This not only provides a formal representation of the referred object and object instances but also helps to find an ambiguity in disambiguating the referred object using a real-time multi-view aggregation algorithm. The real-time multi-view aggregation algorithm processes multiple observations from an environment and finds the unique instances of the referred object. The method of the present disclosure demonstrates significant improvement in qualifying ambiguity detection with accurate, context-specific information so that it is sufficient for a user to come up with a reply towards disambiguation.

The invention relates to the technical field of visual reasoning, and provides a safety monitoring multi-modal model reasoning method and device. According to a user problem and a user image in a security monitoring scene, an object position in a visual scene is converted into text information, and the text information, the user problem and the corresponding user image serve as input information; obtaining visual features according to the input information through a cross-modal semantic converter; constructing a visual scene of the user image into hierarchical description comprising scene description and object description; modeling the context of the user image according to the user question, and generating a text prompt of a visual scene; reasoning is carried out through a large language model according to the visual features, the hierarchical description and the text prompt, reasoning output is obtained, and the problems that in a multi-modal scene, the complex scene perception ability is insufficient, and the large language model reasoning ability is insufficient in utilization in the prior art are solved.

The embodiment of the invention provides a query result generation method and device.The query result generation method comprises the steps that based on user information, a user intention is determined through a language model; based on the user intention, determining an object description and determining a sub-data table from a data table, the data table storing table data of a plurality of objects; and generating a query result through a language model based on the object description and the sub-data table.

The invention relates to an interaction method and device, electronic equipment and a storage medium, and the method comprises the steps: receiving a first reference image, and displaying the identification information of the received first reference image in a guide language input region; the number of the first reference images is multiple; receiving a text cue word, and displaying the text cue word in the guide word input area; the text cue word comprises an object description word group, and the object description word group corresponds to the first reference image; the object description phrases are used for describing objects in the first reference image corresponding to the object description phrases; the identification information of the first reference image is interspersed in the text cue word; the number of characters between the position of the object description phrase in the text cue word and the position of the identification information of the first reference image corresponding to the object description phrase in the text cue word is less than a preset number of characters; a first video is generated based on the content in the guide language input area. The demand that a user hopes to generate a video based on a plurality of first reference images can be met.

The invention provides an image generation method and device, equipment, a readable storage medium and a program product, and can be applied to the fields of games, artificial intelligence and the like, the method comprises the steps that input texts of a game object description text and an image feature description text are acquired, and the game object description text comprises a game object style identifier and a game object category; image generation processing is carried out according to the input text, multiple candidate game object images are obtained, and the candidate game object images comprise target game objects; quality detection is conducted on the multiple candidate game object images, a target game object image is determined from the multiple candidate game object images according to the quality detection result, and the target game object image is the game object image with the highest image quality of the target game object included in the multiple candidate game object images. By adopting the mode, the game object image can be efficiently generated according to the input text, and the image quality of the game object contained in the generated game object image is high.

The invention discloses a distributed operation-oriented multi-agent collaborative recommendation method and system, and relates to the technical field of intelligent recommendation, and the method comprises the steps: configuring a private domain operation agent in each independent private domain platform, deploying a big language model-based construction demand analysis agent at a user side to receive a natural language demand description input by a user, and constructing a multi-agent collaborative recommendation system; generating recommendation task instructions of different private domain platforms; the method comprises the following steps: receiving user preference description and recommendation object description, sending the description to a corresponding private domain operation agent, calling a recommendation model to generate a recommendation result list based on the received user preference description and recommendation object description, collecting recommendation result lists returned by all private domain operation agents by a demand analysis agent, and integrating and classifying the recommendation result lists to generate a comprehensive recommendation list. According to the method, an efficient, safe and extensible distributed recommendation architecture is constructed by introducing a user demand analysis agent, a private domain operation agent and a federal learning mechanism.

The invention relates to the technical field of data processing, can be applied to various scenes such as cloud technology, artificial intelligence, intelligent traffic and auxiliary driving, and particularly relates to an anomaly detection method and device, electronic equipment and a storage medium, and the method comprises the steps: obtaining multimedia data and object description data associated with a target object; according to an input feature type corresponding to a current effective anomaly detection model, to-be-input features are extracted from the multimedia data and the object description data, the effective anomaly detection model is a historical anomaly detection model, or after the historical anomaly detection model reaches a performance anomaly condition, the to-be-input features are extracted from the multimedia data and the object description data; training the obtained target anomaly detection model; and adopting the effective anomaly detection model to obtain an anomaly detection result based on the to-be-input features. Therefore, the processing abnormality of the historical abnormality detection model can be timely detected, the situation that a model with poor performance is adopted for abnormality detection for a long time is avoided, and the abnormality detection accuracy can be improved.

The invention discloses a data information search method and device based on large model enhancement, equipment and a medium. The method comprises the following steps: acquiring search text information input by a search object; and according to the search text information, the object description information corresponding to the search object and a target text optimization model obtained by pre-training, determining search key information of a corresponding point of the search text information. And determining search vector information corresponding to the search key information, determining a vector search result in the data index database, and determining an entity search result in the data index database according to the search key information. And determining a target search result corresponding to the search text information according to a target result evaluation model corresponding to the search object. According to the method, intention recognition, user input enhancement and better understanding of the user intention are carried out by utilizing a large model during search, search optimization is carried out in a mixed retrieval mode, and more accurate and personalized intelligent retrieval is provided in combination with a target result evaluation model.

The invention discloses a cross-platform page code generation method based on a large model, and relates to the technical field of front-end development, and the method comprises the following steps: cooperatively analyzing a UI design drawing through a plurality of special large models, and carrying out real-time cross validation and dynamic compensation on an analysis result by adopting a multi-model mutual verification mechanism; based on a predefined standard specification, dynamic adjustment is carried out through a platform adaptation rule base; generating a page object description tree embedded with the input / output processing function; cross-platform page codes are generated through a code generation engine, and sandbox testing and automatic correction are executed; outputting a target platform code by utilizing a unified compiling engine; an adaptive optimization loop is constructed based on test feedback. According to the method, through integration of multi-model collaborative analysis, platform rule dynamic adaptation, sandbox verification and closed-loop optimization, the problems of large analysis deviation, poor cross-platform compatibility, uncontrollable code quality and the like are solved, and end-to-end high-quality automatic generation from a design drawing to multi-platform codes is realized.

The invention relates to an information display method and device, electronic equipment and a storage medium, and the method comprises the steps: responding to a video browsing instruction, and displaying an object recommendation video of a target object; in the object recommendation video display process, displaying an object recommendation card including preset comment information; the preset comment information is at least one piece of comment information for the target object in the comment information of the object recommendation video, and the object recommendation card is used for presenting description information corresponding to the target object in a card style. According to the embodiment of the invention, the information presentation of the object recommendation card can be improved on the basis of enriching the object description information, so that the object recommendation effect can be better improved, and meanwhile, the convenience of obtaining the comment information can be greatly improved.