1159 results about "Textual information" patented technology

The invention relates to the technical field of intelligent vending, solves the problem that in the prior art, commodity identification cannot be accurately carried out in a multi-target scene, and provides a multi-target commodity identification method, device and system based on multi-modal data processing. The method comprises the following steps: acquiring multiple frames of real-time images in a commodity transaction scene; performing preprocessing and label information extraction on the real-time image, and determining character information corresponding to the target image and the commodity label; performing instance segmentation on the target image, and determining commodity position information; performing feature extraction on the target image, and determining commodity image feature information; according to pre-collected multi-source privatized data in an intelligent vending scene, performing fine adjustment and optimization processing on the open-source multi-modal visual language model to obtain a multi-modal large model; and inputting the commodity image feature information and the text information into the multi-modal large model for information fusion, and determining a commodity target identification result. According to the invention, commodity identification can be accurately carried out in a multi-target scene.

The technology provides rewriting suggestions for communication styles in different pre-defined styles. Rewriting may convert text in different ways, including visually enhancing the message. A method provides input to a trained large language model, the input including curated examples associated with one or more writing style choices. The set of curated examples has a first size. The method also includes generating, using the model, a rewriting corpus according to one or more writing style choices. The rewriting corpus has a size two or more orders of magnitude larger than a size of curated examples. The writing style choices include at least one of tone, conversion, application context, or conversation type. A text rewriting model is trained using at least a subset of the rewriting corpus. The model is configured to generate vivid textual information in response to user input in an interactive domain, according to specific writing style choices.

Automatic recognition of non-verbal messages in speech, and in particular to detection or analysis of prosodic multilayered analysis of intonation units such as prosodic unit prototypes and their multi-labeled variations, may form a hierarchical classification for the analysis of non¬ verbal information or cues in speech. A speech captured by a microphone is fed to a weakly- supervised deep learning acoustic model for speech recognition and transcription, that may be based on encoder-decoder Transformer architecture, such as Whisper by OpenAI. The model is trained to output multiple words form the text in the captured speech, to identify Intonation Units (IUs) that include one or more words, and associate non-verbal labels to each of the IUs. The labels may indicate a prototype, a discourse function (such as a conversation action), an emotion, an emphasis, or an attitude, as well as a genre of a part of, or whole of, the entire captured speech.

The invention discloses a virtual reality scene three-dimensional reconstruction method and system based on multi-source data fusion, and relates to the technical field of underground commercial and traffic integrated space scene three-dimensional reconstruction. The system constructs a multi-dimensional data set through synchronous acquisition of point cloud, images, poses, positioning and semantic text information; a three-dimensional convolutional neural network is adopted to establish an AI reconstruction fusion model, and a three-dimensional model is generated; the system calculates a structural integrity evaluation coefficient JGPG, a fusion consistency evaluation coefficient RHPG and a VR interaction adaptability evaluation coefficient VRJH, compares the coefficients with corresponding thresholds, triggers an optimization strategy, and finally completes model adaptability optimization and packaging output, thereby improving the reduction degree and interaction performance of a virtual reality scene.

The invention relates to the technical field of multi-modal large language models, in particular to a government affair service content navigation method and system based on a large language model, and the method comprises the following steps: receiving multi-modal information input by a user through texts, voices or pictures; the voice is converted into a text, and character information in the picture is analyzed by using an OCR (Optical Character Recognition) technology; fusing multi-modal data, and inputting the fused multi-modal data into a large language model for semantic understanding and context association analysis; matching items are retrieved in combination with a local government affair knowledge base, and an initial recommendation list is generated; the recommendation result is displayed through the intelligent assistant, and interaction optimization options are provided; the method has the beneficial effects that the current government affair service content navigation mode is optimized through the multi-modal capability, the retrieval enhancement capability and the content generation capability of the large language model, so that the navigation is more modal and more intelligent, and meanwhile, the privacy and authority of data reply are ensured.

The invention discloses a recommendation method for enhancing semantics and interest perception by using a large language model. The method comprises the following steps: firstly, performing semantic modeling on unstructured text information such as user comments, article description and the like by utilizing the powerful capability of a large language model in semantic comprehension and user preference modeling aspects, so as to improve the deep perception capability of a recommendation system on user interests and article semantic attributes; then, through a semantic feature alignment and discretization strategy, the problem that continuous semantic representation generated by a large language model is incompatible with features of a traditional recommendation system in the aspect of an expression structure is solved; finally, unified modeling of semantic information and traditional recommendation signals is achieved through a recommendation integration mechanism, and recommendation performance and model interpretability are improved.

The invention relates to the technical field of video detection, in particular to a monitoring video detection and identification method and system based on deep learning, and the method comprises the steps: obtaining a target monitoring video of a monitoring region based on monitoring camera equipment, obtaining a video frame sequence of the target monitoring video, and carrying out the target screening of the video frame sequence, so as to obtain a video frame set; and performing target detection on each video frame in the video frame set based on a trained target detection model to obtain a target detection result set of each video frame in the video frame set. According to the method, through combination of keyword extraction and a large language model, complex monitoring data can be converted into structured event description, the textualized information not only facilitates management personnel to rapidly know the field condition, but also can provide reference for subsequent event processing, and the event description generated in real time is combined, so that the management personnel can conveniently and rapidly know the field condition. And a manager can make a coping strategy more efficiently.

The invention relates to the technical field of natural language processing technologies, and provides a sequence character bill image question and answer data generation method based on a multi-modal large model. The method comprises the following steps: detailedly describing the content of a target image; enabling the multi-modal large model to perform self-questioning and answering on the character information, and generating questions and answers of the character information; enabling the multi-modal large model to perform secondary answering on the proposed question and reasoning to generate an answering basis or reason at the same time; enabling the multi-modal large model to judge whether the question-answer pairs are related to the character information or not, and performing consistency check on the question-answer pairs; and deleting irrelevant and inconsistent question and answer pairs. According to the method, a multi-mode large model is used, and large-scale batch automatic generation of image text information question and answer data in a receipt scene is realized. Therefore, the defects of large-scale multi-mode instruction fine adjustment data taking characters as the center for receipts in quantity and quality are overcome, and the high time and labor cost for obtaining large-scale image-text understanding question and answer data annotations is overcome.

The invention belongs to the field of artificial intelligence, and particularly relates to a multi-modal knowledge matching system which comprises a construction module, an image processing module, a matching module and an output module to realize seamless connection of an image and a text knowledge base. According to the method, the user is allowed to use the picture to perform knowledge retrieval, the user experience is improved, the picture is conveniently used to obtain required information, text information can be accurately extracted from the picture and matched with the text knowledge base, the matching accuracy and integrity are improved, the user can directly use the picture to perform knowledge retrieval, and the user experience is improved. And the system automatically identifies characters in the picture and matches the characters with the knowledge base, so that intelligent processing and utilization of image information are realized, and a more intelligent, convenient and efficient information retrieval and question-answering system is realized.

The invention provides a character recognition method and device, computer equipment and a storage medium. According to the scheme, the slice scanning image is firstly acquired, then the label area is identified according to the slice scanning image, and the character information is close to the label, so that the determined target area is larger than the area of the label so as to cover the character information. And then the slice scanning image is intercepted according to the target area, and a target image only containing key information is obtained. And character recognition is carried out on the target image to obtain a character recognition result. And finally, the character recognition results are screened by using a rule base, and a target recognition result is screened out. According to the scheme, through tag region identification and target region interception, the character identification range is narrowed, the identification efficiency is improved, and the waste of computing resources caused by processing a large amount of irrelevant information is reduced. The rule base can better adapt to complex slice images and constantly changing requirements, information needed by a user is accurately screened out, and the character recognition efficiency is greatly improved.

The invention relates to a knowledge-intensive visual question and answer automatic data generation method and device, and the method comprises the steps: constructing an original visual data set containing the professional knowledge of a target domain according to a static image, a video stream and multimedia content; extracting a representative frame sequence, converting the audio information into text information, and extracting character information in the static image to construct a structured visual instance database; according to the prompt text meeting the preset professional depth condition, establishing a three-level prompt system containing domain knowledge, an evaluation standard and a generation specification; generating a corresponding visual question and answer pair data set according to the dynamic cooperation of the main agent and the domain expert agent; generating a multi-agent quality evaluation system according to the quality evaluation result; and designing a difficulty grading mechanism according to the negative example sample. According to the method, the professionality, the accuracy and the diversity of the visual question and answer data are remarkably improved, and reliable data support is provided for training and evaluation of a multi-modal large model.

The invention discloses an image recognition method for electric energy meter recognition. The method comprises the steps that an image with a metering electric energy meter display screen is acquired; preprocessing the image for correcting light reflection distortion of the image; performing text detection, and identifying and positioning a text region of the preprocessed image; character information identification of the concerned text area is carried out; performing layout analysis, and identifying digital information corresponding to different elements in the layout; performing NER detection filtering to obtain entity data in the image; and displaying the entity data. According to the method, the image is preprocessed for image reflection distortion correction, so that the image recognition result is more accurate. According to the method, the Fast R-CNN method is adopted for layout analysis, and different elements such as characters, charts and pictures in the layout can be quickly and accurately positioned and classified.

Systems and methods are provided for generating and updating a dependency graph that is used in combination with textual information about incidents to improve incident-linking suggestions. Systems and methods are also provided for generating, training, and using a machine learning model configured to perform incident linking using both graph data and text data. Beneficially, these systems and methods align the graph data and text data in order to more efficiently and accurately leverage information from the multi-modal data.

Disclosed is a new document processing solution that combines the powers of machine learning and deep learning and leverages the knowledge of a knowledge base. Textual information in an input image of a document can be converted to semantic information utilizing the knowledge base. A semantic image can then be generated utilizing the semantic information and geometries of the textual information. The semantic information can be coded by semantic type determined utilizing the knowledge base and positioned in the semantic image utilizing the geometries of the textual information. A region-based convolutional neural network (R-CNN) can be trained to extract regions from the semantic image utilizing the coded semantic information and the geometries. The regions can be mapped to the textual information for classification / data extraction. With semantic images, the number of samples and time needed to train the R-CNN for document processing can be significantly reduced.

The invention provides a Web API recommendation method based on a multi-task deep neural network. The Web API recommendation method comprises the following steps: S1, cleaning a Mashup description document and a Web API description document in a data set; s2, mapping character information in the cleaned data set into digital codes; s3, generating sample data based on digital coding; s4, constructing a Web API recommendation model based on the multi-task deep neural network; s5, inputting the sample data into the Web API recommendation model for pre-training to obtain a pre-trained Web API recommendation model; s6, performing fine adjustment on the pre-trained Web API recommendation model by using a Mashup and API matching score prediction task; s7, obtaining a matching score list of the Mashup demand and each Web API (Application Program Interface); and S8, sorting the matching score lists of the Mashup requirements and the Web APIs in a descending order according to the matching scores, and selecting the first K Web APIs to form a final recommendation list. According to the method, the problem that no-interaction or low-interaction Web APIs in historical data are difficult to recommend accurately is solved, and indexes such as the recall rate and the accuracy rate of recommendation are improved.

A computer-implemented method for improving language models includes tokenization processing to concatenate an obtained image and texts with designated tags. An image encoder is trained to encode image information of the image. A two token representation is generated by a multi-modal generative agent using the encoded image information and the obtained texts. A text encoder and the image encoder are trained to encode the obtained image and texts. A text decoder and an image decoder are combined for the multi-modal reasoning segmentation by training the text decoder and the image decoder to generate a segmented image and segmented textual information using the two token representation in combination with an output of the text encoder and an output of the image encoder. The method has applications including, but not limited to, use cases in medical Al / healthcare, for example, for optimizing medical diagnosis or treatment or supporting decision making.

The invention discloses a seal area character recognition method and device and a nonvolatile storage medium. The method comprises the following steps: determining a candidate seal image area in an image according to color information of pixel points in the image; point-by-point sliding convolution processing is carried out on the candidate seal image area through a multi-scale annular convolution kernel group, so that a target seal image area is determined in the candidate seal image area, and the multi-scale annular convolution kernel group comprises a plurality of convolution kernels which are of concentric ring structures and have different radius lengths; mapping the target seal image area into a rectangular expanded image, and identifying and extracting a character image to be identified in the rectangular expanded image; and performing identification processing on the character image to be identified to obtain a seal text corresponding to the target seal image area. The technical problem that the text image processing efficiency is low due to the fact that the text information of the seal area cannot be effectively recognized in the related technology is solved.

The invention relates to the technical field of image processing, solves the problem that in the prior art, video content cannot be accurately identified and analyzed while pictures and materials are accurately associated, and provides a video content analysis method, device and equipment based on multi-modal data processing. The method comprises the following steps: acquiring a to-be-analyzed original video material; performing transcoding processing on the original video material, and determining standard video data in a preset coding format; decomposing the standard video data into a plurality of video clips by using a shot segmentation technology; embedding the blind watermark information into each frame of image in each video clip by using a preset feature processing algorithm; performing compression processing on each video clip, and determining compressed video data; and identifying and analyzing the compressed video data, and outputting text information and target video material information related to video content. According to the method and the device, the accurate association of the video and the material is realized, and the accuracy of identification and analysis of the video content is improved.

Systems, apparatus, articles of manufacture, and methods are disclosed that recognize products using visual and textual information. An example apparatus includes classification circuitry to classify object facings detected in a shelf image to determine object candidates corresponding to a first object, ones of the object candidates to include a respective image-based metric; cluster circuitry to assign words extracted from the shelf image to the first object; rank adjuster circuitry to generate text-based metrics for the object candidates by comparing respective object characteristics of the object candidates to the words, ones of the text-based metrics corresponding to respective ones of the object candidates; and prediction circuitry to determine final metrics for the object candidates by selecting, for the ones of the object candidates, (a) a respective image-based metric or (b) a combination of the respective image-based metric and a respective text-based metric; and rank the object candidates using the final metrics.

The invention provides a large vision-language model reasoning method based on multi-modal tree search, which is characterized in that multi-step multi-modal auxiliary information is called and generated through an auxiliary tool, a candidate reasoning path of a target task is simulated and evaluated in combination with a prediction expansion tree search mechanism, and an optimal path is selected through self-voting, so that the reasoning efficiency is improved. Therefore, through a vision-text interleaving reasoning framework and an extension strategy during testing, vision and text information are fully utilized, the reasoning ability of a large vision-language model in a complex multi-step reasoning task is remarkably improved, a more accurate reasoning result can be obtained, model fine adjustment is not needed, and the reasoning efficiency is improved. Therefore, the reasoning result can be obtained more quickly, and the scheme of the invention has good practical application value.

A management node local to a customer site of a private communications network stores a model. The model is a compact version of a visual language model remote from the customer site. A first screen shot of a dashboard of telemetry data measured from the private communications network is accessed. A prompt is formulated comprising the first screen shot and information to adapt the model to the private communications network via few shot learning. The prompt is submitted to the model. An output is received from the model comprising textual information about anomalies or trends depicted in the first screen shot. The output is checked against data from a statistical model of the telemetry data, the statistical model being independent of the model. In response to the check being successful, an action is triggered to manage the private communications network according to the output.

The invention discloses a voice interaction method and device, equipment, a storage medium and a product, and relates to the technical field of computers, the voice interaction method comprises the following steps: when a user voice stream is received, generating text information of the user voice stream in a streaming manner, and sending the text information to a preset large model; and receiving reply information returned by the large model, playing the reply information, and when the reply information is played, if a new round of user voice stream is received, returning to the step of generating the semantic information of the user voice stream in a streaming manner. According to the method, the character information of the user voice stream is replied by adopting the large model, when the output of the reply information is interrupted by the new round of user voice stream, the reply information is regenerated by returning to the step of generating the semantic information of the user voice stream in a stream mode, so that the voice interruption of the received new round of user voice stream is realized; therefore, the coherence of voice interaction is improved, and the fluency of voice interaction is finally improved.

The invention provides a large model knowledge base construction and retrieval method for a complex long text. The method comprises the steps that multi-granularity partitioning is conducted, text data are received and preprocessed, the text data are split in a multi-granularity hierarchy mode on the basis of a variable-granularity partitioning strategy, and each granularity hierarchy corresponds to a plurality of text blocks with the consistent granularity; the method comprises the following steps: constructing a knowledge base, obtaining text metadata of each text block, vectorizing the text blocks and the text metadata, generating a text block vector and a text metadata vector, and calling a text block vector with more text information according to the text metadata vector with less text information. According to the method, the text blocks can be divided in a multi-granularity hierarchy mode, context logic information in the text data is effectively reserved, and the retrieval accuracy of a knowledge base is improved.

Methods and systems for interacting with users via a chatbot. A natural language query is received and processed by submitting a search query to a search engine. The search engine identifies relevant information including textual information and images for formulating a response. The identified information and query are submitted to a Large Language Model which generates a response displayed via the chatbot. The response may include textual information and relevant images. The system can extract text from images of documents and convert textual information into numerical vector representations for processing. Selectable options based on clustered relevant information can be provided to users for query refinement when appropriate. The chatbot interface enables natural language interactions while leveraging search capabilities and Artificial Intelligence to provide informative and helpful responses with both text and visual elements.

The invention relates to a video data generation method and device for an automatic driving scene, and the method comprises the steps: inputting the text information used for describing the driving scene of a vehicle into a large language model LLM, and outputting the target information used for constructing the automatic driving scene; respectively inputting the traffic road data into a static driving scene reconstruction dedicated neural rendering network and a dynamic traffic participant reconstruction dedicated neural rendering network to obtain output results of the static driving scene reconstruction dedicated neural rendering network and the dynamic traffic participant reconstruction dedicated neural rendering network; and generating video data of an automatic driving scene associated with the vehicle based on the target information and the output result. According to the method and the device, the video data of the automatic driving scene can be generated in batches, so that the technical problem that a driving scene generation mode in the prior art cannot meet the requirement of obtaining large-batch data on a large scale required by an automatic driving perception algorithm is solved.

A system is disclosed for identifying industrial ergonomics risk root-causes and providing risk control actions. The system includes a processor configured to execute machine readable instructions to receive, from a computing device of a user, video of an individual performing a task and receive, from the computing device, textual information related to the task. The processor is also configured to process the textual information and the video of the individual performing the task through a deep learning framework to determine ergonomic risk root-causes. The processor is further configured to determine ergonomic risk control recommendations to mitigate the determined ergonomic risk root-causes. The processor transmits, to the computing device, information indicative of the ergonomic risk control recommendations and the ergonomic risk root-causes.

The invention discloses an electric power knowledge picture content extraction method and system, and belongs to the technical field of electric power knowledge extraction, and the method comprises the steps: obtaining an electric power knowledge picture; recognizing the electric power knowledge picture into text information by adopting a pre-trained electric power field content extraction model; and performing quality judgment on the text information through a data quality discriminator, if the quality judgment is excellent, outputting corresponding formatted information, and otherwise, marking the text information as the corresponding formatted information through an electric power expert. According to the method, the electric power field content extraction model can quickly identify the character information, the quality of the character information is judged through the data quality discriminator, the accuracy is higher, and the processing efficiency is higher.

The invention provides a multilevel multi-model collaborative compliance detection system and method based on a large model, and the system comprises a content analysis and verification module which is used for extracting text information in to-be-detected content and correcting errors in the extracted content; the compliance detection module is used for detecting compliance; the consistency detection module is used for detecting the consistency; and the detection result rechecking module is used for rechecking and checking the detection result based on the compliance criterion. According to the technical scheme, compliance detection can be carried out on various types of materials, and compliance detection capability is flexibly constructed for compliance detection of specific professions and scene vertical fields; the information consistency of specific professional and scene contents can be flexibly detected.

The invention belongs to the field of bill recognition, and provides a bill recognition method, system and equipment based on a large model, and the method comprises the steps: obtaining bill image data containing text information, and carrying out the preprocessing of the obtained bill image data; evaluating image quality, language distribution and layout complexity of the preprocessed image, and determining a processing path of document image data by integrating evaluation results; according to the determined processing path, calling a corresponding optical character recognition model to extract text information in the document image data; performing deep semantic analysis on the extracted text data by using a pre-trained deep learning large model, and extracting key feature information; and performing rule compliance analysis on the text information or the extracted key feature information, and generating an identification analysis report according to an analysis result. According to the invention, the processing speed and accuracy are improved, the adaptability to various document formats is enhanced, and the limitation in the prior art is effectively solved.

The invention discloses a remote cooperation method and device based on a polymorphic large model, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a visual field video stream of a production field and voice data inquired by field personnel, wherein the visual field video stream is acquired by AR glasses worn by the field personnel in real time; analyzing the view video stream to obtain first record content; extracting character information in the voice data, and matching the character information with a pixel position of a target object to obtain second record content; and inputting the first record content and the second record content into a pre-trained polymorphic large model, generating multimedia summary response content, and feeding back the multimedia summary response content to a user terminal of the field personnel, and enabling the field personnel to perform field inspection according to the multimedia summary response content displayed by the user terminal. The remote cooperation efficiency can be improved.