887 results about "Textual information" patented technology

The invention relates to the technical field of intelligent vending, solves the problem that in the prior art, commodity identification cannot be accurately carried out in a multi-target scene, and provides a multi-target commodity identification method, device and system based on multi-modal data processing. The method comprises the following steps: acquiring multiple frames of real-time images in a commodity transaction scene; performing preprocessing and label information extraction on the real-time image, and determining character information corresponding to the target image and the commodity label; performing instance segmentation on the target image, and determining commodity position information; performing feature extraction on the target image, and determining commodity image feature information; according to pre-collected multi-source privatized data in an intelligent vending scene, performing fine adjustment and optimization processing on the open-source multi-modal visual language model to obtain a multi-modal large model; and inputting the commodity image feature information and the text information into the multi-modal large model for information fusion, and determining a commodity target identification result. According to the invention, commodity identification can be accurately carried out in a multi-target scene.

Automatic recognition of non-verbal messages in speech, and in particular to detection or analysis of prosodic multilayered analysis of intonation units such as prosodic unit prototypes and their multi-labeled variations, may form a hierarchical classification for the analysis of non¬ verbal information or cues in speech. A speech captured by a microphone is fed to a weakly- supervised deep learning acoustic model for speech recognition and transcription, that may be based on encoder-decoder Transformer architecture, such as Whisper by OpenAI. The model is trained to output multiple words form the text in the captured speech, to identify Intonation Units (IUs) that include one or more words, and associate non-verbal labels to each of the IUs. The labels may indicate a prototype, a discourse function (such as a conversation action), an emotion, an emphasis, or an attitude, as well as a genre of a part of, or whole of, the entire captured speech.

The invention discloses a virtual reality scene three-dimensional reconstruction method and system based on multi-source data fusion, and relates to the technical field of underground commercial and traffic integrated space scene three-dimensional reconstruction. The system constructs a multi-dimensional data set through synchronous acquisition of point cloud, images, poses, positioning and semantic text information; a three-dimensional convolutional neural network is adopted to establish an AI reconstruction fusion model, and a three-dimensional model is generated; the system calculates a structural integrity evaluation coefficient JGPG, a fusion consistency evaluation coefficient RHPG and a VR interaction adaptability evaluation coefficient VRJH, compares the coefficients with corresponding thresholds, triggers an optimization strategy, and finally completes model adaptability optimization and packaging output, thereby improving the reduction degree and interaction performance of a virtual reality scene.

The invention relates to the technical field of multi-modal large language models, in particular to a government affair service content navigation method and system based on a large language model, and the method comprises the following steps: receiving multi-modal information input by a user through texts, voices or pictures; the voice is converted into a text, and character information in the picture is analyzed by using an OCR (Optical Character Recognition) technology; fusing multi-modal data, and inputting the fused multi-modal data into a large language model for semantic understanding and context association analysis; matching items are retrieved in combination with a local government affair knowledge base, and an initial recommendation list is generated; the recommendation result is displayed through the intelligent assistant, and interaction optimization options are provided; the method has the beneficial effects that the current government affair service content navigation mode is optimized through the multi-modal capability, the retrieval enhancement capability and the content generation capability of the large language model, so that the navigation is more modal and more intelligent, and meanwhile, the privacy and authority of data reply are ensured.

The invention discloses a recommendation method for enhancing semantics and interest perception by using a large language model. The method comprises the following steps: firstly, performing semantic modeling on unstructured text information such as user comments, article description and the like by utilizing the powerful capability of a large language model in semantic comprehension and user preference modeling aspects, so as to improve the deep perception capability of a recommendation system on user interests and article semantic attributes; then, through a semantic feature alignment and discretization strategy, the problem that continuous semantic representation generated by a large language model is incompatible with features of a traditional recommendation system in the aspect of an expression structure is solved; finally, unified modeling of semantic information and traditional recommendation signals is achieved through a recommendation integration mechanism, and recommendation performance and model interpretability are improved.

The invention relates to the technical field of video detection, in particular to a monitoring video detection and identification method and system based on deep learning, and the method comprises the steps: obtaining a target monitoring video of a monitoring region based on monitoring camera equipment, obtaining a video frame sequence of the target monitoring video, and carrying out the target screening of the video frame sequence, so as to obtain a video frame set; and performing target detection on each video frame in the video frame set based on a trained target detection model to obtain a target detection result set of each video frame in the video frame set. According to the method, through combination of keyword extraction and a large language model, complex monitoring data can be converted into structured event description, the textualized information not only facilitates management personnel to rapidly know the field condition, but also can provide reference for subsequent event processing, and the event description generated in real time is combined, so that the management personnel can conveniently and rapidly know the field condition. And a manager can make a coping strategy more efficiently.

The invention provides a character recognition method and device, computer equipment and a storage medium. According to the scheme, the slice scanning image is firstly acquired, then the label area is identified according to the slice scanning image, and the character information is close to the label, so that the determined target area is larger than the area of the label so as to cover the character information. And then the slice scanning image is intercepted according to the target area, and a target image only containing key information is obtained. And character recognition is carried out on the target image to obtain a character recognition result. And finally, the character recognition results are screened by using a rule base, and a target recognition result is screened out. According to the scheme, through tag region identification and target region interception, the character identification range is narrowed, the identification efficiency is improved, and the waste of computing resources caused by processing a large amount of irrelevant information is reduced. The rule base can better adapt to complex slice images and constantly changing requirements, information needed by a user is accurately screened out, and the character recognition efficiency is greatly improved.

The invention relates to a knowledge-intensive visual question and answer automatic data generation method and device, and the method comprises the steps: constructing an original visual data set containing the professional knowledge of a target domain according to a static image, a video stream and multimedia content; extracting a representative frame sequence, converting the audio information into text information, and extracting character information in the static image to construct a structured visual instance database; according to the prompt text meeting the preset professional depth condition, establishing a three-level prompt system containing domain knowledge, an evaluation standard and a generation specification; generating a corresponding visual question and answer pair data set according to the dynamic cooperation of the main agent and the domain expert agent; generating a multi-agent quality evaluation system according to the quality evaluation result; and designing a difficulty grading mechanism according to the negative example sample. According to the method, the professionality, the accuracy and the diversity of the visual question and answer data are remarkably improved, and reliable data support is provided for training and evaluation of a multi-modal large model.

Disclosed is a new document processing solution that combines the powers of machine learning and deep learning and leverages the knowledge of a knowledge base. Textual information in an input image of a document can be converted to semantic information utilizing the knowledge base. A semantic image can then be generated utilizing the semantic information and geometries of the textual information. The semantic information can be coded by semantic type determined utilizing the knowledge base and positioned in the semantic image utilizing the geometries of the textual information. A region-based convolutional neural network (R-CNN) can be trained to extract regions from the semantic image utilizing the coded semantic information and the geometries. The regions can be mapped to the textual information for classification / data extraction. With semantic images, the number of samples and time needed to train the R-CNN for document processing can be significantly reduced.

A computer-implemented method for improving language models includes tokenization processing to concatenate an obtained image and texts with designated tags. An image encoder is trained to encode image information of the image. A two token representation is generated by a multi-modal generative agent using the encoded image information and the obtained texts. A text encoder and the image encoder are trained to encode the obtained image and texts. A text decoder and an image decoder are combined for the multi-modal reasoning segmentation by training the text decoder and the image decoder to generate a segmented image and segmented textual information using the two token representation in combination with an output of the text encoder and an output of the image encoder. The method has applications including, but not limited to, use cases in medical Al / healthcare, for example, for optimizing medical diagnosis or treatment or supporting decision making.

The invention discloses a seal area character recognition method and device and a nonvolatile storage medium. The method comprises the following steps: determining a candidate seal image area in an image according to color information of pixel points in the image; point-by-point sliding convolution processing is carried out on the candidate seal image area through a multi-scale annular convolution kernel group, so that a target seal image area is determined in the candidate seal image area, and the multi-scale annular convolution kernel group comprises a plurality of convolution kernels which are of concentric ring structures and have different radius lengths; mapping the target seal image area into a rectangular expanded image, and identifying and extracting a character image to be identified in the rectangular expanded image; and performing identification processing on the character image to be identified to obtain a seal text corresponding to the target seal image area. The technical problem that the text image processing efficiency is low due to the fact that the text information of the seal area cannot be effectively recognized in the related technology is solved.

The invention provides a large vision-language model reasoning method based on multi-modal tree search, which is characterized in that multi-step multi-modal auxiliary information is called and generated through an auxiliary tool, a candidate reasoning path of a target task is simulated and evaluated in combination with a prediction expansion tree search mechanism, and an optimal path is selected through self-voting, so that the reasoning efficiency is improved. Therefore, through a vision-text interleaving reasoning framework and an extension strategy during testing, vision and text information are fully utilized, the reasoning ability of a large vision-language model in a complex multi-step reasoning task is remarkably improved, a more accurate reasoning result can be obtained, model fine adjustment is not needed, and the reasoning efficiency is improved. Therefore, the reasoning result can be obtained more quickly, and the scheme of the invention has good practical application value.

A management node local to a customer site of a private communications network stores a model. The model is a compact version of a visual language model remote from the customer site. A first screen shot of a dashboard of telemetry data measured from the private communications network is accessed. A prompt is formulated comprising the first screen shot and information to adapt the model to the private communications network via few shot learning. The prompt is submitted to the model. An output is received from the model comprising textual information about anomalies or trends depicted in the first screen shot. The output is checked against data from a statistical model of the telemetry data, the statistical model being independent of the model. In response to the check being successful, an action is triggered to manage the private communications network according to the output.

The invention provides a large model knowledge base construction and retrieval method for a complex long text. The method comprises the steps that multi-granularity partitioning is conducted, text data are received and preprocessed, the text data are split in a multi-granularity hierarchy mode on the basis of a variable-granularity partitioning strategy, and each granularity hierarchy corresponds to a plurality of text blocks with the consistent granularity; the method comprises the following steps: constructing a knowledge base, obtaining text metadata of each text block, vectorizing the text blocks and the text metadata, generating a text block vector and a text metadata vector, and calling a text block vector with more text information according to the text metadata vector with less text information. According to the method, the text blocks can be divided in a multi-granularity hierarchy mode, context logic information in the text data is effectively reserved, and the retrieval accuracy of a knowledge base is improved.

Methods and systems for interacting with users via a chatbot. A natural language query is received and processed by submitting a search query to a search engine. The search engine identifies relevant information including textual information and images for formulating a response. The identified information and query are submitted to a Large Language Model which generates a response displayed via the chatbot. The response may include textual information and relevant images. The system can extract text from images of documents and convert textual information into numerical vector representations for processing. Selectable options based on clustered relevant information can be provided to users for query refinement when appropriate. The chatbot interface enables natural language interactions while leveraging search capabilities and Artificial Intelligence to provide informative and helpful responses with both text and visual elements.

The invention provides a multilevel multi-model collaborative compliance detection system and method based on a large model, and the system comprises a content analysis and verification module which is used for extracting text information in to-be-detected content and correcting errors in the extracted content; the compliance detection module is used for detecting compliance; the consistency detection module is used for detecting the consistency; and the detection result rechecking module is used for rechecking and checking the detection result based on the compliance criterion. According to the technical scheme, compliance detection can be carried out on various types of materials, and compliance detection capability is flexibly constructed for compliance detection of specific professions and scene vertical fields; the information consistency of specific professional and scene contents can be flexibly detected.

The invention belongs to the field of bill recognition, and provides a bill recognition method, system and equipment based on a large model, and the method comprises the steps: obtaining bill image data containing text information, and carrying out the preprocessing of the obtained bill image data; evaluating image quality, language distribution and layout complexity of the preprocessed image, and determining a processing path of document image data by integrating evaluation results; according to the determined processing path, calling a corresponding optical character recognition model to extract text information in the document image data; performing deep semantic analysis on the extracted text data by using a pre-trained deep learning large model, and extracting key feature information; and performing rule compliance analysis on the text information or the extracted key feature information, and generating an identification analysis report according to an analysis result. According to the invention, the processing speed and accuracy are improved, the adaptability to various document formats is enhanced, and the limitation in the prior art is effectively solved.

The invention discloses a remote cooperation method and device based on a polymorphic large model, electronic equipment and a storage medium, and relates to the technical field of computers. The method comprises the following steps: acquiring a visual field video stream of a production field and voice data inquired by field personnel, wherein the visual field video stream is acquired by AR glasses worn by the field personnel in real time; analyzing the view video stream to obtain first record content; extracting character information in the voice data, and matching the character information with a pixel position of a target object to obtain second record content; and inputting the first record content and the second record content into a pre-trained polymorphic large model, generating multimedia summary response content, and feeding back the multimedia summary response content to a user terminal of the field personnel, and enabling the field personnel to perform field inspection according to the multimedia summary response content displayed by the user terminal. The remote cooperation efficiency can be improved.

The invention provides a video cover generation method. The method comprises the following steps: determining a foreground entity of a video cover based on an input video; selecting a target video frame comprising a foreground entity from the video and determining features of the foreground entity in the target video frame; text information of the video cover is generated, and features of the text information are determined; determining a background image of the video cover; and generating the video cover based on the target video frame, the features of the foreground entity, the character information, the features of the character information and the background image of the video cover. The invention further provides a video cover generation device, electronic equipment, a computer readable storage medium and a computer program product.

A method for processing a piece of textual information. The piece of textual information is parsed into a set of plaintext input tokens. Each of the plaintext input tokens is individually transformed using a first binary data transformation, to achieve a set of binary input tokens. Each of the set of binary input tokens is transformed individually or collectively, using an embedding data transformation, into one or several vectorized input tokens. The one or several vectorized input tokens is / are fed to a first neural network. A response is received from the first neural network in the form of one or several vectorized output tokens.

The invention provides a CAD file summary information automatic extraction method and device, a medium and a product, and relates to the technical field of computer picture word process.The method comprises the steps that a target CAD file is analyzed through a CAD file analysis development tool, and first word information and geometric attribute information corresponding to the first word information are obtained; retrieving the first character information according to the retrieval keyword to obtain second character information; according to the corresponding geometric attribute information in the second text information, carrying out positioning adjustment display through a CAD file analysis development tool to obtain a picture file; calling a first large language model to perform character recognition and arrangement processing on the picture to form a text list; and calling the second large language model to obtain summary information of the target CAD file according to a preset prompt template and the text list. According to the scheme, when a large batch of CAD files are processed, the processing efficiency is remarkably improved.

The invention relates to the technical field of computer data processing, and discloses a financial reimbursement data processing method, device and system.The financial reimbursement data processing method comprises the following steps that multi-modal data of a reimbursement voucher is obtained, and the multi-modal data is stored in a block chain; the multi-modal data comprises pictures, voices and videos of reimbursement vouchers; recognizing characters on the picture based on a convolutional neural network, and extracting and analyzing character information of key frames of the voice and the video by using a natural language processing model; and performing feature extraction on the character information to obtain character feature information, judging whether the character feature information falls into a reimbursement directory, and performing local model fitting according to the character feature information. According to the method, the dynamic verification rule is generated based on the historical reimbursement data, the bill anomaly probability is calculated in real time through the anomaly coefficient formula, and the anomaly detection accuracy is improved by combining the error coefficient and keyword matching degree and the historical record relevance index.

Techniques are provided for producing an answer to a question regarding a domain. A natural-language textual sequence representing the question is received. From a knowledge graph associated with the domain, first and second textual passages are received using rankings corresponding to the natural-language textual sequence, a first textual summary is received summarizing textual information in a first vicinity of the first textual passage, and a second textual summary summarizing textual information in a vicinity of the second textual passage is received. An answer to the question is obtained using a language model by encoding a first intermediate output based on the natural-language textual sequence, the first textual passage, and the first textual summary, encoding a second intermediate output based on the natural language textual sequence, the second textual passage, and the second textual summary, and decoding a concatenation of the first and second intermediate outputs. An output is provided.

The invention discloses a sign language recognition glasses device, system and method, and belongs to the technical field of intelligent equipment, and the sign language recognition glasses device comprises a glasses frame which is used for supporting all components of the device; the camera is arranged on the front side of the glasses frame and is used for capturing hand actions in real time to generate image information; the data processing module is embedded in the glasses frame, electrically connected to the camera and used for processing the image information and executing sign language recognition; the main control unit is arranged on an ear rod of the glasses frame, is electrically connected to the data processing module and is used for coordinating the operation of each component; the display module is arranged at the positions of the lenses of the glasses frame, electrically connected to the main control unit and used for displaying character information obtained after sign language recognition, communication between hearing-impaired people and common people is not limited by specific places any more, and instantaneity and convenience of communication are greatly improved.

The invention provides an intelligent psychological counseling system based on a multi-mode large model and a cognitive behavior therapy, and the system comprises the steps: recognizing voice from a user side through the multi-mode large model, and obtaining text information and emotional state information; analyzing the text information by using a large language model, carrying out conversation with the user side, and extracting psychological state information of the user side from the conversation; a cognitive behavior treatment strategy matched with the user side is determined according to the emotional state information, so that a voice response to the user side is generated according to the matched cognitive behavior treatment strategy by using a multi-mode large model; by combining a multi-modal large model and a large language model, accurate and efficient natural language understanding of a user side is realized, highly anthropomorphic voice response is provided, dependence on professional psychological consultants is reduced, limitation of psychological consultation services on time and regions can be broken through, large-scale consultation service requirements can be met in an all-weather and low-cost manner, and the psychological consultation service experience is improved. The man-machine interaction experience in the consultation process is improved, and high-quality and high-practicability psychological consultation and treatment services are provided.

The invention relates to the technical field of query systems, in particular to a semantic recognition system for guiding treatment information query based on LUTS symptoms. The information receiving module is used for receiving voice information, the preprocessing module is used for converting the voice information into text information based on a preset time interval, and the extraction module is used for obtaining a plurality of keywords. The matching module is used for carrying out key information retrieval on the basis of the key words, judging whether retrieval for the key information meets standards or not on the basis of retrieval results of the key information and generating corresponding processing instructions, and the control module is used for re-determining operation parameters of corresponding components on the basis of the processing instructions. The output module is used for outputting a retrieval result; and the database is used for storing a plurality of pieces of preset key information comprising a plurality of preset keywords. According to the method and the system, accurate query of guidance information and treatment information of the LUTS symptom by the semantic recognition system is effectively realized, and the operation efficiency of the semantic recognition system for information query of the LUTS symptom is effectively improved.

The invention discloses a scene graph construction method, a target retrieval method and a related device. The scene graph construction method comprises the following steps: constructing a topological node set and a topological edge set; for each topological node in the topological node set, taking a set of visual information, text information and spatial position information of the topological node as multi-modal data of the topological node; for each topological node in the topological node set, determining multi-modal data of an object node corresponding to the topological node and a connection edge set corresponding to the topological node based on the visual information of the topological node; and constructing a target scene graph based on the multi-modal data of each topological node, the multi-modal data of each object node, the topological edge set and the connection edge set corresponding to each topological node. The target scene graph has semantic consistency and spatial continuity, and the expression ability and cross-scene generalization performance of the scene graph in a complex environment are remarkably improved.

The invention provides a scheme detection method and device for a complex image-text mixed file, and the method comprises the steps: extracting standard review items and standard review contents in a standard manual, and carrying out the structural processing, and obtaining a standard review file; converting a to-be-detected file into a to-be-detected image, extracting a to-be-examined multi-scale candidate region image from the to-be-detected image by using an initial recognition model and a matching model obtained by OCR and pre-training, and extracting character information of the multi-scale candidate image to obtain character information of the candidate region; inputting the multi-scale candidate region image and the standard review file into a lightweight hybrid twin network obtained by pre-training for image feature matching, and outputting a matched feature image; inputting the matched feature image, the standard review file and the candidate area text information into a multi-modal large model obtained by pre-training for compliance analysis, and outputting a detection result; according to the method, the automation and intelligence degree of review of the complex image-text mixed file can be remarkably improved.

The invention relates to the technical field of image processing, in particular to an image detection method and device and electronic equipment. The method comprises the steps of obtaining image information of a to-be-detected object; wherein the image information comprises one or more of text features, film features and portrait features. The text feature comprises text information. The characteristics of the covering film comprise at least one of a fitting characteristic and a flattening characteristic of the covering film and the to-be-detected object. The portrait features comprise at least one of face features and body features. And detecting whether the image information is abnormal according to a preset detection rule. And if abnormity occurs, generating alarm information. Wherein the exceptions comprise one or more of text feature detection exceptions, film feature detection exceptions and portrait feature detection exceptions. According to the scheme, the detection efficiency and accuracy are improved, and the manual quality inspection cost is saved.

The invention provides a maintenance guidance method and device for a household electrical appliance. The method comprises the following steps: constructing a unified vector database in advance based on text data, image data and audio data related to a household electrical appliance fault; acquiring field condition information of the household electrical appliance, wherein the field condition information comprises video information, audio information and / or text information; respectively carrying out image matching, audio matching and / or text matching on the field condition information based on the unified vector database, and outputting knowledge fragments as retrieval results; and inputting a retrieval result into the large language model, and outputting a maintenance guidance scheme of the household electrical appliance. According to the method, a retrieval and reasoning framework based on multi-modal feature direct fusion can be constructed, information dimension reduction loss is avoided by directly analyzing image, audio and text features of faults, and the comprehensive accuracy of fault diagnosis is improved to 95% or above; and through a dynamic knowledge retrieval and enhanced generation technology, the risk of'knowledge illusion 'possibly occurring in the professional field of the general large language model is effectively reduced.