131 results about "Multimodal analysis" patented technology

The invention relates to an intelligent advertisement material automatic optimization method, system and device and a readable storage medium, and the method comprises the steps: receiving the real-time putting effect number of a target advertisement video; analyzing the target advertisement video through a pre-trained multi-modal analysis model, and generating a commodity display sequence, copywriting content and background music features in the advertisement video; associating the delivery effect data to the analyzed commodity display sequence, copywriting content and background music features, and inputting the data into a dynamic optimization decision engine to generate a three-element optimization instruction; and executing an automatic modification operation on the target advertisement video according to the three-element optimization instruction, generating an optimized advertisement video and putting the optimized advertisement video so as to realize the purposes of real-time analysis, accurate optimization and efficient putting of the advertisement video content and improvement of the user attraction and conversion effect of the advertisement.

The invention relates to a video frame data segmentation right confirmation and dynamic transaction method and system based on a block chain. The method comprises the following steps: segmenting video stream data according to a preset frame rate to generate independent frame images, and carrying out anti-collision hash calculation on the independent frame images to generate a unique identifier; generating a comprehensive value score through a multi-modal analysis model in combination with the independent frame image and the corresponding script text data; generating NFT assets anchored to the block chain according to the comprehensive value score and the unique identifier; and dynamically adjusting the income distribution proportion of the NFT assets through the smart contract based on the user film watching data. According to the method, uniqueness right confirmation of frame data is realized through precise segmentation and anti-collision hash coding, value evaluation is performed by using a multi-modal analysis model, and capitalization and on-chain right confirmation are completed by means of a block chain technology. In addition, the income distribution proportion is dynamically adjusted in combination with the smart contract, efficient transaction and right distribution of the video frame data are realized, and the commercial value and transaction efficiency of the video data are improved.

The invention is suitable for the technical field of intelligent medical treatment, and provides an intelligent decision-making system for clinical diagnosis decision analysis, which comprises a data acquisition module, a knowledge graph module, a multi-modal analysis engine, a clinical evaluation module, a clinical treatment scheme decision-making module and a feedback optimization module, the data acquisition module is used for integrating multi-source heterogeneous clinical data; the knowledge graph module is used for dynamically storing a medical entity relationship; the multi-modal analysis engine extracts features through deep learning; the clinical treatment scheme decision module is used for outputting diagnosis suggestions and providing explanations; the feedback optimization module is used for receiving a doctor correction result, automatically iterating a training model and forming closed-loop learning; according to the system, real-time association medical new discovery is supported through a dynamic knowledge graph, and hospital information system data are not effectively integrated; through cooperative work of the rule engine and the AI model, clinical medical quality and efficiency are improved.

The invention relates to the technical field of video generation, in particular to an intelligent commodity video generation method based on multi-modal analysis and a dynamic narrative architecture, which comprises the following steps: inputting original commodity data and a user behavior log, and outputting a dynamic selling point weight vector through a selling point value evaluation engine; s2, inputting the dynamic selling point weight vector output in S1 into a narrative flow generator, and executing the following steps: intercepting first K core selling points according to the weight vector to form a narrative trunk chain; based on the historical preference data of the user, inserting an emotion enhancement node to generate a branch enhancement narrative flow; generating a multi-version narrative flow instruction set in combination with the real-time network bandwidth data; retrieving a preset video clip library according to the instruction set to generate an initial video sequence; detecting semantic fault regions between adjacent segments to generate compensation animation parameters; and injecting compensation animation parameters and outputting a continuous narrative video stream. According to the invention, the jamming feeling and the frame skipping phenomenon caused by the change of a narrative structure or different sources of video clips are reduced, and the overall watching experience of a user is improved.

The invention relates to an intelligent video editing method fusing human face features and human voice features, which comprises the steps of input preprocessing, multi-modal analysis, fusion scoring and automatic editing, and adopts a multi-modal mode for analysis, so that the recall rate of key segments is improved, and the efficiency of editing is improved. A personalized editing strategy is supported, facial expression changes and voice emotion peak values are aligned through dynamic time warping, a CLIP-like structure is used for training a bimodal encoder, the human face and the voice are mapped to a unified vector space, the association weight of the human face and the voice is automatically learned, and the intelligent degree of editing is improved.

The invention discloses an intelligent document analysis method of a knowledge base management system, and relates to the field of artificial intelligence large model multi-modal analysis. Comprising the following steps: step 1, preliminarily identifying a document format according to a document extension name, and performing corresponding analysis according to the type of the document format: performing text analysis for pdf, docx, doc and txt document formats, and extracting texts and images in the pdf document format for the pdf document format; aiming at the docx document format, analyzing the compressed package in the docx document format according to the natural sequence of the document, and extracting a text and a picture; aiming at the doc document format, identifying and extracting all text information in the doc document format; for a txt document format, character codes are recognized and processed, and various languages and symbols are extracted; the method comprises the following steps of 1, analyzing a text, 2, processing the analyzed text by utilizing a large model, namely segmenting the text, performing vectorization processing, converting the text into a numerical form which can be understood by the large model, and storing the converted text in a knowledge base for information retrieval, data analysis and knowledge mining, and 3, performing visual output on the text at the same time.

The invention relates to an intelligent ship repair quotation method and system based on multi-modal analysis and a large language model, and the method comprises the steps: firstly receiving a ship repair inquiry sheet file uploaded by a user, and carrying out the feature enhancement of image data through a DocEnTR model, and obtaining a binary image; automatically extracting unstructured text data, semi-structured table data and text information in the binarized image by adopting an OCR (Optical Character Recognition) technology, and performing multi-modal analysis on image features of the binarized image and the extracted text information by adopting a large language model to generate an initial data stream; synchronously generating a standard semantic tag and a non-standard content tag by adopting semantic association of the large language model, automatically generating a dynamic cue word according to the standard semantic tag, inputting the dynamic cue word into the large language model, and outputting a standardized engineering description; performing manual auditing on the non-standard content mark to correct the engineering item field so as to generate structured engineering data; and finally, calculating the total price based on the structured engineering data and the material quantity parameters, and further generating a final quotation list to complete ship repair quotation.

The embodiment of the invention relates to the technical field of artificial intelligence, in particular to a security behavior event recognition method and system based on multi-modal analysis, and the method comprises the steps: collecting a multi-modal monitoring data set of a target monitoring scene, the multi-modal monitoring data set comprising a visual imaging data stream and an environment sensing data stream; performing spatial feature extraction processing on the visual imaging data stream to obtain a motion state feature set of the target object; time sequence feature extraction processing is carried out on the environment sensing data stream to obtain an environment state feature set, and the environment state feature set comprises a sound field distribution feature, a temperature gradient feature and an illumination intensity feature; and carrying out space-time fusion processing on the motion state feature set and the environment state feature set to generate a fusion monitoring feature vector, calling an AI security behavior recognition model to carry out event classification processing on the fusion monitoring feature vector, and outputting a security behavior event recognition result of the target monitoring scene.

The invention discloses an intelligent verification system and method for full-process review based on multi-modal analysis, and belongs to the technical field of intelligent review. Comprising a processing analysis module, an experiment extraction module, a risk early warning module, a reproduction verification module, a chart comparison module, a dispute tracing module, a feedback analysis module, a load monitoring module, a report generation module and a preference analysis module. According to the invention, comprehensive and multi-angle intelligent review can be realized, more detailed risk early warning and positioning information can be generated, the experiment reproducibility can be quickly verified at low cost, the manual verification burden is greatly reduced, the innovation and risk of the thesis viewpoint are scientifically evaluated, the fatigue of a manuscript reviewer is relieved, the manuscript review efficiency and quality are improved, and the method is suitable for popularization and application. And dynamically adjusting the review content recommendation sequence.

The invention relates to the technical field of unmanned aerial vehicle training, and discloses an unmanned aerial vehicle training system and method based on multi-modal analysis and a dynamic scene, and the method comprises the steps: obtaining multi-source data, carrying out the preprocessing, obtaining multi-modal data, and inputting the multi-modal data into an intelligent analysis module; receiving the multi-modal data, analyzing the multi-modal data to obtain scene modeling information, and unifying the format of the scene modeling information; carrying out scene coverage simulation based on the scene modeling information, calculating a scene coverage rate, carrying out dynamic adjustment on the scene modeling information based on the scene coverage rate, updating the scene coverage simulation, calculating a training adaptation degree by fusing the hardware performance of the unmanned aerial vehicle after updating, and judging whether training is allowed or not; if yes, obtaining training performance data, calculating training performance values and sorting; training data of the trainees are obtained based on scene coverage simulation, the training proficiency is calculated, the training levels of the trainees are dynamically adjusted, training demand values of the trainees are calculated according to historical training data and sorted for analysis, and the training scene and the unmanned aerial vehicle are matched.

The invention discloses a multi-modal analysis-based image identification system and method, and the method comprises the steps: constructing a multi-modal feature index model, employing a deep learning-based image segmentation technology, designing a deep learning-based multi-modal matching mechanism, developing a visual analysis module, and displaying the identification process and result in a visual form. The transparency of the system is enhanced; and a man-machine interaction module is designed. According to the image identification system and method based on multi-modal analysis, through combination of deep learning, multi-modal data matching and visualization technologies, the system can automatically segment and match various elements such as topics, seals and picture features in a drawing, and meanwhile, provides visual display of associated information; and in addition, the defects of an existing system in the aspects of diversified data processing, transparency of the identification process and man-machine cooperation are overcome, and scientificity and efficiency of identification work are remarkably improved.

The invention relates to the technical field of artificial intelligence, and discloses a dialogue text mining method and system based on code scanning consumption cash back of big data, and the system comprises a consumption data collection module, a multi-modal analysis module, a strategy decision engine and a dynamic execution module. When code scanning consumption cash-back strategy decision making is carried out, real-time dialogue text semantics and consumption time-space behavior data are fused, a deep association mechanism of consumption intentions and payment scenes is established, the matching deviation of user language expression and actual consumption behaviors can be eliminated in real time, the consistency of a cash-back strategy and real consumption appeals is ensured, and the payment efficiency is improved. The problem of strategy mismatching caused by semantic understanding isolation in a traditional system is effectively avoided, the precision of personalized marketing is improved, double feedback of dialogue emotion tendency and historical strategy effects is dynamically analyzed, the deviation value of strategy feature matching is corrected in real time, and the system can actively adjust weight distribution of feature combinations in a strategy library.

The invention relates to the technical field of document intelligent review, in particular to a multi-mode digital publishing intelligent review system and method based on a large model, and the system comprises a sample construction module, a multi-mode analysis module, a structure recognition module, a term verification module and an annotation output module. According to the method, a model configuration data set is generated by analyzing a subject-object combination relationship in a text and jointly screening part-of-speech, sentence pattern and semantic structure, the accuracy of semantic recognition and structure judgment is improved, and local features of semantic offset and image-text expression separation are recognized by combining a comparison relationship between an image action path and a text behavior object; a paragraph logic structure and a theme connection mode are analyzed based on semantic offset information, the problems of content dislocation and theme jump between paragraphs are effectively revealed, matching change tracks and part-of-speech continuation fluctuation in term cross-paragraph contexts are recognized, label overlapping and coverage redundancy conditions are analyzed from sentence group hierarchy, and the term cross-paragraph contexts are obtained. And forming a label combination suggestion and constructing an annotation structure record.

The invention relates to the field of campus business management, discloses an AIoT-based campus business one-stop service system, and aims to solve the problems of low processing efficiency, serious information island, poor user experience, extensive energy consumption management and unintelligent equipment management of campus business in a traditional mode. An Internet of Things equipment integration unit of the system collects campus energy consumption, equipment state and people flow density data; the edge calculation unit preprocesses the data and identifies abnormity; the AI capability core unit provides natural language interaction and multi-modal analysis functions; the lightweight AI reasoning unit predicts equipment faults and gives maintenance suggestions; the dynamic model management unit realizes dynamic loading switching and framework expansion of multiple models; therefore, intelligent processing of campus business is realized, business processing efficiency is improved, information islands are broken, user experience is improved, energy consumption management is optimized, stable operation of equipment is guaranteed, and campus informatization management level is effectively improved.

The invention discloses a geographical intelligent data base station, which comprises a data sensing layer, a communication transmission layer, a management and calculation layer and a storage layer, and relates to the technical field of data communication. According to the geographical intelligent data base station, the intelligent data base station is arranged, the function boundary of a traditional base station is broken through, data localization storage, preprocessing and multi-modal analysis are achieved at the hierarchy of the base station, real-time decision feedback and data collaboration between distributed nodes are supported, and through a built-in self-adaptive algorithm, the intelligent data base station is obtained. Different scene requirements can be dynamically matched to complete tasks such as predictive analysis, anomaly detection and intelligent optimization, closed-loop data processing ecology is formed, response efficiency and system robustness are remarkably improved, infrastructure-level support is provided for intelligent internet-of-things application in a complex environment, various environmental factors are associated to establish an evaluation model, and the evaluation efficiency is improved. Data analysis and resource allocation are efficiently realized, and an execution strategy is timely generated for regulation and control.

The invention relates to the technical field of intelligent commodity management, in particular to an intelligent commodity multi-dimensional comparison method and system based on multi-modal analysis. The method comprises the following steps: acquiring a text input by a user; performing similar commodity screening in a preset commodity database based on the user input text to generate similar commodity screening data; performing commodity display definition evaluation on the similar commodity screening data to generate commodity first-dimension evaluation data; obtaining commodity description parameters; carrying out commodity three-dimensional modeling on the multi-modal feature commodity by utilizing the commodity description parameters based on the image features of the commodity, and generating a commodity display three-dimensional model; and performing commodity display accuracy evaluation on the commodity display three-dimensional model to generate commodity second dimension evaluation data. According to the method, multi-dimensional evaluation, dynamic display adjustment and a visual report are introduced, so that the comprehensiveness of commodity comparison is improved.

The invention discloses a vehicle fault diagnosis method and device, electronic equipment and a readable storage medium, the method is applied to a vehicle fault diagnosis system, and the vehicle fault diagnosis system comprises a vehicle, a cloud server and terminal equipment; a vehicle collects vehicle data, determines fault data in the vehicle data based on parallel analysis of a multi-core processor, and generates common fault information and repair suggestions based on the fault data; the vehicle sends the fault data to a cloud server, the cloud server carries out deep diagnosis on the fault data through a multi-modal analysis system and generates a diagnosis report, and the diagnosis report comprises complete fault information and a solution of the vehicle; and the terminal equipment receives and displays the generated common fault information, repair suggestions and diagnosis reports, and provides technical guidance of vehicle fault maintenance for a user based on remote connection. Therefore, the method disclosed by the invention is based on local processing of the vehicle, the fault response speed is improved, the load of the cloud server is reduced, and the stability of the system is improved.

The invention relates to the technical field of network information extraction, in particular to a visual webpage data crawling method and system based on a large model, and the method comprises the following steps: automatic browsing and screenshot, image preprocessing, OCR (Optical Character Recognition), multi-modal large model analysis, result fusion, large language model analysis, and data storage and output. The method has the beneficial effects that various picture or dynamic webpages can be handled through screenshot and multi-modal analysis, and dependence on a DOM (Document Object Model) structure is not needed; oCR and a multi-mode large model are complementary, and the extraction accuracy of data in different fonts, complex typesetting and charts is remarkably improved; a real browsing behavior is simulated and manual verification is combined, so that a common anti-crawling strategy can be effectively avoided; the large language model intelligently recombines page information according to Prompt, and the integrity and logicality of output data are guaranteed.

The invention discloses a fault repairing method of a server single board and electronic equipment, and relates to the field of server hardware reliability, and the method comprises the steps: scanning a plurality of target signal links of the server single board through a time domain reflectometer, correspondingly comparing the new impedance value and the new reflection waveform characteristic in the scanning result with the initial impedance value and the initial reflection waveform characteristic, determining at least one fault link of the server single board when the comparison result is inconsistent, and performing multi-modal analysis by combining the operation state data of the server single board. And determining the fault level of at least one fault link in the server single board and correspondingly generating a fault repair strategy to switch the fault link to the standby link, thereby solving the problems of single detection dimension, complex fault monitoring blind area, high-frequency signal monitoring failure and the like in the related technology, and improving the detection accuracy. Through time domain reflectometer monitoring, multi-modal data fusion analysis and redundancy switching, fault prediction, accurate positioning and lossless self-repairing of a server single board target signal link are realized.

The invention provides an intelligent digital person recommendation method based on multi-modal emotion calculation and cross-domain behavior analysis, and relates to the technical field of digital person personalized recommendation, and the method comprises the steps: collecting cross-domain behavior data streams of a target user in real time, including behavior events from different application fields and timestamps of the behavior events; performing multi-modal analysis on the behavior event, extracting a corresponding emotion feature, and generating a corresponding emotion vector; on the basis of the timestamps of the behavior events and in combination with the corresponding emotion vectors, behavior vectors of the target user are generated, and the behavior vectors represent key interests and demand states of the target user in the current time window; and according to the behavior vector of the target user, carrying out matching to obtain a corresponding recommendation action, so that the digital person executes personalized recommendation. According to the method, through cross-domain data fusion and emotion calculation, the recommendation accuracy, timeliness and emotion adaptability are remarkably improved, and the core defects that in the prior art, data are one-sided and emotion is neglected are overcome.

The invention discloses a children's language narrative ability assessment method and tool based on multi-modal analysis, and relates to the technical field of natural language processing, and the technical scheme is characterized in that audio data and video data of a testee during a narrative process are collected, and voice rhythm features, text language features and visual behavior features are extracted for fusion; the method comprises the steps of inputting a multi-modal scoring model to generate multi-dimensional quantitative scores of a macrostructure, a microstructure, a language organization and a pragmatic function of the language narrative ability of a testee child, generating a structured evaluation report for the testee child according to the scores, and giving personalized intervention suggestions. According to the method, multi-modal data such as voice, language texts and visual behaviors are collected, so that the narrative ability of the children can be comprehensively analyzed from multiple dimensions such as macrostructures, microstructures, language organizations and pragmatic functions, and the method is closer to the real natural expression state of the children; therefore, the comprehensive language ability and communication performance of children can be reflected more comprehensively and truly.

The invention discloses a mechanism investigation PDF-oriented multi-modal analysis method, and relates to the technical field of electrical digital data processing. The method comprises the following steps: step 1, analyzing a PDF file based on a layout segmentation model to obtain a structured region and an unstructured region; 2, processing the table of the structured area through a cross-page table recombination algorithm to obtain complete table data, extracting through a rule template matching algorithm to obtain key fields, and forming structured data; 3, the format self-attention enhancement model comprises an input layer, a Transform coding layer and a pre-training task module which are connected in sequence, a cross-modal attention mechanism and a multi-modal mask pre-training task are adopted in the pre-training stage, and layered fine adjustment and adversarial training are adopted in the fine adjustment stage; 4, the unstructured area is processed by a format self-attention enhancement model to obtain modularized information; through a format self-attention enhancement model and the like, high analysis efficiency, good integrity and good accuracy are realized.

The invention provides a major hazard source risk assessment method and system based on multi-modal analysis, and the method comprises the steps: determining a certain major hazard source as a to-be-assessed target, obtaining the multi-modal monitoring data of the to-be-assessed target, and determining the comprehensive risk value of the to-be-assessed target according to the multi-modal monitoring data; therefore, the risk level of the to-be-assessed target is determined according to the comprehensive risk value of the to-be-assessed target, the risk level of the major hazard source can be rapidly and accurately determined, real-time assessment analysis and display of the security risk of the major hazard source are realized, and sending, checking, feedback and supervision of early warning information can be timely and automatically completed according to the early warning level. That is to say, the method provided by the invention can quickly and accurately realize risk assessment of major hazard sources, improve accuracy and timeliness, and meet risk assessment requirements of users in different usage scenarios.

This invention discloses an intelligent alarm and control method for personnel safety in power plants, relating to the field of power industry safety control technology. The method includes: acquiring multimodal sensing data related to personnel based on the existing sensing system of the power plant; using a multimodal analysis model to fuse and analyze the collected multimodal sensing data to identify personnel safety risks; generating an alarm trigger signal when the risk level reaches a preset threshold; combining intelligent agent technology to generate a personalized alarm strategy after receiving the alarm trigger signal, simultaneously finding the nearest alarm device based on the real-time location of the personnel for continuous alarm, and establishing a continuous monitoring mechanism to track personnel status in real time; continuously monitoring the behavior and location changes of the alarmed personnel until the risk is confirmed to be completely eliminated. This invention enables cross-system correlation analysis, improves the accuracy of risk identification, effectively shortens response time, and improves the safety level of power plants.

This invention relates to the field of video inspection technology and discloses a multimodal model AI video inspection system and method based on cameras. The system includes: a video acquisition and processing module, an embedded vector calculation module, a difference judgment and triggering module, a multimodal analysis module, a work order management integration module, and a strategy dynamic adjustment module. During video inspection, by dynamically adjusting the screenshot acquisition interval, the detection frequency can be adaptively optimized according to the degree of scene change, avoiding resource waste caused by fixed intervals. This allows the system to reduce unnecessary analysis operations when there are no obvious changes in the image, thereby improving the efficiency of computing resources and the real-time performance of system operation, ensuring energy-saving and efficient inspection. During video inspection, by combining a multimodal large model with text prompting technology, visual features and scene context information are integrated for anomaly recognition, which can reduce the risk of misjudging subtle changes by a single model and improve the adaptability to complex scenes.

The invention provides a safety early warning method, system and equipment for a galvanizing process production line and a medium. The safety early warning method comprises the following steps: acquiring multi-source data of the galvanizing process production line; performing multi-modal analysis and vectorization processing on the multi-source data, constructing a galvanizing process production line knowledge graph, and storing the knowledge graph to an AI data lake; reading a knowledge graph from an AI data lake to adjust the general large model; transversely splitting the galvanization process into a plurality of process thinking chain nodes according to a process sequence; longitudinally splitting each process thinking chain node into an analysis node and an auxiliary node executed by the potential safety hazard analysis model to form a tree structure thinking chain; traversing the tree structure thinking chain based on a preset sequence, and analyzing the potential safety hazard of each process; and summarizing the potential safety hazard analysis results of all the processes, and generating a potential safety hazard report and rectification measures for safety early warning. The method helps managers to master potential safety hazards possibly existing after operation of a production line and corresponding rectification measures in the galvanization industry.

The invention discloses a software testing method based on multi-modal analysis and a terminal. The method comprises the following steps: monitoring a picture of equipment running to-be-tested software in real time, obtaining a frame-dropping picture in the picture and context data thereof, and analyzing the frame-dropping picture by using a multi-modal picture analysis model based on the context data to obtain a picture analysis text; extracting keywords in the picture analysis text, performing clustering operation on the extracted keywords, generating a question label corresponding to each cluster, and determining all check points according to the question labels; and checking the frame drop picture according to all the check points, marking the check points which do not pass the check result as problem data, and generating a test report, so that picture-level BUG detection can be realized, and the software test quality and efficiency are improved.

The invention relates to the field of intelligent recognition, in particular to an intelligent sign language ability identity recognition system based on multi-modal data analysis, which comprises the following steps of: S10, constructing a sign language animation resource library, and decomposing a sign language expression system in a national general sign language dictionary into 500 basic gesture units and 1200 composite action templates, combining an artificial intelligence technology to synthesize sign language animations; s20, randomly extracting four sign language animations and corresponding text vocabularies from the resource library, performing out-of-order processing on the text vocabularies, and performing distortion interference imaging processing; s30, pushing an interactive webpage containing the four sign language animations and the vocabularies subjected to graphical processing, starting 60-second countdown, and requiring a user to click matched vocabulary images according to an animation sequence; and S40, performing multi-modal analysis based on the user operation time difference and the coordinate position, and verifying the sign language ability of the user. Therefore, more resources can provide efficient services for the hearing-impaired speech disabled while the safety problem is solved through the system.

This invention discloses an automated data security assessment method and system based on intelligent agents, belonging to the field of data security technology. It includes defining and constructing multiple intelligent agents, determining agent attributes, and generating capability profiles for the agents based on these attributes, providing a reliable foundation for subsequent modal analysis and resource scheduling. Multimodal analysis is performed on data assets, and resource scheduling and allocation are performed based on the agent capability profiles. Output conflicts are handled through a confidence-flow arbitration mechanism. Traditional automated tools often generate numerous false positives or false negatives due to rigid rules. This system no longer simply takes the union or intersection of values, but uses a dynamically flowing confidence algorithm for automatic adjudication. The risks of data assets are quantified based on attack graphs and Bayesian networks. Reasonable scheduling and coordination of multiple intelligent agents resolve conflicts between them, improving the efficiency and reliability of automated data security assessment and meeting complex and ever-changing security assessment needs.