126 results about "Preference learning" patented technology

The invention discloses a machine behavior learning method based on emotion driving and human expert feedback, and the method comprises the following steps: a) fusing environment image features extracted by BLIP-2 and text instruction semantic features analyzed by GPT-4, and forming cross-modal input; b) training a basic VLA model through supervised fine tuning (SFT) by using human expert remote control trajectory data and cross-modal input to obtain a basic behavior strategy; c) combining an emotion recognition module with a multi-head self-attention mechanism, fusing emotional interaction dependency into a basic strategy, and generating high-order emotion driven behavior representation; and d) inputting the high-order emotional behavior representation into a reinforcement learning module, storing a track by using a Replay Buffer, carrying out optimization through human expert feedback preference learning, and outputting a final behavior strategy. Compared with an existing method, the method has the advantages of being high in multi-modal feature extraction capacity, high in emotion fusion degree, sufficient in expert feedback utilization and the like, and the response accuracy and interaction experience of a machine to human instructions and emotions can be improved to a certain degree.

The invention relates to a text-to-SQL (Structured Query Language) generation method and system based on large language model fine tuning. The method comprises the following steps: collecting a triple data set; generating a reasoning path for the triple data set, and obtaining an optimized data set comprising the triple data set and the corresponding reasoning path; selecting a basic model, and performing supervised fine tuning training on the basic model through the optimized data set to obtain a reference model; executing feedback through a database corresponding to an output result of the reference model, and performing direct preference optimization training on the reference model to obtain a generative model; and performing SQL generation on a to-be-generated text through the generation model to obtain a generation result. The method utilizes the advantages of the model language reasoning ability and the preference learning algorithm to improve the accuracy of the model in the aspect of text-to-SQL (Structured Query Language) generation tasks.

The invention relates to the technical field of artificial intelligence, in particular to an intelligent content evaluation and optimization method and system based on multi-standard preference learning, and the method comprises the steps: constructing target user preference data, and generating an evaluation track containing evaluation rules and judgment results; based on the evaluation trajectory, screening samples and distributing credits through sorting and consistency rules to obtain preference pair training data; jointly training a generative reward model by adopting response supervision fine tuning and a direct preference optimization strategy; recombining the original evaluation trajectory into alternate accepting and rejecting samples, forming long thinking chain training data, and further training to obtain a final model; and evaluating and optimizing the alignment degree of the generated text and the user preference through the final model. According to the method, through multi-stage optimization and process supervision, the alignment degree and the overall performance of the language model and human preference are improved, the problems of composite errors, data sparseness and the like of a traditional reward model are solved, and the method is excellent in performance in out-of-distribution evaluation.

The invention discloses a large model scene question and answer optimization method and system based on preference learning, and the method comprises the steps: obtaining the query data of a user in a specific scene application, and generating a preset number of candidate answers for each question; introducing a human feedback mechanism, obtaining a preference sequence of a preset number of candidate answers corresponding to each question by a human, and forming a feedback data set; and carrying out iterative training on the large model by using the feedback data set in combination with a loss function of preference alignment, so as to realize optimization and adjustment of parameters of the large model. Through the technical scheme of the invention, different business scenes can be more flexibly adapted, the effectiveness in practical application is improved, the adaptability of the model is remarkably improved, the accuracy of answers is improved, the semantic correlation of the answers is ensured, and the efficiency of model training is improved.

The invention discloses a system for automatically generating video script creation, which relates to the technical field of script generation and comprises a content generation module for automatically generating a video script by adopting a script generation algorithm according to a theme and keywords input by a user; the retrieval experience enhancement module is used for collecting feedback data of the user to the video script and processing the feedback data to obtain quantized data, and adjusting a script generation algorithm based on the quantized data; the preference learning module is used for collecting historical creation data of a plurality of video scripts generated by using a script generation algorithm and corresponding feedback data, and training an online preference model; and the online iterative optimization module is used for acquiring feedback data of the user on the generated video script and the trained online preference model, and carrying out iterative optimization on a script generation algorithm. Through the scheme, multiple video scripts can be quickly generated according to user requirements, the topic selection direction of video creation is enriched, the video creation time is shortened, and the creation quality of the video scripts is continuously improved according to historical experience summarization.

The invention provides a preference learning combined multi-agent recommendation method and device, electronic equipment, a storage medium and a program product, and the method comprises the steps: determining task query information of a query user, translating the task query information, and obtaining task demand information; reasoning based on the query user, the task query information and the task demand information to obtain an initial answer; evaluating the initial answer through a pre-trained preference learning alignment model to obtain optimization feedback; and adjusting the initial answer based on the optimization feedback to obtain an optimization recommendation answer. According to the method, the fine preference and the context information of the user can be fully utilized, so that the relevance and the accuracy of recommendation are improved.

The invention provides a preference learning method which can be applied to the technical field of artificial intelligence. The preference learning method comprises the steps that sample questions matched with preset standards are determined from a text document pre-trained by a large model, the preset standards comprise a forward standard and a reverse standard, and the sample questions are used for representing questions generated when a user interacts with the large model; inputting the plurality of sample questions into a large model to generate a plurality of question and answer pairs according to answers corresponding to the sample questions output by the large model; evaluating answers in each question and answer pair based on a forward standard by using a plurality of evaluation models to obtain an evaluation label corresponding to each question and answer pair; and training the large model by using preference data generated based on the evaluation label to obtain a target model aligned with the forward standard. The invention further provides a model application method, a preference learning device, equipment, a storage medium and a program product.

The invention provides a method and system for enhancing the domain specific capability of a large language model based on preference learning, and the method comprises the steps: obtaining unlabeled data, carrying out the reasoning of the unlabeled data through a preset domain expert model, and generating corresponding output distribution; inputting the output distribution into a large language model, and generating preference enhancement data based on the output distribution through the large language model; summarizing the preference enhancement data, arranging the preference enhancement data into a new training data set, and performing fine adjustment on the large language model through the new training data set; and performing model evaluation on the fine-tuned large language model, and performing secondary adjustment on the large language model based on an evaluation result to complete field specific capability enhancement of the large language model. The problem that text generation of an existing large language model in a specific field is inaccurate is solved.

The invention relates to the technical field of image processing, and provides an indoor three-dimensional image intelligent rendering method based on image processing. Comprising the following steps of indoor multi-view image acquisition, image preprocessing, intelligent analysis of indoor scene elements, construction of an indoor three-dimensional initial model with attributes, intelligent generation of rendering parameters, adaptive LOD real-time rendering and intelligent interaction optimization. According to the intelligent interaction optimization system, through deep fusion of natural language processing, user preference learning and real-time rendering technologies, a set of efficient, visual and personalized virtual scene rendering interaction process is constructed. According to the method, the problems that traditional graphic software is complex in operation, high in learning cost and low in debugging efficiency are solved, and the user satisfaction and creation efficiency are remarkably improved through an intelligent recommendation and rapid iteration mechanism.

The invention relates to the technical field of intelligent lighting control, in particular to a multi-user sharing-oriented self-adaptive lighting intelligent identification control method, which comprises the following steps of: identifying user identities through a multi-user feature identification system, acquiring historical use habit data, generating personalized lighting parameters based on the data and current environment data, the group preference learning system constructs similar user clusters to optimize shared space illumination; the conflict adjustment system identifies and processes multi-user demand conflicts; the priority distribution system dynamically adjusts user control authority; the feedback learning system obtains user feedback, calculates satisfaction, updates a preference model and continuously optimizes an illumination control strategy; intelligent lighting control based on user features and group preferences is achieved, conflict processing and continuous learning capabilities are achieved, rapid and accurate recognition of users is achieved through a face recognition technology, lighting parameters are automatically adjusted in combination with historical use habit data, and personalized lighting experience is provided; dynamic changes of user demands are captured, and the accuracy of illumination scene recommendation is improved.

The invention relates to an intelligent clothing recommendation system based on image recognition and a recommendation method thereof, and the system obtains a human body image through a multi-modal data collection module, constructs a three-dimensional body model in combination with a deep learning algorithm, and achieves the precise measurement of key parameters such as shoulder breadth and chest circumference. And customized recommendation schemes are generated based on body types such as apple types and pear types. The clothing recommendation module integrates fashion trend analysis and a scene adaptation algorithm, provides style screening, single article combination and dynamic AR fitting functions, and supports 360-degree rotation observation and fabric texture simulation. The virtual fitting module generates dynamic wrinkles of clothes through motion capture, and the error is controlled within 150ms. The system integrates an e-commerce platform and a social sharing function, supports user preference learning and intelligent reminding, improves the accuracy of body type measurement, and improves the recommendation matching degree. Through three-dimensional body shape analysis, dynamic physical simulation and multi-mode interaction, the garment fitting precision and the user experience are remarkably improved.

Embodiments described herein provide a reinforcement learning framework for neural network models to generate outputs that align with desired human preference. In at least one embodiment, cross-prompts are generated from an original prompt to elicit a response from the neural network model.

The invention belongs to the technical field of precision equipment assembly, and discloses a human-in-the-loop precision equipment assembly precision regulation and control method and related device.The method comprises the steps that parameter input and precision output are obtained according to data obtained through actual measurement and adjustment of precision equipment, and a high-precision sample set database is established in combination with geometric parameters and physical parameters; obtaining empirical data, and constructing an empirical model and a preference learning model based on the empirical data; bayesian optimization is carried out based on a high-precision sample set database, a digital twinborn model is established in combination with a human-in-the-loop, optimization is carried out through the digital twinborn model, an empirical model and a preference learning model are combined, and an assembly adjustment scheme meeting the assembly precision and performance is selected. According to the method, empirical data are introduced into optimization regulation and control of complex precision equipment assembly, an effective method is provided for achieving rapid and accurate regulation and control of complex precision equipment, and the method has great significance in greatly improving the assembly precision of the complex precision equipment.

The invention discloses an MDI insulin dose decision-making system with expert preference learning, which belongs to the technical field of insulin injection therapy and comprises a model learning module fusing physiological information, a closed-loop feedback type optimization decision-making module, a safety constraint module and a human-in-loop decision-making self-adaption module. According to the MDI insulin dosage decision-making system with expert preference learning, expert decision-making experience can be effectively applied to insulin dosage optimization, individualized self-adaption of bottom layer insulin dosage optimization decision-making and top layer decision-making is achieved, and the MDI treatment effect is improved.

The invention provides a BERT model training method and system, computer equipment, a storage medium and a program product, and belongs to the technical field of natural language processing. The method comprises the following steps: respectively generating a plurality of sequences through a plurality of decoding paths by utilizing a pre-constructed BERT model; dividing the plurality of sequences into positive samples and negative samples; sampling the positive sample and the negative sample by using the reference model to construct a preference learning loss function; constructing a regular penalty term loss function; constructing a mask prediction loss function; and determining the total loss of the BERT model according to the preference learning loss function, the regular penalty term loss function and the mask prediction loss function to obtain a trained BERT model. By adopting the scheme, the BERT model is guided to learn the influence of different decoding paths on output, and the ability of the BERT model to select paths in a complex decoding space and generate sequences with higher quality is enhanced.

The invention discloses a method, a system and a device for designing a large model reasoning path verifier based on a preference tree. Aiming at the application of a large language model in a complex reasoning task, the invention provides a novel reasoning path verifier, namely a preference tree-based verifier. According to the method, a preference tree is constructed through a best-first search algorithm, and step-level pairing data is collected to perform preference learning training on a verifier, so that the problem that an existing verifier is only trained on a binary annotation reasoning path and cannot fully utilize a relative relationship of intermediate steps is solved. Compared with a traditional binary classification training method, the verifier carefully captures nuisance between reasoning steps through step-level preference, and therefore the whole reasoning path is evaluated more accurately. The method effectively improves the accuracy and reliability of the verifier, especially has excellent performance in arithmetic and common sense reasoning tasks, and can be further used for assisting math teaching and other tasks.

The invention discloses an intelligent exhibition item module clustering management method and system supporting multi-scene switching. The intelligent exhibition item module clustering management method comprises the following steps: step 1, real-time state acquisition; 2, generating a dynamic strategy; step 3, content updating and issuing; step 4, operation monitoring and fault processing; 5, multi-scene adaptation is carried out; and step 6, audience preference learning. According to the method, intelligent collaborative scheduling of exhibition item clusters is realized through a real-time state acquisition and dynamic strategy generation mechanism, the system can automatically optimize resource allocation based on multi-dimensional load evaluation and an audience interest model, the problem of resource scrambling caused by independent operation of traditional exhibition items is solved, content hot update and cluster-level synchronization are supported, and the system is suitable for popularization and application. The bottleneck of traditional display content homogenization is broken through, meanwhile, a predictive operation and maintenance closed loop is constructed by a dual-channel redundancy monitoring and three-level fault response system, the equipment downtime risk is remarkably reduced, and the system stability is improved.

The invention discloses a layering and preference learning-based personalized dynamic treatment strategy generation system for Parkinson's disease, which comprises a dynamic state and preference representation module, a preference-based reward function learning module, a layering treatment strategy generation module and a treatment path simulation and interactive output module, the system learns preferences from doctor-patient interaction and generates layered, dynamic and explainable personalized long-term treatment strategies, so that making of personalized long-term treatment decisions is achieved, and the problems that in an existing Parkinson's disease reinforcement learning scheme, an award function is fixed, and long-term planning ability is lacked are solved.

The invention belongs to the field of artificial intelligence, particularly relates to an intelligent sound field adaptive system of digital professional sound equipment, and aims to solve the problems of inaccurate sound field regulation and control, response lag, dependence on manual tuning and the like in a complex acoustic environment. The system comprises a sound field sensing module, an acoustic modeling and analysis module, a self-adaptive sound field regulation and control engine, a multi-channel digital audio processing unit and a feedback optimization module, and dynamic sound field modeling, real-time audio processing and environment self-adaptive regulation and control are realized through distributed sensing, hybrid modeling, multi-target optimization and closed-loop feedback. The system supports rapid re-calibration, multi-scene memory and user preference learning, ensures voice clarity and music fidelity, balances full-field hearing consistency, and significantly reduces manual intervention requirements.

Aiming at the problem of how to improve the accuracy of a false news detection model, the invention designs a multi-modal false news detection method based on content and external facts. In order to enable the two modes to better focus on own preference parts and eliminate other redundant information as far as possible, a deep heterogeneous graph convolutional network (HDGCN) is introduced as a preference learning device, which can generate own required preference perception for the two models and reduce redundant information. Meanwhile, in order to more effectively model the node representation of the graph neural network, style words in the news are expanded by semantic mining, entity words in the news are expanded by entity links, and the obtained five word representations are jointly used as the node representation of the graph network. Finally, experiments on a public data set show the effectiveness of the model. In the field of false news detection, an idea and a wide application prospect are provided.

The invention discloses a priority acquisition system for elevator scheduling, and relates to the technical field of elevator scheduling control, the system comprises a data acquisition module, a time sequence analysis module, an emotion recognition engine, a personalized elevator taking preference learning module, a scheduling decision module and a communication and execution module; according to the data acquisition module, a high-definition camera, a microphone array and a sensor are arranged in an elevator car, facial expressions and voice data of passengers are analyzed in real time through an emotion recognition engine, the comprehensive emotion value of the passengers is quantified, corresponding adjustment is conducted according to the emotion state, and meanwhile, the emotion recognition engine is used for recognizing the emotion of the passengers. Through the personalized elevator taking preference learning module, the elevator taking habits and preferences of the passengers are deeply excavated, exclusive preference portraits are constructed, the intelligent level of elevator dispatching is improved, elevator services can be individually adjusted according to the requirements and emotional states of different passengers, and therefore the elevator taking experience and satisfaction degree of the passengers are remarkably improved.

The invention belongs to the technical field of natural language processing, and particularly provides an NL2SQL automatic prompting method based on a retail commodity database, and the method comprises the steps: collecting NL2SQL problem pairs of the retail commodity industry, and carrying out the data augmentation through employing a data augmentation strategy, and obtaining an augmented database; in combination with augmented database execution feedback and ground-truth SQL contribution feedback, constructing a preference learning data set; performing preference learning fine tuning on the DELLM by using a DPO algorithm; and generating knowledge by using the DELLM subjected to preference learning fine tuning, and generating an SQL statement by combining user query, database architecture and instructions and assisting a Text-to-SQL model. Through the generation of the dynamic cue word and the expansion of knowledge annotation, the accuracy and efficiency of the NL2SQL model in processing complex and personalized query problems are remarkably improved. Besides, through supervised fine tuning and reinforcement learning, the model can continuously learn and adapt to a new query mode, so that more stable and reliable performance is provided in practical application.

The invention relates to the technical field of electrical engineering and artificial intelligence, and discloses a strong and weak current line intelligent layout and interference prediction analysis method and system.The strong and weak current line intelligent layout and interference prediction analysis method comprises the steps that a state representation and interference propagation model of electronic system layout is constructed; uncertainty quantification is carried out based on Bayesian reasoning and a particle filtering method; solving a Pareto optimal layout scheme set of the strong and weak current line by applying an improved multi-objective optimization algorithm; realizing robust reinforcement learning strategy optimization based on the distributed deep Q network; performing decision support through a preference learning model; according to the method, the problems of difficulty in multi-target balance, weak anti-interference capability and imperfect decision support in the traditional wiring technology are solved, the line layout design quality and efficiency are improved, and the electromagnetic interference risk is reduced.

The invention provides a text classification model optimization method and a text classification method and device.The method comprises the steps that each text in an original data set is divided, and semantic units of the texts at different levels are obtained; respectively enhancing the semantic units of each level by adopting different data enhancement modes, obtaining a preset number of supplementary texts for each text, and arranging the supplementary texts into an original data set to obtain an enhanced data set; constructing a preference reward function based on the keyword semantic score and the global semantic score; and on the enhanced data set, performing fine adjustment on the original text classification model by using a preference reward function to obtain an optimized text classification model. According to the method, a multi-strategy fine-grained data enhancement method is realized, a preference award function is constructed, and preference learning is introduced, so that model training is guided in the process of optimizing a text classification model, and the text classification model with higher accuracy and robustness is obtained through optimization.

The invention discloses a video rapid editing method based on artificial intelligence, and particularly relates to the technical field of video processing. Comprising the steps of S01, video data feature intelligent acquisition, S02, video data feature intelligent identification, S03, video clip intelligent generation, S04, video clip effect intelligent management, S05, video clip strategy evaluation and S06, user preference learning. A video frame to be edited is converted into a comparable standard numerical sequence from multi-dimensional quantitative evaluation, the intelligence and efficiency of editing are improved, the editing fluency of a video frame with a poor editing result is improved by continuously iterating and optimizing a video editing strategy through an editing effect fluency evaluation coefficient between adjacent video frames, and the editing efficiency is improved. By learning the preference of the user, the generated initial editing sequence conforms to the habits of the user.

The invention relates to the technical field of network data security, discloses a fact checking method and system based on multiple agents, a storage medium and electronic equipment, and aims at solving the problem that the understanding ability of complex declarations is insufficient in the prior art. Decomposing the complex original declaration into a plurality of sub-declarations which cannot be subdivided; in order to solve the problem that external knowledge is lacked to assist in verifying facts in the prior art, a retrieval agent taking Qwen2.5-72B as a basic model is constructed, and fact verification evidences are retrieved from a fact verification knowledge base or the Internet for sub-declarations; in order to solve the problem that in the prior art, a fact checking result lacking logic verification is directly output, Qwen2.5-72B trained through a human preference learning algorithm is constructed to serve as a decision-making agent, whether all evidences ei and answers ai in a sub-declaration sequence logically support an original declaration or not is evaluated, and a final fact checking conclusion is output.

Embodiments described herein provide for training an artificial intelligence model to become a preference-aware model. The artificial intelligence model preferences as the artificial intelligence model trains. Reinforcement learning is used to train experts in the artificial intelligence model such that each expert is trained to converge to a unique preference. The architecture of the artificial intelligence model is highly flexible. Upon executing a trained model, users can select automatically images according to various preferences based on medical professional preferences, geographic preferences, patient anatomy, and institutional guidelines.

The invention relates to the technical field of intelligent sunshade equipment control, in particular to an automatic control system and method for a sunshade, and aims to solve the problem that an existing automatic control system for the sunshade generally does not fully integrate multi-dimensional environment parameters and user intervention behaviors in the aspect of strategy generation, and the strategy generation efficiency is high. The problems that energy consumption, comfort and illumination uniformity are difficult to optimize synchronously under a dynamic meteorological condition, so that a control strategy is lack of individuation and foresight are solved; multi-dimensional environment semantic modeling, short-term illumination trend prediction and user preference learning are fused through a sun shading strategy dynamic generation module, a dynamic weighted multi-target optimization mechanism is constructed, energy consumption, comfort and illumination uniformity are synchronously considered under the condition that equipment constraints are met, a control target and an evaluation standard are updated in real time based on user intervention records, and the system is high in practicability. Accurate, prospective and highly personalized sunshade strategy generation is realized, and the intelligent level and comprehensive performance of the system are remarkably improved.

The application discloses a kind of based on key value memory network's complex electromechanical system design knowledge active recommendation method.First, the scene feature semantic information extraction of software platform log file in the design process of complex electromechanical system is carried out, and scene ontology is established based on scene feature semantic information, and system scene knowledge base is formed by scene ontology and original knowledge base;Then, the scene maximum frequent sequence is used to describe the scene sequence feature similarity between designers;Again, the knowledge item interaction sequence of all designers is learned, and the knowledge item sequence preference vector corresponding to all designers is obtained;Further, input to key value memory network, and the initial design knowledge active recommendation sequence is obtained;Finally, after knowledge item selection, the final design knowledge active recommendation sequence of each designer is obtained.The application obtains design knowledge active recommendation sequence more in line with the needs of designers, and then improves the efficiency of complex electromechanical system design.

The invention discloses a memory enhancement-based large language model lifelong alignment method. The method comprises the steps of focus preference optimization and short-time to long-time memory consolidation. The focus preference optimization adaptively focuses the learning focus on a new preference sample or a preference sample with an uncertain model through an improved preference learning loss function, meanwhile, the updating amplitude of fully learned knowledge is reduced, and a historical alignment result is protected while new preference is learned; the short-term to long-term memory consolidation is used for simulating a human memory mechanism, denoising is performed on short-term parameter update through singular value decomposition, an update part conflicting with past knowledge is identified and suppressed by projecting to a historical knowledge subspace, and finally refined conflict-free knowledge is integrated into long-term parameters of a model. According to the method, knowledge of the model is effectively accumulated and reserved in continuous and diversified alignment tasks, catastrophic forgetting is remarkably inhibited, and the stability, reliability and alignment consistency of the model in a dynamic and long-term deployment environment are improved.