A thinking machine control system and method for interactive graphics and texts

Abstract

The present invention discloses a thinking machine control system and method for graphic-text interaction, which specifically relates to the field of intelligent interaction and is used to solve the problems of context confusion and resource allocation optimization in graphic-text interaction. The method integrates natural language processing and image semantic analysis to monitor in real time and accurately identify the subject and intention of graphic content input by users, so as to achieve accurate understanding of user needs. In combination with a multimodal semantic analysis algorithm, the method can effectively compare the current input with the previous interactive content, judge the change of subject or intention, generate a new context identifier and complete context segmentation, so as to ensure that new problems are processed in an independent context environment. The method allocates a unique session identifier and adopts a distributed session management architecture to avoid context confusion and data cross-interference, so as to ensure data independence in different context areas. The method is based on a priority dynamic scheduling algorithm and a resource pool management technology to monitor the system load in real time and flexibly adjust resource allocation, so as to improve the efficiency of multi-context parallel processing.

Description

Technical Field

[0001] The present invention relates to the field of intelligent interaction, and more specifically, to a thinking machine control system and method for graphic-text interaction. Background Art

[0002] In the control system of the graphic-text interactive thinking machine, sometimes users input images and text information at the same time through a single interactive window to achieve control and management of the thinking machine. However, the existing context tracking mechanism has obvious limitations when dealing with multiple rounds of dialogue or continuous interaction. Specifically, when users ask multiple questions or instructions in the same window, the system often cannot accurately distinguish the independence of each interactive stage, resulting in the new input content being incorrectly associated with the previous context. This context confusion not only affects the system's accurate analysis of user intentions, causing the response to deviate from expectations, but also significantly increases the complexity of computing and resource consumption. In addition, with the continuous accumulation of interactive content, the response speed and processing efficiency of the system under high load conditions further decrease, seriously affecting the user experience and the overall availability of the system. The existing technology lacks effective solutions in context management and resource scheduling, and cannot meet the dual requirements of accuracy and efficiency of graphic-text interactive systems in complex interactive environments. Therefore, there is an urgent need for a technical method that can accurately divide the interactive stages and effectively manage the context to improve the accuracy, response speed and user satisfaction of the graphic-text interactive thinking machine control system.

[0003] In order to solve the above problems, a technical solution is now provided. Summary of the invention

[0004] In order to overcome the above-mentioned defects of the prior art, an embodiment of the present invention provides a thinking machine control system and method for interactive graphics and texts. By integrating natural language processing and image semantic analysis technology, the theme and intention of the graphic content input by the user can be monitored in real time and accurately identified, ensuring that the system accurately understands the user's needs. By using a multimodal semantic analysis algorithm, it is possible to effectively compare the current input with the previous interactive content, intelligently judge the changes in the theme or intention, thereby generating a new context identifier and performing context segmentation to ensure that new problems are processed in an independent context environment. By assigning a unique session identifier and adopting a distributed session management architecture, independent processing of data between different contexts is achieved, avoiding context confusion and cross-data interference. In addition, the priority-based dynamic scheduling algorithm and resource pool management technology can monitor the load in real time and dynamically adjust the allocation of computing resources, optimize the performance of multi-context parallel processing, and improve the response speed and processing efficiency of the system under high load and complex interactive environments to solve the problems raised in the above background technology.

[0005] To achieve the above object, the present invention provides the following technical solutions:

[0006] A method for controlling a thinking machine with interactive graphics and texts, comprising the steps of:

[0007] S1: Use natural language processing and image semantic analysis technology to monitor the text and image content input by users in real time and identify the subject and intention of the current input;

[0008] S2: Using a multimodal semantic analysis algorithm, compare the current input topic and intent with the previous interaction index information to determine whether there is a change in topic or intent;

[0009] S3: After detecting the change, a new context identifier is generated and the current input content is contextually segmented from the previous interaction content. The new question is processed in an independent context environment.

[0010] S4: Assign a unique session identifier to each independent context area and adopt a distributed session management architecture to achieve independent data processing between different contexts;

[0011] S5: Uses priority-based dynamic scheduling algorithms and resource pool management technology to monitor system load in real time and dynamically adjust computing resource allocation to optimize the performance of multi-context parallel processing.

[0012] In a preferred embodiment, step S1 includes the following contents:

[0013] First, the text and image data input by the user are preprocessed; the text part is cleaned and standardized through word segmentation, part-of-speech tagging and entity recognition natural language processing technology to generate word vector representation ; The image part uses a convolutional neural network to extract features and generate image feature vectors ;

[0014] Next, the feature vectors of text and image are fused through a multimodal fusion network , generate a comprehensive feature vector , and its calculation formula is: ;in, represents the fully connected layer, Represents a vector concatenation operation;

[0015] Then, using the topic modeling algorithm Perform topic identification on the comprehensive feature vector to generate the topic vector , the calculation process is as follows: ;in, represents the latent Dirichlet allocation model for extracting topic distribution in text and image content;

[0016] Next, apply the intent classifier Perform intent recognition on the comprehensive feature vector and generate intent labels , the calculation process is as follows: ;in, represents the fully connected layer, Used to generate probability distributions, representing the probabilities of different intent categories;

[0017] Finally, the identified topic vectors and intent labels are structured and stored as context identifiers.

[0018] In a preferred embodiment, step S2 includes the following contents:

[0019] The interaction index information includes the multimodal synergy index and the feature distribution divergence index;

[0020] Get the current context identifier and the previous context identifier; extract the fused feature vector of the current and previous contexts and , which correspond to the comprehensive characteristics of the current and previous interactions, respectively, through the multimodal synergy index and characteristic distribution divergence index After completing the calculation of the multimodal synergy index and the characteristic distribution divergence index, the synergy divergence nonlinear coupling measure SC-NLM is used for comprehensive analysis.

[0021] Multimodal Synergy Index The calculation process is:

[0022] Fusion feature vector based on current context and the fused feature vector of the previous context First, we construct a collaborative weighted matrix , where each matrix element express and The degree of point-to-point coordination in the feature space is defined as ,in is a parameter that controls the Gaussian decay rate; then, all elements in the co-weighted matrix are summed to obtain the co-weighted sum : , and calculate the normalization factor , the normalization factor eliminates the influence of the absolute size of the eigenvalue through the bi-norm product of two vectors; finally, the multimodal synergy index is defined as the ratio of the synergy weighted sum to the normalization factor, and its result measures the synergy consistency between the current and previous contexts in the multimodal feature space.

[0023] Feature distribution divergence index The calculation process is:

[0024] Based on kernel density estimation, by fusing features of the current context Fusion features with previous context The probability distribution of and Fitting is performed to capture distribution differences; first, kernel density estimation is performed on the data point set of the current context fusion feature and the previous context fusion feature to generate a smooth probability distribution and ; Then, we calculate the Jensen-Shannon divergence of the two distributions, defined as ,in is the average of two probability distributions, and is the Kullback-Leibler divergence, which is used to quantify the relative entropy between distributions. At the same time, in order to improve the sensitivity to local distribution differences, the calculation of segment entropy difference is used, which is defined as ,in and is the current and previous distribution at the sampling point The kernel density estimate at ; Finally, by combining the Jensen-Shannon divergence with the sensitivity to local distribution differences, the smooth divergence is defined as , and apply the hyperbolic tangent map to the result To enhance the ability to distinguish significant distribution differences.

[0025] In a preferred embodiment, step S3 includes the following contents:

[0026] When the cooperative divergence nonlinear coupling measurement value exceeds the corresponding threshold, it indicates that the current context has changed significantly and the context segmentation process needs to be started;

[0027] Extract the fused feature vector of the current context and the previous context; use the feature cross-segmentation formula to calibrate the feature dimension set where the segmentation point is located : ;in, is the feature dimension, is the sensitive threshold of the corresponding dimension, which is used to capture the significant difference dimension in the multimodal features; based on the feature dimension set, the current context feature segmentation weight matrix is ​​calculated , whose elements Representation Dimension With Dimension The segmentation synergy strength is: ;in, and Characteristics and The distribution scale parameter is used to avoid bias caused by differences in feature magnitude; the feature dimensions of the current context identifier and the previous context identifier are remapped using the current context feature segmentation weight matrix to generate an independent context data structure after segmentation; the semantic and intentional independence of the new and old context data is ensured through segmentation mapping; a new context identifier is generated for the segmented independent context data.

[0028] In a preferred embodiment, step S4 includes the following contents:

[0029] After confirming that context segmentation is required, first obtain the newly generated context identifier and the previous context identifier from step S3; next, retrieve the historical session records of the current user from the session management database to prevent repeated generation of session identifiers; in order to generate a unique session identifier, connect the new context identifier, the current timestamp and the user identifier, and process them through an irreversible hash function to generate a fixed-length session identifier with a low conflict rate; then, adopt a distributed session management architecture to assign the generated session identifier to independent data storage nodes, each node is responsible for storing the interaction data of the corresponding context area; in addition, establish a bidirectional mapping relationship for each session identifier, associate the session identifier with its corresponding context identifier and storage node, and store these mapping information in a structured manner in the session mapping table in the session management database; ensure the logical and physical independence of data in different context areas, avoid data cross-references and processing conflicts; finally, assign unique session identifiers and realize independent data processing.

[0030] In a preferred embodiment, step S5 includes the following contents:

[0031] In the distributed session management architecture, the session identifier and context identifier entries generated and persistently stored in step S4 are first retrieved; then, the operation index and load factor of each session are calculated by real-time monitoring of the processing queue, network congestion level, and processing capacity of each storage node during system operation; the session time sensitivity parameter and the computing resource usage rate parameter are combined to form a dimensionless comparable coefficient, thereby characterizing the urgent need for resources of the session; specifically, the time sensitivity is defined as It is the logarithmic curve of the difference between the latest session completion time and the current time;

[0032] On the other hand, the operating coefficient is defined as an exponential mapping of the ratio of resource occupancy rate to available rate .

[0033] Combine time sensitivity and operation coefficient to construct scheduling decision coefficient : ; By monitoring the scheduling decision coefficients of all active sessions, dynamically sort and put high-priority sessions into the front-end queue of the resource pool management module, and allocate higher execution threads or node bandwidth to them when allocating resources; after completing a scheduling cycle, the latest session execution status, including the scheduling decision coefficient and session execution node information, is appended to the scheduling status table of the session management database in a structured form, so that it can be directly retrieved and updated in subsequent scheduling cycles.

[0034] A thinking machine control system for interactive graphics and texts, comprising: a content analysis module, a semantic judgment module, an upper and lower segmentation module, a session management module and a resource scheduling module;

[0035] Content parsing module: uses natural language processing and image semantic analysis technology to parse the text and image content input by users in real time, identify the subject and intent, and pass the generated subject and intent information to the semantic judgment module;

[0036] Semantic judgment module: Through the multimodal semantic analysis algorithm, the current input topic and intention are compared with the previous interaction index information to determine whether there are significant changes, and the judgment results are passed to the upper and lower segmentation modules;

[0037] Upper and lower segmentation module: Generates a new context identifier based on the semantic judgment result, performs context segmentation on the current and previous input content, and passes the generated new context identifier and the segmented context data to the session management module;

[0038] Session management module: assigns a unique session identifier to each independent context area, and implements independent data processing through a distributed session management architecture, passing the assigned session identifier and context area information to the resource scheduling module;

[0039] Resource scheduling module: It uses a priority-based dynamic scheduling algorithm and resource pool management technology to adjust computing resource allocation in real time to optimize the performance of multi-context parallel processing.

[0040] The technical effects and advantages of the present invention's graphic-text interactive thinking machine control system and method are as follows:

[0041] The present invention integrates natural language processing and image semantic analysis technology to monitor and accurately identify the subject and intention of the graphic content input by the user in real time, ensuring that the system accurately understands the user's needs. By using a multimodal semantic analysis algorithm, it is possible to effectively compare the current input with the previous interactive content, intelligently judge the change of the subject or intention, thereby generating a new context identifier and performing context segmentation, ensuring that new problems are processed in an independent context environment. By allocating a unique session identifier and adopting a distributed session management architecture, data between different contexts can be processed independently, avoiding context confusion and data cross-interference. In addition, the priority-based dynamic scheduling algorithm and resource pool management technology can monitor the load in real time and dynamically adjust the allocation of computing resources, optimize the performance of multi-context parallel processing, and improve the response speed and processing efficiency of the system under high load and complex interactive environments. Overall, the present invention significantly improves the accuracy, response speed and user satisfaction of the graphic interactive thinking machine control system, ensures that the system runs stably and efficiently in multi-round dialogues and high-concurrency interactive scenarios, and provides excellent user experience and intelligent service capabilities. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] Figure 1 A schematic diagram of a flow chart of a method for controlling a thinking machine with interactive graphics and texts according to the present invention;

[0043] Figure 2 It is a structural schematic diagram of a thinking machine control system with interactive graphics and texts according to the present invention. DETAILED DESCRIPTION

[0044] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0045] Embodiment 1: Figure 1 The present invention provides a method for controlling a thinking machine with interactive graphics and texts, comprising:

[0046] S1: Use natural language processing and image semantic analysis technology to monitor the text and image content input by users in real time and identify the subject and intention of the current input;

[0047] S2: Using a multimodal semantic analysis algorithm, compare the current input topic and intent with the previous interaction index information to determine whether there is a change in topic or intent;

[0048] S3: After detecting the change, a new context identifier is generated and the current input content is contextually segmented from the previous interaction content. The new question is processed in an independent context environment.

[0049] S4: Assign a unique session identifier to each independent context area and adopt a distributed session management architecture to achieve independent data processing between different contexts;

[0050] S5: Uses priority-based dynamic scheduling algorithms and resource pool management technology to monitor system load in real time and dynamically adjust computing resource allocation to optimize the performance of multi-context parallel processing.

[0051] In the graphic-text interactive thinking machine control system, users input image and text information simultaneously through a single interactive window to achieve control and management of the thinking machine. However, the existing context tracking mechanism has obvious limitations when dealing with multiple rounds of dialogue or continuous interaction. When users continue to input new graphic content in the same window, the system often cannot accurately identify and distinguish the interactive content at different stages, resulting in the new input content being incorrectly associated with the previous context. This context confusion not only affects the system's accurate analysis of user intentions, causing the response to deviate from expectations, but also significantly increases the complexity of computational processing and resource consumption, reducing the system's processing efficiency and response speed. Therefore, there is an urgent need for a technical method that can accurately divide the interaction stage and effectively manage the context to improve the accuracy, response speed and user satisfaction of the graphic-text interactive thinking machine control system.

[0052] Step S1 includes the following contents:

[0053] First, the text and image data input by the user are preprocessed. The text part is cleaned and standardized through natural language processing technologies such as word segmentation, part-of-speech tagging and entity recognition to generate word vector representation. The image part uses a convolutional neural network to extract features and generate image feature vectors .

[0054] Next, the feature vectors of text and image are fused through a multimodal fusion network , generate a comprehensive feature vector , and its calculation formula is: ;in, represents the fully connected layer, Represents a vector concatenation operation.

[0055] Multimodal fusion network plays a key role in the control system of text-image interactive thinking machine. It realizes comprehensive understanding of user input by comprehensively processing and integrating multiple information sources from text and images. The network first uses natural language processing technology and convolutional neural network to extract high-dimensional feature vectors from text and image, respectively, which are recorded as word vectors and image feature vectors. Subsequently, the fully connected layer is used to perform independent nonlinear transformation on these two feature vectors to enhance their respective expression capabilities. Then, the transformed text features and image features are fused to generate a comprehensive feature vector through vector concatenation operation. This comprehensive vector contains the joint semantic information of text and image, which can more accurately reflect the overall intention and theme of user input. In order to further improve the fusion effect, the multimodal fusion network may introduce an attention mechanism to dynamically adjust the importance of different modal features to ensure that key features are given priority. In addition, the network can further refine and optimize the comprehensive features through multi-layer perceptron or other deep learning architectures to enhance the model's ability to understand complex interactive content. Finally, the comprehensive feature vector is used for subsequent topic identification and intent classification to ensure that the system can accurately parse user intentions under multimodal input and improve the relevance and accuracy of the response. This multimodal fusion method not only improves the system's ability to process complex graphic and text inputs, but also achieves a higher level of semantic understanding and intelligent response through deep feature integration.

[0056] Then, using the topic modeling algorithm Perform topic identification on the comprehensive feature vector to generate the topic vector , the calculation process is as follows: ;in, Represents the Latent Dirichlet Allocation model for extracting topic distributions in text and image content.

[0057] The Dirichlet allocation model is a generative probabilistic model for topic analysis of discrete data (such as text or image features). The Dirichlet allocation model assumes that each document is composed of several latent topics, and each topic is generated by a set of words according to a certain probability distribution. The core idea of ​​the model is to reveal the latent semantic structure in the data by inferring the topic distribution of each document and the word distribution of each topic. In implementation, the Dirichlet allocation model regards documents as a collection of words and defines two main probability distributions: the distribution from documents to topics (usually modeled by the Dirichlet distribution) and the distribution from topics to words (also modeled by the Dirichlet distribution).

[0058] The specific process includes: first assigning a topic distribution to each document, which is sampled from the Dirichlet distribution; then, assigning a word distribution to each topic, also sampled from the Dirichlet distribution; then, for each word in the document, first sample a topic from the topic distribution, and then sample a word from the word distribution corresponding to the topic. The goal of the Dirichlet allocation model is to estimate the distribution of topic distribution and word distribution based on the observed document data through an inverse generation process, usually using methods such as variational inference or Gibbs sampling for parameter learning.

[0059] In the text-image interactive system, the Dirichlet allocation model can be used to analyze the semantic information in text features and image features and identify the topic distribution of each input. By combining the input text with the image feature vector and inputting it into the Dirichlet allocation model, the potential topic structure of multimodal data can be captured, providing support for subsequent intent classification and context management. This modeling method is not only open and comprehensive, but also can effectively handle complex semantic relationships in massive data, laying the foundation for intelligent response in multimodal interaction scenarios.

[0060] Next, apply the intent classifier Perform intent recognition on the comprehensive feature vector and generate intent labels , the calculation process is as follows: ;in, represents the fully connected layer, Used to generate probability distributions representing the probabilities of different intent categories.

[0061] Finally, the identified topic vectors and intent labels are structured and stored as context identifiers for context management and processing in subsequent steps. The format of structured data storage is as follows:

[0062] ;

[0063] Represents the topic information of the current context. The topic distribution vector extracted by the topic modeling algorithm (such as LDA) is used to represent the main semantic scope and content category of the current context.

[0064] Indicates the intent information of the current context. The label or vector obtained by the intent classifier (such as a multi-layer perceptron) is used to characterize the user's operation goal or problem requirement in the current interaction.

[0065] Indicates the timestamp when the context interaction occurs. It is used to record the time point of the current interaction and supports time series analysis and management of the context.

[0066] Indicates the unique user identifier corresponding to the current context, which is used to distinguish and manage different user contexts in multi-user interaction scenarios.

[0067] Through the above processing flow, the subject and intent of the graphic content input by the user are monitored in real time and accurately identified, providing key basic data for subsequent context segmentation and session management. The generated context identifier will be used as a basis for judging whether there is a change in subject or intent, ensuring that new questions are handled in an independent context environment.

[0068] After completing step S1, the topic and intent of the user's current input have been successfully extracted and identified, and a structured context identifier has been generated. This result lays a solid foundation for comparing the current input with the previous interactive content and judging the change of topic or intent through the multimodal semantic analysis algorithm in step S2, ensuring that the system can accurately divide and manage different interaction stages in subsequent steps.

[0069] Step S2 is based on the context identifier generated in step S1, combined with the calculation results of the multimodal synergy index and the feature distribution divergence index, and further conducts a comprehensive analysis of the synergy divergence nonlinear coupling measure to quantitatively evaluate the multimodal feature consistency and distribution differences between the current context and the previous context, to determine whether there are significant changes in the subject or intention, and to provide a scientific basis for subsequent context segmentation and new identifier generation.

[0070] Step S2 includes the following contents:

[0071] The interaction index information includes the multimodal synergy index and the feature distribution divergence index.

[0072] Get the current context identifier and the previous context identifier; extract the fused feature vector of the current and previous contexts and , which correspond to the comprehensive characteristics of the current and previous interactions, respectively, through the multimodal synergy index and characteristic distribution divergence index Calculation is performed to determine whether there is a change in the subject or intent of the current input.

[0073] The fused feature vector is a high-dimensional representation vector generated by multimodal feature fusion of the text feature vector of the current or previous context (extracted by the natural language processing model) and the image feature vector (extracted by the convolutional neural network). It comprehensively reflects the multimodal semantic feature correlation of the context in terms of semantics, contextual and visual information.

[0074] After the calculation of the multimodal synergy index and the characteristic distribution divergence index is completed, the synergy divergence nonlinear coupling measure SC-NLM is used for comprehensive analysis. For example, the following calculation formula is used: ;in Used to describe the gradual change of the degree of synergy in the high value range, Emphasize the impact of distribution divergence when there are large differences, The other part is to describe the relative deviation between the multimodal synergy index and the feature distribution divergence index. is a nonlinear order (which can be adjusted according to the experiment), thereby capturing potential mutations in topics or intentions in multiple dimensions. When the value of the co-divergence nonlinear coupling measure exceeds the corresponding threshold, it can be determined that the current input and the previous context have significantly changed in multimodal features and distribution, thus providing a basis for subsequent context segmentation and the generation of new context identifiers.

[0075] Multimodal Synergy Index The calculation process is:

[0076] Fusion feature vector based on current context and the fused feature vector of the previous context First, we construct a collaborative weighted matrix , where each matrix element express and The degree of point-to-point coordination in the feature space is defined as ,in is a parameter that controls the Gaussian decay rate and is used to emphasize the contribution of eigenvalue differences to the synergistic relationship. Subsequently, all elements in the synergistic weighted matrix are summed to obtain the synergistic weighted sum. : , and calculate the normalization factor , the normalization factor eliminates the influence of the absolute size of the eigenvalue through the bi-norm product of two vectors, ensuring the comparability of the comparison of the synergy values; finally, the multimodal synergy index is defined as the ratio of the weighted sum of synergies to the normalization factor, and its result measures the synergy consistency between the current and previous contexts in the multimodal feature space. The higher the value, the stronger the degree of synergy and the closer the relationship between the features.

[0077] The multimodal synergy index is used to indicate the degree of synergy consistency between the current context features and the previous context features in the multimodal space. By quantifying the point-to-point synergy relationship between the current fused feature vector and the previous fused feature vector, the multimodal synergy index reflects whether there is a close semantic association between the text and image features in the feature space after multimodal fusion. The combination of the synergy weighting matrix and the Gaussian attenuation kernel function not only captures the similarity between feature points, but also adjusts the weights of the eigenvalue differences, so that the multimodal synergy index can accurately measure the consistency of the current context and the previous context in multimodal feature representation. A higher multimodal synergy index value indicates that the current context has a strong synergy with the previous context in feature distribution, and may belong to continuous inputs with the same semantics or intent.

[0078] Feature distribution divergence index The calculation process is:

[0079] Based on kernel density estimation, by fusing features of the current context Fusion features with previous context The probability distribution of and Fitting is performed to capture distribution differences; first, kernel density estimation is performed on the data point set of the current context fusion feature and the previous context fusion feature to generate a smooth probability distribution and ; Then, we calculate the Jensen-Shannon divergence of the two distributions, defined as ,in is the average of two probability distributions, and is the Kullback-Leibler divergence, which is used to quantify the relative entropy between distributions. At the same time, in order to improve the sensitivity to local distribution differences, the calculation of segment entropy difference is used, which is defined as ,in and is the current and previous distribution at the sampling point The kernel density estimate at ; Finally, by combining the Jensen-Shannon divergence with the sensitivity to local distribution differences, the smooth divergence is defined as , and apply the hyperbolic tangent map to the result This is to enhance the ability to distinguish significant distribution differences and ultimately reflect the degree of significant change in the distribution of two contextual features in the multimodal space.

[0080] Jensen-Shannon divergence is a symmetric measure of the difference between probability distributions. It introduces an intermediate distribution, quantifies the difference between the target distribution and the intermediate distribution, and takes the average value to measure the similarity of the two distributions. This divergence has symmetry and range limitation, and can effectively describe the overall difference between distributions. The smaller the value, the closer the distributions are. Kullback-Leibler divergence is an asymmetric information theory measure used to evaluate the degree of difference between a probability distribution when another probability distribution is assumed as a reference model. It is usually reflected in the size of the information loss. The larger the value, the more significant the difference between the two distributions. It is often used in probability distribution fitting and model optimization.

[0081] The feature distribution divergence index is used to indicate the overall degree of difference in feature distribution between the current context and the previous context. By fitting the probability distribution of the current and previous contexts through kernel density estimation, and combining Jensen-Shannon divergence and local distribution difference sensitivity, the feature distribution divergence index can accurately quantify the distribution changes of the two contexts in the multimodal fusion feature space. The feature distribution divergence index not only captures the differences between global distributions, but also emphasizes the significant changes of local features in different contexts through segment entropy differences. A higher feature distribution divergence index value indicates that there is a large difference in the distribution pattern between the current context and the previous context, which may point to different semantic scenarios or intent categories.

[0082] After completing step S2, the system successfully obtains the multimodal synergy index and feature distribution divergence index by deeply calculating the fused feature vector of the current context identifier and the previous context identifier, and further performs a comprehensive analysis through the synergy divergence nonlinear coupling measure to accurately quantify the comprehensive relationship between the current input and the previous context in terms of multimodal feature synergy consistency and distribution difference. When the synergy divergence nonlinear coupling measure value exceeds the preset threshold, the system can determine that the subject or intention of the current input has changed significantly, thereby triggering context segmentation and the generation of new context identifiers. This result ensures the accuracy of the system's context management in complex multimodal interaction scenarios, provides clear and reliable semantic structure support for subsequent steps, and significantly improves the system's response quality and intelligence level in multi-round interactions.

[0083] In step S2, the system determines whether there is a significant change in the subject or intent of the current input by calculating the multimodal synergy index and the feature distribution divergence index, combined with a comprehensive analysis of the synergy divergence nonlinear coupling measure. When the synergy divergence nonlinear coupling measure value exceeds the preset threshold, the system confirms that the current context is significantly different from the previous context, and context segmentation and new identifier generation are required to ensure independent processing of different semantic content or intent. This process provides a clear basis for judgment and necessary initial conditions for the implementation of step S3.

[0084] Step S3 includes the following contents:

[0085] When the value of the nonlinear coupling measure of the co-divergence exceeds the corresponding threshold, it indicates that the current context has changed significantly and the context segmentation process needs to be started. The current context identifier and the previous context identifier are obtained to provide basic data for subsequent segmentation and identifier generation.

[0086] Extract the fusion feature vector of the current context and the previous context. Use the feature cross-segmentation formula to calibrate the feature dimension set where the segmentation point is located. : ;in, is the feature dimension, It is the sensitive threshold of the corresponding dimension, which is used to capture the significant difference dimension in multimodal features.

[0087] Based on the feature dimension set, calculate the current context feature segmentation weight matrix , whose elements Representation Dimension With Dimension The segmentation synergy strength is: ;in, and Characteristics and The distribution scale parameter of is used to avoid bias caused by differences in feature magnitude.

[0088] The feature dimensions of the current context identifier and the previous context identifier are remapped using the current context feature segmentation weight matrix to generate an independent context data structure after segmentation. and : ;Through this segmentation mapping, the independence of new and old context data in terms of semantics and intent is ensured.

[0089] Generate new context identifiers for the segmented independent context data , the identifier is stored in a structured manner as follows: ;in, and based on The semantics of .

[0090] After completing step S3, the system successfully achieves the segmentation of the current context and the previous context, and generates a new context identifier. By segmenting the feature weight matrix and segmenting the data mapping, the independence of the context in terms of semantics and intent is ensured, effectively avoiding content confusion and processing conflicts. The generation of the new identifier provides a clear context boundary for the next step of session management, further optimizing the response accuracy and processing efficiency of multimodal interactions. This process not only maintains the accuracy of context segmentation, but also provides solid technical support for subsequent independent session management.

[0091] In step S3, the context segmentation is completed and a new context identifier is generated for the change between the current context and the previous context. This identifier, combined with the previous context identifier, constitutes the key input data required for subsequent session management. With this result, step S4 needs to assign a unique session identifier to each independent context area, and use a distributed session management architecture to achieve independent processing between different contexts. This process must be connected with the context identifier obtained in the previous article, and must also ensure the correct tracing and isolation of the new session in terms of technical logic.

[0092] Step S4 includes the following contents:

[0093] After confirming that context segmentation is required, firstly, the newly generated context identifier and the previous context identifier are obtained from step S3, which contain the subject, intent, timestamp and user identifier of the current and previous interactions respectively. Next, the historical session records of the current user are retrieved from the session management database to prevent the repeated generation of session identifiers. In order to generate a unique session identifier, the new context identifier, the current timestamp and the user identifier are connected and processed by an irreversible hash function to generate a fixed-length session identifier with a low conflict rate. Subsequently, a distributed session management architecture is adopted to assign the generated session identifier to independent data storage nodes, each of which is responsible for storing the interaction data of the corresponding context area. In addition, a bidirectional mapping relationship is established for each session identifier, the session identifier is associated with its corresponding context identifier and storage node, and the mapping information is structured and stored in the session mapping table in the session management database. This process ensures the logical and physical independence of data in different context areas, avoiding data cross-references and processing conflicts. Finally, by assigning unique session identifiers and realizing independent data processing, the system provides clear context boundaries and efficient data management mechanisms for multi-context parallel processing.

[0094] After completing step S4, each independent context area has been associated with a unique session identifier and occupies an independent data storage node in the distributed session management architecture, and the record of the previous context is updated or appended according to the actual situation. This process prevents cross-references or interference between data in different contexts, laying the foundation for parallel processing and resource allocation under multiple rounds of graphic and text interactions. By accurately generating and maintaining session identifiers, subsequent steps can implement priority scheduling and resource allocation for different contexts in complex scenarios, ensuring the efficiency and stability of multi-context parallel processing.

[0095] In step S4, each independent context area has been successfully assigned a unique session identifier, and a bidirectional mapping relationship with the context identifier has been established in the distributed session management architecture, forming a complete session record entry, ensuring the logical isolation and physical independent storage of data between different contexts. Based on this result, step S5 needs to monitor the system load in real time and flexibly allocate computing resources through dynamic scheduling strategies and resource pool management technology in a high-concurrency and multi-context parallel processing environment, so that each session obtains a reasonable execution priority and processing power, and maintains the efficient operation of the system in complex interactive scenarios.

[0096] Step S5 includes the following contents:

[0097] In the distributed session management architecture, first retrieve the session identifier and context identifier entries generated and persistently stored in step S4, and read the user ID, context topic, intent category, and actual storage node information of the session. Then, calculate the operation index and load factor of each session by real-time monitoring the processing queue, network congestion level, and processing capacity of each storage node during system operation. In order to clearly distinguish between urgent sessions and ordinary sessions, the session time sensitivity parameter and the computing resource usage rate parameter are used to combine the two to form a dimensionless comparable coefficient, thereby characterizing the session's urgent demand for resources. Specifically, define time sensitivity is the logarithmic curve of the difference between the latest session completion time and the current time, recorded as the output of the following function: ;in, Indicates the latest completion time that the session can tolerate in terms of semantics or tasks. is the system clock, 1 is an exponent used to adjust the sensitivity to time differences. On the other hand, the operating coefficient is defined as an exponential mapping of the ratio of resource occupancy rate to available rate. : ;in, Indicates the cumulative resource usage rate of the session since it was started. is the adjustment coefficient, which is used to amplify the value when the resource usage is low or compress the value when the resource usage is high. These two items are combined to construct the scheduling decision coefficient : ;

[0098] The scheduling decision coefficient comprehensively considers the urgency of the session and the current resource usage. The larger the value, the more urgent the session is for resources and the resource usage is not excessive, thus giving it a higher priority. Conversely, the smaller the value, the lower the urgency of the session or the higher the resource usage, and the lower the priority. By monitoring the scheduling decision coefficients of all active sessions, the high-priority sessions are dynamically sorted and placed in the front-end queue of the resource pool management module, and higher execution threads or node bandwidth are allocated to them during resource allocation. After completing a scheduling cycle, the system appends the latest session execution status (including the scheduling decision coefficient and session execution node information) to the scheduling status table of the session management database in a structured form, so that it can be directly retrieved and updated in subsequent scheduling cycles. The above process is executed cyclically with a fixed time step or load change threshold as the trigger condition during system operation, and if a change in session demand (such as a sharp increase in time sensitivity) is detected in any scheduling cycle, the immediate scheduling can be triggered in advance to avoid blocking.

[0099] Through the implementation of step S5, the graphic-text interactive thinking machine control system can quickly calculate the scheduling determination coefficient of the session and perform flexible resource allocation based on the session identifier, context identifier and data isolation structure provided in the previous step when facing a multi-context high-concurrency scenario. Each session is given a clear priority ranking and corresponding computing and network resources based on the combination of "time sensitivity-operation coefficient", ensuring that user needs can be responded to in a timely manner under complex interactive conditions. This dynamic scheduling method not only avoids the mutual scramble for resources between different contexts, but also makes full use of the horizontal expansion capabilities of the distributed architecture, providing high-efficiency and high-stability technical support for the graphic-text interactive thinking machine control system in an application environment with multiple rounds of dialogue and massive input.

[0100] Embodiment 2: Figure 2 The invention provides a thinking machine control system for interactive graphics and texts, including: a content analysis module, a semantic judgment module, an upper and lower segmentation module, a session management module and a resource scheduling module;

[0101] Content parsing module: uses natural language processing and image semantic analysis technology to parse the text and image content input by users in real time, identify the subject and intent, and pass the generated subject and intent information to the semantic judgment module;

[0102] Semantic judgment module: Through the multimodal semantic analysis algorithm, the current input topic and intention are compared with the previous interaction index information to determine whether there are significant changes, and the judgment results are passed to the upper and lower segmentation modules;

[0103] Upper and lower segmentation module: Generates a new context identifier based on the semantic judgment result, performs context segmentation on the current and previous input content, and passes the generated new context identifier and the segmented context data to the session management module;

[0104] Session management module: assigns a unique session identifier to each independent context area, and implements independent data processing through a distributed session management architecture, passing the assigned session identifier and context area information to the resource scheduling module;

[0105] Resource scheduling module: It uses a priority-based dynamic scheduling algorithm and resource pool management technology to adjust computing resource allocation in real time to optimize the performance of multi-context parallel processing.

[0106] The above formulas are all dimensionless and numerical calculations. The formula is a formula for the most recent real situation obtained by collecting a large amount of data and performing software simulation. The preset parameters in the formula are set by technicians in this field according to actual conditions.

[0107] The above description is only by way of illustration of certain exemplary embodiments of the present invention. It is undoubted that those skilled in the art can modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and descriptions are illustrative in nature and should not be construed as limiting the scope of protection of the claims of the present invention.

[0108] It should be noted that, in this article, if there are relational terms such as first and second, etc., they are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "including a..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0109] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A method for controlling a thinking machine with interactive graphics and text, characterized in that: Includes steps: S1: Use natural language processing and image semantic analysis technology to monitor the text and image content input by users in real time and identify the subject and intention of the current input; S2: Using a multimodal semantic analysis algorithm, compare the current input topic and intent with the interaction index information corresponding to the previous interaction to determine whether there is a change in topic or intent; S3: After detecting the change, a new context identifier is generated and the current input content is contextually segmented from the previous interaction content. The new question is processed in an independent context environment. S4: Assign a unique session identifier to each independent context area and adopt a distributed session management architecture to achieve independent data processing between different contexts; S5: Adopts priority-based dynamic scheduling algorithm and resource pool management technology to monitor system load in real time and dynamically adjust computing resource allocation to optimize the performance of multi-context parallel processing; The interaction index information includes the multimodal synergy index and the feature distribution divergence index; Get the current context identifier and the previous context identifier; extract the fused feature vector of the current and previous contexts and , which correspond to the comprehensive characteristics of the current and previous interactions, respectively, through the multimodal synergy index and characteristic distribution divergence index After the calculation of the multimodal synergy index and the characteristic distribution divergence index, the synergy divergence nonlinear coupling measure SC-NLM is used for comprehensive analysis; Multimodal Synergy Index The calculation process is: Fusion feature vector based on current context and the fused feature vector of the previous context First, we construct a collaborative weighted matrix , where each matrix element express and The degree of point-to-point coordination in the feature space is defined as ,in is a parameter that controls the Gaussian decay rate; then, all elements in the co-weighted matrix are summed to obtain the co-weighted sum : , and calculate the normalization factor , the normalization factor eliminates the influence of the absolute size of the eigenvalue through the bi-norm product of two vectors; finally, the multimodal synergy index is defined as the ratio of the synergy weighted sum to the normalization factor, and its result measures the synergy consistency between the current and previous contexts in the multimodal feature space; Feature distribution divergence index The calculation process is: Based on kernel density estimation, by fusing features of the current context Fusion features with previous context The probability distribution of and Fitting is performed to capture distribution differences; first, kernel density estimation is performed on the data point set of the current context fusion feature and the previous context fusion feature to generate a smooth probability distribution and ; Then, we calculate the Jensen-Shannon divergence of the two distributions, defined as ,in is the average of two probability distributions, and is the Kullback-Leibler divergence, which is used to quantify the relative entropy between distributions. At the same time, in order to improve the sensitivity to local distribution differences, the calculation of segment entropy difference is used, which is defined as ,in and is the current and previous distribution at the sampling point The kernel density estimate at ; Finally, by combining the Jensen-Shannon divergence with the sensitivity to local distribution differences, the smooth divergence is defined as , and apply the hyperbolic tangent map to the result To enhance the ability to distinguish significant distribution differences.

2. A method for controlling a thinking machine with interactive graphics and text according to claim 1, characterized in that: Step S1 includes the following contents: First, the text and image data input by the user are preprocessed; the text part is cleaned and standardized through word segmentation, part-of-speech tagging and entity recognition natural language processing technology to generate word vector representation ; The image part uses a convolutional neural network to extract features and generate image feature vectors ; Next, the feature vectors of text and image are fused through a multimodal fusion network , generate a comprehensive feature vector , and its calculation formula is: ;in, represents the fully connected layer, Represents a vector concatenation operation; Then, using the topic modeling algorithm Perform topic identification on the comprehensive feature vector to generate the topic vector , the calculation process is as follows: ;in, represents the latent Dirichlet allocation model for extracting topic distribution in text and image content; Next, apply the intent classifier Perform intent recognition on the comprehensive feature vector and generate intent labels , the calculation process is as follows: ;in, represents the fully connected layer, Used to generate probability distributions, representing the probabilities of different intent categories; Finally, the identified topic vectors and intent labels are structured and stored as context identifiers.

3. The method for controlling a thinking machine with interactive graphics and text according to claim 1, characterized in that: Step S3 includes the following contents: When the cooperative divergence nonlinear coupling measurement value exceeds the corresponding threshold, it means that the current context has changed significantly and the context segmentation process needs to be started; Extract the fused feature vector of the current context and the previous context; use Feature cross-segmentation formula, calibrating the feature dimension set where the segmentation point is located : ;in, is the feature dimension, is the sensitive threshold of the corresponding dimension, which is used to capture the significant difference dimension in the multimodal features; based on the feature dimension set, the current context feature segmentation weight matrix is ​​calculated , whose elements Representation Dimension With Dimension The segmentation synergy strength is: ;in, and Characteristics and The distribution scale parameter is used to avoid the bias caused by the difference in feature magnitude; the feature dimensions of the current context identifier and the previous context identifier are remapped using the current context feature segmentation weight matrix to generate an independent context data structure after segmentation; the independence of the new and old context data in terms of semantics and intent is ensured through segmentation mapping; a new context identifier is generated for the independent context data after segmentation.

4. A method for controlling a thinking machine with interactive graphics and text according to claim 3, characterized in that: Step S4 includes the following contents: After confirming that context segmentation is required, first obtain the newly generated context identifier and the previous context identifier from step S3; Next, the historical session records of the current user are retrieved from the session management database to prevent duplicate generation of session identifiers. In order to generate a unique session identifier, the new context identifier, the current timestamp, and the user identifier are concatenated and processed through an irreversible hash function to generate a fixed-length session identifier with a low conflict rate. Subsequently, a distributed session management architecture is adopted to distribute the generated session identifiers to independent data storage nodes, and each node is responsible for storing the interaction data of the corresponding context area; In addition, a bidirectional mapping relationship is established for each session identifier, the session identifier is associated with its corresponding context identifier and storage node, and the mapping information is structured and stored in a session mapping table in the session management database; Ensures logical and physical independence of data in different context areas, avoiding data cross-references and processing conflicts; ultimately, by assigning unique session identifiers and implementing independent data processing.

5. A method for controlling a thinking machine with interactive graphics and text according to claim 4, characterized in that: Step S5 Includes the following: In the distributed session management architecture, the session identifier and context identifier entries generated and persistently stored in step S4 are first retrieved; Then, by real-time monitoring of the processing queue, network congestion, and processing capacity of each storage node during system operation, the operation index and load factor of each session are calculated. The session time sensitivity parameter and the computing resource usage rate parameter are combined to form a dimensionless comparable coefficient to characterize the session's urgent demand for resources. Specifically, the time sensitivity is defined as It is the logarithmic curve of the difference between the latest session completion time and the current time; On the other hand, the operating coefficient is defined as an exponential mapping of the ratio of resource occupancy rate to available rate .

6. A method for controlling a thinking machine with interactive graphics and text according to claim 5, characterized in that: Step S5 also Includes the following: Combine time sensitivity and operation coefficient to construct scheduling decision coefficient : ; By monitoring the scheduling decision coefficients of all active sessions, dynamically sort and put high-priority sessions into the front-end queue of the resource pool management module, and allocate higher execution threads or node bandwidth to them when allocating resources; after completing a scheduling cycle, the latest session execution status, including the scheduling decision coefficient and session execution node information, is appended to the scheduling status table of the session management database in a structured form, so that it can be directly retrieved and updated in subsequent scheduling cycles.

7. A graphic-text interactive thinking machine control system, used to implement a graphic-text interactive thinking machine control method according to any one of claims 1 to 6, characterized in that: include: Content parsing module, semantic judgment module, upper and lower segmentation module, session management module and resource scheduling module; Content parsing module: uses natural language processing and image semantic analysis technology to parse the text and image content input by users in real time, identify the subject and intent, and pass the generated subject and intent information to the semantic judgment module; Semantic judgment module: Through the multimodal semantic analysis algorithm, the current input topic and intention are compared with the previous interaction index information to determine whether there are significant changes, and the judgment results are passed to the upper and lower segmentation modules; Upper and lower segmentation module: Generates a new context identifier based on the semantic judgment result, performs context segmentation on the current and previous input content, and passes the generated new context identifier and the segmented context data to the session management module; Session management module: assigns a unique session identifier to each independent context area, and implements independent data processing through a distributed session management architecture, passing the assigned session identifier and context area information to the resource scheduling module; Resource scheduling module: It uses a priority-based dynamic scheduling algorithm and resource pool management technology to adjust computing resource allocation in real time to optimize the performance of multi-context parallel processing.

Images