A multi-modal information fusion intelligent decision method and system

Abstract

This invention relates to the field of intelligent decision-making technology, and discloses an intelligent decision-making method and system based on multimodal information fusion. The system includes: a construction module for constructing a decision confidence interval based on the operational constraints and historical state information of the decision object; a constraint module for mapping different modal information into constraints acting on the decision confidence interval; a pruning module for progressively pruning the decision confidence interval according to the constraints; a decision module for determining the final decision result from the pruned decision confidence interval when it meets preset convergence conditions; and for adjusting the constraints or expanding and re-pruning the pruned decision confidence interval when it does not meet the convergence conditions; and an update module for dynamically updating the decision confidence interval. This invention reduces the uncertainty and error impact of a single information source, thereby significantly improving the accuracy and reliability of the decision results.

Description

Technical Field

[0001] This invention relates to the field of intelligent decision-making technology, and more specifically, to an intelligent decision-making method and system based on multimodal information fusion. Background Technology

[0002] In today's complex fields of industrial production, intelligent transportation, and medical diagnosis, decision-making objects are often in dynamically changing environments, and their state is influenced by a combination of factors. Traditional decision-making methods mostly rely on single-modal information or simple information superposition, making it difficult to effectively handle the correlation and complementarity between information from different sources and of different types, resulting in insufficient accuracy and reliability of decision results. For example, in industrial equipment fault diagnosis, relying solely on vibration signals or temperature data may not fully reflect the true state of the equipment, easily leading to misjudgments or omissions; in intelligent traffic scheduling, single video surveillance information or traffic flow data is insufficient to accurately predict the development trend of traffic congestion, affecting the timeliness and effectiveness of scheduling decisions. Furthermore, existing technologies also have significant shortcomings in dealing with information uncertainty, noise interference, and decision credibility assessment, failing to provide robust support for decision-making and making it difficult to meet the needs of high-precision and high-reliability decision-making in practical applications.

[0003] Therefore, it is necessary to design an intelligent decision-making method and system based on multimodal information fusion to solve the problems existing in the current technology. Summary of the Invention

[0004] In view of this, the present invention proposes an intelligent decision-making method and system based on multimodal information fusion, aiming to solve the problem of meeting the demand for high-precision and high-reliability decision-making in practical applications in the current technology.

[0005] This invention proposes an intelligent decision-making system based on multimodal information fusion, comprising: The construction module is used to construct the decision confidence interval based on the operational constraints and historical state information of the decision object; The constraint module is used to collect multimodal information related to the decision object and map different modal information into constraints that act on the decision confidence interval. The pruning module is used to progressively prune the decision confidence interval according to the constraints, and to detect the validity of the decision confidence interval during the pruning process; The decision module is used to determine the final decision result from the trimmed decision confidence interval when the trimmed decision confidence interval meets the preset convergence condition; It is also used to adjust the constraint conditions or expand and re-trim the pruned decision confidence interval when the pruned decision confidence interval does not meet the convergence condition. The update module is used to output the final decision result and dynamically update the decision confidence interval when new multimodal information arrives.

[0006] Furthermore, when constructing the decision confidence interval based on the operational constraints and historical state information of the decision object, it includes: The operational constraints are parsed to obtain the operational boundary parameters; Within the range of values ​​defined by the operating boundary parameters, the historical state information of the decision object is filtered to determine the historical effective decision interval; The historical effective decision intervals are processed by interval synthesis, and the intervals that exceed the range of the operating boundary parameters are removed to form candidate decision intervals; The candidate decision interval is subjected to integrity and consistency verification. If the verification passes, the candidate decision interval is determined as the decision confidence interval.

[0007] Furthermore, mapping different modal information into constraints acting on the decision confidence interval includes: The collected modal information is classified and processed to distinguish modal information from different sources and types; Feature extraction is performed on all the modal information respectively to obtain modal feature parameters associated with the running state of the decision object; Based on the modal feature parameters, generate corresponding modal constraint rules or modal constraint parameters; The modal constraint rules or modal constraint parameters are applied to the decision confidence interval to form constraints that limit the range of values ​​in the decision confidence interval.

[0008] Furthermore, when progressively pruning the decision confidence interval according to the constraints, the process includes: Collect the length of the decision confidence interval, the type of the decision object, and the strictness of the constraints; The pruning step size for progressively pruning the decision confidence interval is determined based on the interval length. The cutting step size is optimized based on the type and severity level to obtain an optimized cutting step size; The decision confidence interval is then gradually trimmed.

[0009] Furthermore, when determining the pruning step size for progressively pruning the decision confidence interval based on the interval length, the process includes: The length of the interval is compared with the length of the first interval and the length of the second interval, and the cutting step size is determined based on the comparison result; wherein the length of the first interval is less than the length of the second interval. When the length of the interval is less than or equal to the length of the first interval, the cutting step size is determined to be the first cutting step size; When the length of the interval is greater than the length of the first interval and less than or equal to the length of the second interval, the cutting step size is determined to be the second cutting step size. When the length of the interval is greater than the length of the second interval, the cutting step size is determined to be the third cutting step size.

[0010] Furthermore, optimizing the cutting step size based on the type and strictness to obtain the optimized cutting step size includes: The type is compared with the basic optimization coefficient mapping table, and the basic optimization coefficient of the trimming step size is determined based on the comparison result. The core optimization coefficients for the trimming step size are obtained based on the aforementioned stringency. The basic optimization coefficients and the core optimization coefficients are weighted and calculated to obtain the comprehensive optimization coefficients; The optimized cutting step size is obtained by multiplying the cutting step size by the comprehensive optimization coefficient.

[0011] Furthermore, when detecting the validity of the decision confidence interval during the pruning process, it includes: After each pruning of the decision confidence interval, obtain the interval state information of the pruned decision confidence interval; Based on the interval state information, an integrity check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval is an empty interval or an invalid interval. Based on the interval state information, a consistency check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval satisfies the operational constraints. When both the integrity and consistency checks pass, the decision confidence interval after pruning is determined to be valid. If the integrity check or consistency check fails, the pruned decision confidence interval is deemed invalid.

[0012] Furthermore, when the pruned decision confidence interval satisfies the preset convergence condition, determining the final decision result from the pruned decision confidence interval includes: Obtain the target decision confidence interval that satisfies the preset convergence condition; Based on the interval feature information of the target decision confidence interval, a set of candidate decision results is determined; The validity of the candidate decision result set is determined, and candidate decision results that do not meet the operation constraints or the constraint conditions are eliminated; From the candidate decision results that have passed the initial assessment, the final decision result is determined according to the preset decision selection rules; Output the final decision result.

[0013] Furthermore, when the pruned decision confidence interval does not satisfy the convergence condition, adjusting the constraint condition or expanding and then pruning the pruned decision confidence interval includes: Based on the decision confidence interval state information obtained during the pruning process, the pruned decision confidence interval is subjected to convergence analysis according to a preset judgment rule. The judgment rule includes one or more of the interval length change, interval reduction ratio, or interval overlap ratio. When it is determined that the reduction ratio of the decision confidence interval after pruning exceeds a preset threshold due to the superposition of the constraints, or the proportion of feasible intervals is lower than a preset lower limit, it is determined to adjust the constraints. The adjustment includes relaxing some modal constraint rules or reducing the restriction intensity of the corresponding modal constraint parameters. When it is determined that the length of the pruned decision confidence interval is less than the minimum effective interval threshold, or the overlap ratio with the historical effective decision interval is less than the preset threshold, the pruned decision confidence interval is expanded to generate an expanded decision confidence interval. After the constraint adjustment or decision confidence interval expansion is completed, the stepwise pruning of the decision confidence interval is re-executed based on the updated constraint or expanded decision confidence interval.

[0014] Compared with existing technologies, the advantages of this invention are as follows: This invention constructs an initial decision confidence interval based on the operational constraints and historical state information of the decision object through a construction module, providing a scientifically sound and reasonable range basis for decision-making. The constraint module accurately maps multi-source heterogeneous multimodal information into constraints acting on this interval, achieving effective definition of the decision space. The pruning module performs refined pruning of the decision confidence interval through dynamically adjusted optimized pruning step sizes and simultaneously performs validity checks, ensuring the rigor and reliability of the pruning process. The decision module flexibly adopts strategies such as determining the final decision result or adjusting constraints, expanding the interval, and re-pruning based on the convergence of the pruning results, effectively improving the robustness of the decision. The update module not only outputs the decision result but also dynamically updates the decision confidence interval when new information arrives, enabling the system to continuously learn and adapt to dynamic environments. The entire system, by transforming multimodal information into progressive constraints and pruning of the decision confidence interval, effectively integrates the advantages of information from different sources, reduces the uncertainty and error impact of a single information source, and thus significantly improves the accuracy and reliability of the decision results, better meeting the needs of high-precision and high-reliability decision-making in practical applications.

[0015] In another aspect, this invention also proposes an intelligent decision-making method based on multimodal information fusion, comprising the following steps: Construct a decision confidence interval based on the operational constraints and historical state information of the decision object; Collect multimodal information related to the decision object, and map different modal information into constraints acting on the decision confidence interval respectively; The decision confidence interval is progressively pruned according to the constraints, and the validity of the decision confidence interval is detected during the pruning process. When the pruned decision confidence interval meets the preset convergence condition, the final decision result is determined from the pruned decision confidence interval; When the pruned decision confidence interval does not meet the convergence condition, the constraint condition is adjusted or the pruned decision confidence interval is expanded and then pruned again. The final decision result is output, and the decision confidence interval is dynamically updated when new multimodal information arrives.

[0016] It is understandable that the aforementioned intelligent decision-making methods and systems based on multimodal information fusion have the same beneficial effects, and will not be elaborated further here. Attached Figure Description

[0017] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. Furthermore, the same reference numerals denote the same parts throughout the drawings. In the drawings: Figure 1 This is a structural block diagram of an intelligent decision-making system for multimodal information fusion provided in an embodiment of the present invention; Figure 2 A flowchart of an intelligent decision-making method for multimodal information fusion provided in an embodiment of the present invention. Detailed Implementation

[0018] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided to enable a more thorough understanding of the present disclosure and to fully convey the scope of the disclosure to those skilled in the art. It should be noted that, unless otherwise specified, embodiments and features in the embodiments of the present invention can be combined with each other. The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0019] See Figure 1 As shown in some embodiments of this application, this embodiment provides an intelligent decision-making system based on multimodal information fusion, including: The construction module is used to construct the decision confidence interval based on the operational constraints and historical state information of the decision object; The constraint module is used to collect multimodal information related to the decision object and map different modal information into constraints that act on the decision confidence interval. The pruning module is used to progressively prune the decision confidence interval according to the constraints, and to detect the validity of the decision confidence interval during the pruning process; The decision module is used to determine the final decision result from the trimmed decision confidence interval when the trimmed decision confidence interval meets the preset convergence condition; It is also used to adjust the constraint conditions or expand and re-trim the pruned decision confidence interval when the pruned decision confidence interval does not meet the convergence condition. The update module is used to output the final decision result and dynamically update the decision confidence interval when new multimodal information arrives.

[0020] It is understood that the multimodal information fusion intelligent decision-making system provided in this embodiment constructs an initial decision credibility interval based on the operational constraints and historical state information of the decision object through a construction module, providing a scientific and reasonable range basis for decision-making. The constraint module accurately maps multi-source heterogeneous multimodal information into constraints acting on this interval, achieving effective definition of the decision space. The pruning module performs refined pruning of the decision credibility interval through dynamically adjusted optimized pruning step sizes, and simultaneously performs validity checks, ensuring the rigor and reliability of the pruning process. The decision module flexibly adopts strategies such as determining the final decision result or adjusting constraints, expanding the interval, and re-pruning based on the convergence of the pruning results, effectively improving the robustness of the decision. The update module not only outputs the decision result but also dynamically updates the decision credibility interval when new information arrives, enabling the system to continuously learn and adapt to dynamic environments. The entire system effectively integrates the advantages of information from different sources by transforming multimodal information into progressive constraints and pruning of the decision confidence interval. This reduces the uncertainty and error impact of a single information source, thereby significantly improving the accuracy and reliability of decision results and better meeting the needs of high-precision and high-reliability decision-making in practical applications.

[0021] Specifically, when constructing the decision confidence interval based on the operational constraints and historical state information of the decision object, the following should be included: The operational constraints are parsed to obtain the operational boundary parameters; Within the range of values ​​defined by the operating boundary parameters, the historical state information of the decision object is filtered to determine the historical effective decision interval; The historical effective decision intervals are processed by interval synthesis, and the intervals that exceed the range of the operating boundary parameters are removed to form candidate decision intervals; The candidate decision interval is subjected to integrity and consistency verification. If the verification passes, the candidate decision interval is determined as the decision confidence interval.

[0022] In this embodiment, the decision-making object is preferably a complex equipment or system in an industrial production process, such as a production line control unit in a smart factory, a reaction process system in a large chemical plant, or a regional dispatch center of a smart grid. These decision-making objects typically exhibit multivariate coupling, strong dynamic characteristics, complex operating environments, and high requirements for decision-making accuracy and real-time performance. Their operating state is comprehensively influenced by multiple modal information, including process parameters, equipment status, environmental factors, and market demand.

[0023] It is understandable that, taking the production line control unit of a smart factory as an example, its operational constraints include equipment operating parameters. Parsing the operational constraints yields the operational boundary parameters: Pmin=20, Pmax=90. Filtering historical state information within the range (20, 90) yields the following historical effective decision intervals: First interval: (25, 40); Second interval: (35, 60); Third interval: (50, 85). These historical effective decision intervals are then synthesized to obtain the synthesized interval (25, 85). After removing the intervals exceeding the operational boundary parameters, a candidate decision interval (25, 85) is formed. The candidate decision interval length is verified to be 60, which is greater than the minimum effective interval length threshold of 5; simultaneously, the candidate decision interval falls entirely within the (20, 90) operational boundary; therefore, (25, 85) is determined as the reliable decision interval.

[0024] Specifically, mapping different modal information to constraints acting on the decision confidence interval includes: The collected modal information is classified and processed to distinguish modal information from different sources and types; Feature extraction is performed on all the modal information respectively to obtain modal feature parameters associated with the running state of the decision object; Based on the modal feature parameters, generate corresponding modal constraint rules or modal constraint parameters; The modal constraint rules or modal constraint parameters are applied to the decision confidence interval to form constraints that limit the range of values ​​in the decision confidence interval.

[0025] In this embodiment, taking the production line control unit of a smart factory as an example, the modal classification includes Modal 1: sensor load data; Modal 2: historical fault statistics data; Modal 3: ambient temperature data. Feature parameter extraction (numerical): Modal 1 feature parameter: current load average = 0.75; Modal 2 feature parameter: fault frequency = 0.12; Modal 3 feature parameter: ambient temperature = 42℃; Generate modal constraint parameters: Based on the load average of 0.75, generate constraint: P≤75; Based on the fault frequency of 0.12, generate constraint: P≥30; Based on the ambient temperature of 42℃, generate constraint: P≤70; Forming constraint conditions: After the above constraints work together, the restriction result on the decision confidence interval (25, 85) is (30, 70).

[0026] Understandably, this mapping method transforms different modal information into specific numerical constraints on the decision confidence interval, making the originally abstract multimodal information quantifiable and operable, thereby achieving precise definition of the decision space. For example, sensor load data reflects the current load capacity of the equipment. When the average load reaches 0.75, to avoid overload risk, a constraint of "P≤75" is generated, directly limiting the upper limit of the decision variable. Historical fault statistics provide a reference from the perspective of the probability of fault occurrence. A fault frequency of 0.12 means that the fault risk is significantly reduced at parameter values ​​below 30, thus generating a lower limit constraint of "P≥30". An ambient temperature of 42℃ is a relatively high temperature, which may affect the stability of equipment performance, further tightening the upper limit constraint to "P≤70". These three constraints from different modalities work together on the initial decision confidence interval (25, 85), and through intersection operation, (30, 70) is obtained, achieving an initial effective reduction of the decision range and laying the foundation for subsequent pruning steps. This multimodal information mapping process fully considers the inherent relationship between various types of information and the operational state of the decision-making object, ensuring the pertinence and effectiveness of the constraints, and enabling each type of information to play its unique value in the decision-making process.

[0027] In this embodiment, modal constraint rules refer to conditional rules formulated based on the logical correlation or empirical rules between modal feature parameters and decision confidence intervals, used to restrict decision confidence intervals.

[0028] Specifically, when progressively pruning the decision confidence interval according to the constraints, the process includes: Collect the length of the decision confidence interval, the type of the decision object, and the strictness of the constraints; The pruning step size for progressively pruning the decision confidence interval is determined based on the interval length. The cutting step size is optimized based on the type and severity level to obtain an optimized cutting step size; The decision confidence interval is then gradually trimmed.

[0029] Specifically, when determining the pruning step size for progressively pruning the decision confidence interval based on the interval length, the process includes: The length of the interval is compared with the length of the first interval and the length of the second interval, and the cutting step size is determined based on the comparison result; wherein the length of the first interval is less than the length of the second interval. When the length of the interval is less than or equal to the length of the first interval, the cutting step size is determined to be the first cutting step size; When the length of the interval is greater than the length of the first interval and less than or equal to the length of the second interval, the cutting step size is determined to be the second cutting step size. When the length of the interval is greater than the length of the second interval, the cutting step size is determined to be the third cutting step size.

[0030] Understandably, the preferred length of the first interval is 10, and the preferred length of the second interval is 20; the first trimming step size is 1, the second trimming step size is 3, and the third trimming step size is 5. For example, if the current decision confidence interval length is 8 (less than the first interval length of 10), then the trimming step size is determined to be 1; if the interval length is 15 (greater than the first interval length of 10 and less than or equal to the second interval length of 20), then the trimming step size is 3; if the interval length is 25 (greater than the second interval length of 20), then the trimming step size is 5. This setting allows for flexible adjustment of the trimming precision according to different interval lengths. When the interval is short, a small step size is used to avoid over-trimming and loss of effective information; when the interval is long, a large step size is used to improve trimming efficiency, balancing the accuracy and speed of trimming.

[0031] Specifically, optimizing the cutting step size based on the type and severity level to obtain the optimized cutting step size includes: The type is compared with the basic optimization coefficient mapping table, and the basic optimization coefficient of the trimming step size is determined based on the comparison result. The core optimization coefficients for the trimming step size are obtained based on the aforementioned stringency. The basic optimization coefficients and the core optimization coefficients are weighted and calculated to obtain the comprehensive optimization coefficients; The optimized cutting step size is obtained by multiplying the cutting step size by the comprehensive optimization coefficient.

[0032] It is understood that in this embodiment, the basic optimization coefficient mapping table can be pre-set according to the risk level and control precision requirements of the industry sector to which the decision-making object belongs. For example, for a high-risk, high-precision chemical reaction process system (type), the corresponding basic optimization coefficient is 0.8; for a medium-risk smart grid regional dispatch center, the basic optimization coefficient is 1.0; and for a relatively low-risk ordinary manufacturing production line control unit, the basic optimization coefficient is 1.2. The core optimization coefficient is divided into multiple levels according to the strictness of the constraints. For example, the core optimization coefficient is 0.7 when the strictness is extremely high, 0.9 when the strictness is high, 1.0 when the strictness is medium, 1.1 when the strictness is low, and 1.3 when the strictness is extremely low. Assuming that the current decision-making object is a chemical reaction process system (basic optimization coefficient 0.8), and the current multimodal information contains early warning information of key safety indicators exceeding the standard, the constraint is determined to be extremely strict (core optimization coefficient 0.7). The weights of the basic optimization coefficient and the core optimization coefficient are set to 0.4 and 0.6 respectively, then the comprehensive optimization coefficient = 0.74. If the trimming step size previously determined based on the interval length is 5, then the optimized trimming step size = 5 × 0.74 = 3.7. In practical applications, it can be rounded down to 4 as needed. Through this weighted calculation, the inherent attributes of the decision-making object and the urgency of the current constraints can be comprehensively considered in the optimization of the trimming step size, making the trimming process more in line with the needs of the actual decision-making scenario. For example, in high-risk and high-strictness situations, a more cautious and refined trimming can be performed by reducing the trimming step size, avoiding the risk of decision-making errors that may be caused by large step size trimming.

[0033] Specifically, when detecting the validity of the decision confidence interval during the pruning process, it includes: After each pruning of the decision confidence interval, obtain the interval state information of the pruned decision confidence interval; Based on the interval state information, an integrity check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval is an empty interval or an invalid interval. Based on the interval state information, a consistency check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval satisfies the operational constraints. When both the integrity and consistency checks pass, the decision confidence interval after pruning is determined to be valid. If the integrity check or consistency check fails, the pruned decision confidence interval is deemed invalid.

[0034] Understandably, taking the production line control unit of a smart factory as an example, during the trimming process, assuming that after one trimming operation, the decision confidence interval is trimmed from (30, 70) to (30, 65), the state information of the trimmed interval is obtained, including the upper and lower limits of the interval and whether it meets the preset integrity and consistency standards. For integrity checking, it checks whether (30, 65) is an empty interval. Obviously, the lower limit of 30 is less than the upper limit of 65, so it is not an empty interval, and there is no invalid situation where the interval boundary value is abnormal due to trimming. Therefore, the integrity check passes. The consistency check determines whether the trimmed interval meets the initial operating constraints, that is, whether it completely falls within the operating boundary of (20, 90). Since 30 is greater than 20 and 65 is less than 90, it fully meets the requirements of the operating constraints, and the consistency check also passes. Therefore, (30, 65) is determined to be a valid interval, and subsequent trimming steps can continue. If, after a pruning step, the interval becomes (75, 70), the lower limit is greater than the upper limit, indicating an empty interval. The integrity check fails, and the pruning result is invalid, requiring backtracking and adjustment of the pruning strategy. Alternatively, if the pruned interval is (15, 60), where the lower limit 15 is less than the running boundary Pmin=20, it does not meet the consistency requirement and is also deemed an invalid interval, requiring re-evaluation of the pruning process. This validity check allows for the timely identification and elimination of unreasonable pruning results, ensuring that the decision confidence interval always remains within a valid and constrained range, thus guaranteeing the accuracy of the final decision.

[0035] Specifically, when the pruned decision confidence interval meets the preset convergence condition, determining the final decision result from the pruned decision confidence interval includes: Obtain the target decision confidence interval that satisfies the preset convergence condition; Based on the interval feature information of the target decision confidence interval, a set of candidate decision results is determined; The validity of the candidate decision result set is determined, and candidate decision results that do not meet the operation constraints or the constraint conditions are eliminated; From the candidate decision results that have passed the initial assessment, the final decision result is determined according to the preset decision selection rules; Output the final decision result.

[0036] Understandably, the preset convergence condition can be set as the length of the target decision confidence interval being less than or equal to a preset threshold (e.g., 5) and the interval length change rate being less than 10% after three consecutive trimmings. Taking the intelligent factory production line control unit as an example, after multiple rounds of trimming, the target decision confidence interval converges to (45, 49), with an interval length of 4, satisfying the preset threshold of 5. The interval length changes of the three most recent trimmings are from (42, 50) (length 8) to (44, 49) (length 5) and then to (45, 49) (length 4), with change rates of 37.5%, 20%, and 20% respectively. Although it does not fully satisfy the requirement of a change rate of less than 10% for three consecutive trimmings, considering that the interval length has been significantly reduced and is close to stable, it can be flexibly determined to meet the convergence condition in combination with the actual scenario. Subsequently, the interval feature information of the target interval is obtained, including the upper and lower limits of the interval (45, 49), the midpoint of the interval (47), and the historical decision success rate corresponding to each point in the interval. Based on these characteristics, a set of candidate decision results is determined. For example, the midpoint of the interval (47), the historically highest success rate (46) within the interval, and the upper and lower limits (45 and 49) are selected as candidate decision results, forming the set {45, 46, 47, 49}. Next, the validity of the candidate decision result set is determined, checking whether each candidate value satisfies all operational constraints and conditions. Assuming the operational constraint is P∈(20, 90), and the condition is P≥30 and P≤70, all candidate values ​​clearly satisfy these constraints. Furthermore, if there are additional process requirements for specific parameter values, such as 45 corresponding to a higher raw material loss rate and 49 corresponding to higher energy consumption, then 45 and 49 can be eliminated, leaving {46, 47}. Finally, the final decision result is determined according to the preset decision selection rules. If the rule is "select the parameter value with the highest historical success rate and the lowest energy consumption", after querying historical data, the historical success rate of 46 is 92% and the energy consumption index is 1.1, and the historical success rate of 47 is 90% and the energy consumption index is 1.05. After comprehensive consideration, 47 is selected as the final decision result and output to the production line control unit to achieve precise control of production parameters.

[0037] Specifically, when the pruned decision confidence interval does not meet the convergence condition, adjusting the constraint condition or expanding and then pruning the pruned decision confidence interval includes: Based on the decision confidence interval state information obtained during the pruning process, the pruned decision confidence interval is subjected to convergence analysis according to a preset judgment rule. The judgment rule includes one or more of the interval length change, interval reduction ratio, or interval overlap ratio. When it is determined that the reduction ratio of the decision confidence interval after pruning exceeds a preset threshold due to the superposition of the constraints, or the proportion of feasible intervals is lower than a preset lower limit, it is determined to adjust the constraints. The adjustment includes relaxing some modal constraint rules or reducing the restriction intensity of the corresponding modal constraint parameters. When it is determined that the length of the pruned decision confidence interval is less than the minimum effective interval threshold, or the overlap ratio with the historical effective decision interval is less than the preset threshold, the pruned decision confidence interval is expanded to generate an expanded decision confidence interval. After the constraint adjustment or decision confidence interval expansion is completed, the stepwise pruning of the decision confidence interval is re-executed based on the updated constraint or expanded decision confidence interval.

[0038] Understandably, taking the decision credibility interval pruning of an urban traffic signal control system as an example, suppose that during the morning rush hour, the decision credibility interval constructed by the system using multimodal information (such as real-time traffic flow, pedestrian density, weather conditions, etc.) still fails to meet the convergence condition after multiple rounds of pruning, with the interval length remaining at 15 (the preset convergence threshold is 10), and the interval reduction ratio of five consecutive prunings being less than 5%. In this case, a convergence analysis based on the interval state information reveals that the current constraints simultaneously include multiple strict rules such as "priority passage for vehicles on main roads," "pedestrian crossing safety time guarantee," and "priority dispatching for buses," leading to excessive compression of the decision credibility interval, with the feasible interval accounting for only 20% of the initial interval (below the preset lower limit of 30%). Therefore, it is determined that the constraints need to be adjusted. For example, the constraint strength of "priority dispatching for buses" can be temporarily relaxed, increasing the maximum waiting time for buses to pass through intersections from 30 seconds to 45 seconds, or the weight of bus priority levels can be reduced during non-bus lane periods. If the analysis finds that the length of the pruned decision confidence interval is 8 (less than the minimum effective interval threshold of 10), and the overlap ratio with the effective decision interval of the same period in the morning peak is only 15% (less than the preset threshold of 25%), then it is determined that the interval needs to be expanded. During expansion, historical data can be used to expand the current interval (50, 58) to both sides. For example, based on the fluctuation range of the decision interval for the same type of morning peak in the past month, it can be expanded to (45, 63), restoring the interval length to 18 and ensuring that the overlap ratio with the historical effective interval is increased to 35%. After completing the constraint adjustment or interval expansion, the system, based on the adjusted constraints or expanded interval, performs the step-by-step pruning operation again according to the aforementioned pruning step size optimization and effectiveness detection process until the decision confidence interval meets the convergence condition. This avoids decision stagnation caused by excessive constraints or excessively small intervals, ensuring the system's continuous decision-making capability in complex dynamic environments.

[0039] See Figure 2 As shown in some embodiments of this application, this embodiment provides an intelligent decision-making method based on multimodal information fusion, including the following steps: S100: Construct a decision confidence interval based on the operational constraints and historical state information of the decision object; S200: Collect multimodal information related to the decision object, and map different modal information into constraints acting on the decision confidence interval respectively; S300: The decision confidence interval is gradually pruned according to the constraints, and the validity of the decision confidence interval is detected during the pruning process; S400: When the pruned decision confidence interval meets the preset convergence condition, the final decision result is determined from the pruned decision confidence interval; S500: When the pruned decision confidence interval does not meet the convergence condition, adjust the constraint condition or expand the pruned decision confidence interval and then prune it again. S600: Output the final decision result and dynamically update the decision confidence interval when new multimodal information arrives.

[0040] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program goods. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program goods on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0041] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program goods according to embodiments of this application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart... Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0042] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0043] These computer program instructions can also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0044] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and not to limit it. Although the present invention has been described in detail with reference to the above embodiments, those skilled in the art should understand that modifications or equivalent substitutions can still be made to the specific implementation of the present invention. Any modifications or equivalent substitutions that do not depart from the spirit and scope of the present invention should be covered within the scope of protection of the claims of the present invention.

Claims

1. An intelligent decision-making system based on multimodal information fusion, characterized in that, include: The construction module is used to construct the decision confidence interval based on the operational constraints and historical state information of the decision object; The constraint module is used to collect multimodal information related to the decision object and map different modal information into constraints that act on the decision confidence interval. The pruning module is used to progressively prune the decision confidence interval according to the constraints, and to detect the validity of the decision confidence interval during the pruning process; The decision module is used to determine the final decision result from the trimmed decision confidence interval when the trimmed decision confidence interval meets the preset convergence condition; It is also used to adjust the constraint conditions or expand and re-trim the pruned decision confidence interval when the pruned decision confidence interval does not meet the convergence condition. The update module is used to output the final decision result and dynamically update the decision confidence interval when new multimodal information arrives.

2. The intelligent decision-making system based on multimodal information fusion according to claim 1, characterized in that, When constructing the decision confidence interval based on the operational constraints and historical state information of the decision object, the following should be included: The operational constraints are parsed to obtain the operational boundary parameters; Within the range of values ​​defined by the operating boundary parameters, the historical state information of the decision object is filtered to determine the historical effective decision interval; The historical effective decision intervals are processed by interval synthesis, and the intervals that exceed the range of the operating boundary parameters are removed to form candidate decision intervals; The candidate decision interval is subjected to integrity and consistency verification. If the verification passes, the candidate decision interval is determined as the decision confidence interval.

3. The intelligent decision-making system based on multimodal information fusion according to claim 2, characterized in that, Mapping different modal information to constraints acting on the decision confidence interval includes: The collected modal information is classified and processed to distinguish modal information from different sources and types; Feature extraction is performed on all the modal information respectively to obtain modal feature parameters associated with the running state of the decision object; Based on the modal feature parameters, generate corresponding modal constraint rules or modal constraint parameters; The modal constraint rules or modal constraint parameters are applied to the decision confidence interval to form constraints that limit the range of values ​​in the decision confidence interval.

4. The intelligent decision-making system based on multimodal information fusion according to claim 3, characterized in that, When progressively pruning the decision confidence interval according to the constraints, the following steps are included: Collect the length of the decision confidence interval, the type of the decision object, and the strictness of the constraints; The pruning step size for progressively pruning the decision confidence interval is determined based on the interval length. The cutting step size is optimized based on the type and severity level to obtain an optimized cutting step size; The decision confidence interval is then gradually trimmed.

5. The intelligent decision-making system based on multimodal information fusion according to claim 4, characterized in that, When determining the pruning step size for progressively pruning the decision confidence interval based on the interval length, the following is included: The length of the interval is compared with the length of the first interval and the length of the second interval, and the cutting step size is determined based on the comparison result; wherein the length of the first interval is less than the length of the second interval. When the length of the interval is less than or equal to the length of the first interval, the cutting step size is determined to be the first cutting step size; When the length of the interval is greater than the length of the first interval and less than or equal to the length of the second interval, the cutting step size is determined to be the second cutting step size. When the length of the interval is greater than the length of the second interval, the cutting step size is determined to be the third cutting step size.

6. The intelligent decision-making system based on multimodal information fusion according to claim 5, characterized in that, Optimizing the cutting step size based on the type and severity level to obtain the optimized cutting step size includes: The type is compared with the basic optimization coefficient mapping table, and the basic optimization coefficient of the trimming step size is determined based on the comparison result. The core optimization coefficients for the trimming step size are obtained based on the aforementioned stringency. The basic optimization coefficients and the core optimization coefficients are weighted and calculated to obtain the comprehensive optimization coefficients; The optimized cutting step size is obtained by multiplying the cutting step size by the comprehensive optimization coefficient.

7. The intelligent decision-making system based on multimodal information fusion according to claim 6, characterized in that, When detecting the validity of the decision confidence interval during the pruning process, the following are included: After each pruning of the decision confidence interval, obtain the interval state information of the pruned decision confidence interval; Based on the interval state information, an integrity check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval is an empty interval or an invalid interval. Based on the interval state information, a consistency check is performed on the pruned decision confidence interval to determine whether the pruned decision confidence interval satisfies the operational constraints. When both the integrity and consistency checks pass, the decision confidence interval after pruning is determined to be valid. If the integrity check or consistency check fails, the pruned decision confidence interval is deemed invalid.

8. The intelligent decision-making system based on multimodal information fusion according to claim 7, characterized in that, When the trimmed decision confidence interval meets the preset convergence condition, the final decision result is determined from the trimmed decision confidence interval, including: Obtain the target decision confidence interval that satisfies the preset convergence condition; Based on the interval feature information of the target decision confidence interval, a set of candidate decision results is determined; The validity of the candidate decision result set is determined, and candidate decision results that do not meet the operation constraints or the constraint conditions are eliminated; From the candidate decision results that have passed the initial assessment, the final decision result is determined according to the preset decision selection rules; Output the final decision result.

9. The intelligent decision-making system based on multimodal information fusion according to claim 8, characterized in that, When the pruned decision confidence interval does not meet the convergence condition, adjusting the constraint condition or expanding and then pruning the pruned decision confidence interval includes: Based on the decision confidence interval state information obtained during the pruning process, the pruned decision confidence interval is subjected to convergence analysis according to a preset judgment rule. The judgment rule includes one or more of the interval length change, interval reduction ratio, or interval overlap ratio. When it is determined that the reduction ratio of the decision confidence interval after pruning exceeds a preset threshold due to the superposition of the constraints, or the proportion of feasible intervals is lower than a preset lower limit, it is determined to adjust the constraints. The adjustment includes relaxing some modal constraint rules or reducing the restriction intensity of the corresponding modal constraint parameters. When it is determined that the length of the pruned decision confidence interval is less than the minimum effective interval threshold, or the overlap ratio with the historical effective decision interval is less than the preset threshold, the pruned decision confidence interval is expanded to generate an expanded decision confidence interval. After the constraint adjustment or decision confidence interval expansion is completed, the stepwise pruning of the decision confidence interval is re-executed based on the updated constraint or expanded decision confidence interval.

10. A multimodal information fusion intelligent decision-making method, applied to the multimodal information fusion intelligent decision-making system as described in any one of claims 1-9, characterized in that, include: Construct a decision confidence interval based on the operational constraints and historical state information of the decision object; Collect multimodal information related to the decision object, and map different modal information into constraints acting on the decision confidence interval respectively; The decision confidence interval is progressively pruned according to the constraints, and the validity of the decision confidence interval is detected during the pruning process. When the pruned decision confidence interval meets the preset convergence condition, the final decision result is determined from the pruned decision confidence interval; When the pruned decision confidence interval does not meet the convergence condition, the constraint condition is adjusted or the pruned decision confidence interval is expanded and then pruned again. The final decision result is output, and the decision confidence interval is dynamically updated when new multimodal information arrives.

Images