A quality information management method and system based on multimodal AI

Abstract

This invention provides a quality information management method and system based on multimodal AI, belonging to the field of information management technology. It includes: performing multimodal detection on the set targets of each work step according to the multimodal detection method in the work process; performing data analysis on the multimodal data of each work step to evaluate the quality level of the set targets under the corresponding work step; when the quality level meets the set standard of the corresponding work step, storing the multimodal data of the corresponding work step in database one; otherwise, determining the storage address of the multimodal data of the corresponding work step in database two and managing its storage based on the difference between the quality level of the corresponding work step and the set standard, combined with the comprehensive level of the target item determined based on all quality levels. This facilitates the storage management of qualified and unqualified quality data, facilitates subsequent retrieval and use, and improves the efficiency of problem troubleshooting.

Description

Technical Field

[0001] This invention relates to the field of information management technology, and in particular to a quality information management method and system based on multimodal AI. Background Art

[0002] In the robot manufacturing process, quality information management is crucial for ensuring product quality and improving production efficiency. With the rapid development of robot manufacturing technology, the manufacturing process is becoming increasingly complex, encompassing multiple operational steps such as parts processing, assembly, and debugging, and each step has a critical impact on the final product quality.

[0003] Because there is a lot of information and data in the robot manufacturing process, but the current common practice is to use a unified storage strategy, the multimodal data generated is stored in the same location regardless of the quality level of the operation step. When quality problems occur and backtracking analysis is needed, it is difficult for staff to quickly locate the key data, resulting in low efficiency in problem investigation and seriously affecting production progress and quality improvement processes.

[0004] Therefore, this invention proposes a quality information management method and system based on multimodal AI. Summary of the Invention

[0005] This invention provides a quality information management method and system based on multimodal AI to solve the aforementioned technical problems.

[0006] This invention provides a quality information management method based on multimodal AI, comprising:

[0007] Step 1: Determine the work process of the target project. Based on the multimodal detection method under each work step in the work process, perform multimodal detection on the set target of the corresponding work step to obtain multimodal data.

[0008] Step 2: Perform data analysis on the multimodal data for each work step to evaluate the quality level of the set target under the corresponding work step;

[0009] Step 3: When the quality level meets the set standard of the corresponding work step, store the multimodal data under the corresponding work step into database one;

[0010] Step 4: When the quality level does not meet the set standard of the corresponding operation step, determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management based on the difference between the quality level of the corresponding operation step and the set standard, and in combination with the comprehensive level of the target project determined based on all quality levels.

[0011] Preferably, the data analysis process for multimodal data under each operation step includes:

[0012] Fill each modal data into the modal blank table that matches the corresponding operation step, determine whether there are missing objects in the filled table, and if so, lock the missing location point of the missing object;

[0013] Based on the format of the filled table, obtain the first reference value of the first non-missing point closest to the missing point, the missing point closest to the missing point, and the second reference value of the second non-missing point closest to the missing point;

[0014] If the number of the nearest missing points is 0, then the missing data at the missing location points is filled in for the first time based on the first reference value;

[0015] If the number of the most recent missing points is 1, then the data of the missing points is filled in a second time based on the first reference value and the second reference value;

[0016] If the number of the most recent missing points is 2, then the data of the missing points is filled in a third time based on the first reference value and two second reference values;

[0017] The complete data for the corresponding modality is obtained based on the completion results.

[0018] Preferably, the data at the missing location points is supplemented a third time based on the first reference value and two second reference values, including:

[0019] ;

[0020] in, These are the first reference value, one second reference value, and another second reference value, respectively. These represent values ​​based on the first reference value. slope of neighboring points of corresponding point Based on a second reference value slope of neighboring points of corresponding point The slope of the neighboring points corresponding to another second reference value C3 ; Indicates the first reference value Corresponding point and a second reference value The slope between corresponding points; Indicates the first reference value Corresponding point and another second reference value The slope between corresponding points; Represents a symbolic function; This represents the total number of non-missing points in the filled table. This indicates the third padding value.

[0021] Preferably, the evaluation of the quality level of the set target under the corresponding work steps includes:

[0022] Convert the data analysis results for each modality into corresponding feature vectors;

[0023] Based on the characteristics and requirements of the set objectives, multiple quality assessment indicators related to each work step are determined, and corresponding weights are assigned to the quality assessment indicators under each work step. The quality assessment indicators include: installation accuracy, operational standardization, and operational stability.

[0024] The feature vectors are matched with each quality assessment index to construct a multimodal result matrix. ,in, 1 represents the installation accuracy, operational standardization, and operational stability under the first operation step, respectively; These represent the installation accuracy, operational standardization, and operational stability at the nth operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability in the first operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability at the nth operation step, respectively.

[0025] Extract the oblique vector of the multimodal result matrix J, and map each element in the oblique vector to the level lookup table to obtain the initial level of the corresponding element;

[0026] The amount of change is determined based on the connection points of the aforementioned work steps;

[0027] The initial grade is adjusted based on the amount of change to obtain the quality grade.

[0028] Preferably, the amount of change is determined based on the connection point of the work steps, including:

[0029] When the operation step is the first step, the initial level is regarded as the quality level;

[0030] When the operation step is not the first step, obtain the impact of the complete quality of the previous step on the corresponding operation step;

[0031]

[0032] in, This represents the influence coefficient on the i-th operation step; This indicates the sensitivity of the i-th step to the influence of the (i-1)-th step at the connection position; This indicates the degree of correlation between the i-th step and the (i-1)-th step; n represents the total number of steps in the operation. This represents the directional judgment coefficient for the i-th task step. When it is a positive direction, The value is 1, which indicates a negative direction. The value is -1;

[0033] The adjustment level is determined based on the influence coefficient and the preset unit influence, and the initial level is adjusted to obtain the quality level.

[0034] Preferably, the quality grade is obtained by adjusting the initial grade, including:

[0035]

[0036]

[0037] Where ZD represents quality level; CD represents initial level; TD represents adjustment level; This indicates the floor function.

[0038] Preferably, determining the storage address of the multimodal data for the corresponding work step in database two includes:

[0039] The overall grade is obtained by combining the step weight of each task step with the quality level of each task step, and the overall difference between the overall grade and the overall standard is obtained.

[0040] Based on the quality level and set standards, the difference array Cs= is obtained. ,in, This represents the difference in level between the i-th task steps;

[0041] Calculate the symbol combination {E1, E2} for each remaining task step (excluding the nth task step) and the next task step, where E1 is the symbol for the corresponding remaining task step and E2 is the symbol for the next task step.

[0042] If the corresponding remaining steps do not meet the set criteria, then the storage address in database two is obtained by matching the step-symbol combination-double difference-address lookup table based on the step weight, symbol combination, level difference, and comprehensive difference of the corresponding remaining steps.

[0043] This invention provides a quality information management system based on multimodal AI, comprising:

[0044] The data acquisition module is used to determine the work process of the target project, and to perform multimodal detection on the set target of the corresponding work step according to the multimodal detection method under each work step in the work process to obtain multimodal data.

[0045] The rating assessment module is used to perform data analysis on multimodal data under each work step and to assess the quality level of the set target under the corresponding work step.

[0046] The standard storage module is used to store the multimodal data of the corresponding work step into database one when the quality level meets the set standard of the corresponding work step.

[0047] The non-standard storage module is used to determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management when the quality level does not meet the set standard of the corresponding operation step. This is based on the difference between the quality level of the corresponding operation step and the set standard, and combined with the comprehensive level of the target project determined based on all quality levels.

[0048] Preferably, a storage medium includes: a memory for storing the quality information management method based on multimodal AI.

[0049] Preferably, a computer device includes a processor for executing the aforementioned quality information management method based on multimodal AI.

[0050] Compared with the prior art, the beneficial effects of this application are as follows:

[0051] On the one hand, it facilitates quality management during the project process; on the other hand, it facilitates the storage and management of qualified and unqualified quality data, thereby facilitating subsequent retrieval and use, and improving the efficiency of problem investigation. Attached Figure Description

[0052] To more clearly illustrate the technical solutions in this invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0053] Figure 1 This is a flowchart of a quality information management method based on multimodal AI provided in an embodiment of the present invention;

[0054] Figure 2 This is a structural diagram of a quality information management system based on multimodal AI provided in an embodiment of the present invention. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of this invention clearer, the technical solutions of this invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of this invention. All other embodiments obtained by those skilled in the art based on the embodiments of this invention without creative effort are within the scope of protection of this invention.

[0056] This invention provides a quality information management method based on multimodal AI, such as... Figure 1 As shown, it includes:

[0057] Step 1: Determine the work process of the target project. Based on the multimodal detection method under each work step in the work process, perform multimodal detection on the set target of the corresponding work step to obtain multimodal data.

[0058] Step 2: Perform data analysis on the multimodal data for each work step to evaluate the quality level of the set target under the corresponding work step;

[0059] Step 3: When the quality level meets the set standard of the corresponding work step, store the multimodal data under the corresponding work step into database one;

[0060] Step 4: When the quality level does not meet the set standard of the corresponding operation step, determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management based on the difference between the quality level of the corresponding operation step and the set standard, and in combination with the comprehensive level of the target project determined based on all quality levels.

[0061] In this embodiment, the target project is the manufacturing of an industrial robot. The workflow includes multiple steps such as machining of mechanical parts (e.g., milling and drilling of arms and joints), assembly of electrical components (installation of motors, sensors, etc.), software programming and debugging (writing control programs and testing operational logic), and overall assembly (assembling the machined mechanical parts and electrical components). In the machining of mechanical parts, various detection methods are employed, including visual inspection (using a high-definition camera to photograph the surface of parts to detect cracks, defects, and other appearance problems), dimensional measurement sensor detection (e.g., using laser rangefinders to measure whether the key dimensions of parts meet design requirements), and acoustic detection (using sonar devices to detect abnormal vibrations and noise during processing to determine if the processing equipment is operating normally). This results in a multimodal data set encompassing various data types, including numerical values ​​(e.g., part dimensions), images (part appearance images), and audio (audio of the processing equipment's operation), comprehensively reflecting the status information of the target set in the work steps. Multimodal detection can integrate multiple data sources to fully present the actual situation of the work steps. For example, dimensional measurement alone cannot detect minute cracks on the surface of parts, while visual inspection can effectively supplement this; acoustic detection can provide early warning of potential equipment malfunctions. Comprehensive data acquisition provides sufficient evidence for subsequent quality assessments, thereby improving the accuracy of the assessments.

[0062] In this embodiment, machine learning algorithms, such as convolutional neural networks (CNNs), are used to process image data and extract the appearance features of parts; signal processing methods, such as Fourier transform, are used to analyze audio data to determine whether the sound frequency is normal; and statistical analysis is performed on numerical data to calculate statistics such as mean and standard deviation. Quality levels can be divided into 1, 2, 3, and 4, with different levels represented by corresponding numerical values; the higher the level, the more qualified the quality.

[0063] In this embodiment, the set standard is a quality requirement that is predetermined for each work step, that is, a grade standard, such as a grade of 4.

[0064] Database 1 is used to store data that meets quality standards. It can be a relational database, such as MySQL, with table structures built according to work steps, data types, etc., facilitating data storage and retrieval. Database 2 is specifically used to store data that does not meet quality standards. This can be a distributed database, such as Hadoop Distributed File System (HDFS) combined with HBase, to handle potentially large volumes of non-compliant data requiring rapid querying and analysis. When the quality level does not meet the standard, the storage address is determined by combining the difference and overall level, allowing for more precise classification and storage of problematic data. This enables quality management personnel to quickly locate the relevant work steps and analyze the root cause when quality problems occur. For example, by analyzing the non-compliant data in Database 2, it can be discovered that a batch of robots exhibits quality fluctuations across multiple work steps, allowing for in-depth investigation into whether the problem lies with raw materials or production equipment malfunctions, and timely corrective measures can be taken.

[0065] In this embodiment, the grade difference refers to, for example, if the quality grade is level 2 and the standard is level 4, then the grade difference is level -2, and "-2" can be used directly when applying it.

[0066] In this embodiment, the overall quality level is calculated based on the quality level of each work step, using a weighted average or other methods to obtain the overall quality level of the entire industrial robot manufacturing project.

[0067] The beneficial effects of the above technical solution are: on the one hand, it facilitates quality management during the project process; on the other hand, it facilitates the storage and management of qualified and unqualified quality data, thereby facilitating subsequent retrieval and use, and improving the efficiency of problem investigation.

[0068] This invention provides a quality information management method based on multimodal AI, which includes the following steps in the process of analyzing multimodal data for each work step:

[0069] Fill each modal data into the modal blank table that matches the corresponding operation step, determine whether there are missing objects in the filled table, and if so, lock the missing location point of the missing object;

[0070] Based on the format of the filled table, obtain the first reference value of the first non-missing point closest to the missing point, the missing point closest to the missing point, and the second reference value of the second non-missing point closest to the missing point;

[0071] If the number of the nearest missing points is 0, then the missing data at the missing location points is filled in for the first time based on the first reference value;

[0072] If the number of the most recent missing points is 1, then the data of the missing points is filled in a second time based on the first reference value and the second reference value;

[0073] If the number of the most recent missing points is 2, then the data of the missing points is filled in a third time based on the first reference value and two second reference values;

[0074] The complete data for the corresponding modality is obtained based on the completion results.

[0075] Preferably, the data at the missing location points is supplemented a third time based on the first reference value and two second reference values, including:

[0076] ;

[0077] in, These are the first reference value, one second reference value, and another second reference value, respectively. These represent values ​​based on the first reference value. slope of neighboring points of corresponding point Based on a second reference value slope of neighboring points of corresponding point The slope of the neighboring points corresponding to another second reference value C3 ; Indicates the first reference value Corresponding point and a second reference value The slope between corresponding points; Indicates the first reference value Corresponding point and another second reference value The slope between corresponding points; Represents a symbolic function; This represents the total number of non-missing points in the filled table. This indicates the third padding value.

[0078] In this embodiment, the modal blank table is a blank table designed for each type of modality data. For example, the columns of the visual modality blank table may include image number, shooting time, component name, image storage path, etc.; the columns of the sensor modality blank table may include sensor number, acquisition time, measurement parameter name, measurement value, etc. Taking sensor modality data as an example, if a specific sensor does not acquire data within a certain period of time, then the sensor measurement value during that period is the missing object. In the sensor modality blank table, the intersection of the row and column corresponding to the missing data is the missing location point. For example, data is missing in a certain row (corresponding to a specific acquisition time) and a certain column (corresponding to a specific measurement parameter). Filling the data into the corresponding blank tables according to modality facilitates independent management and analysis of each type of modality data, improving the targeting of data processing. Locating the missing location point provides a clear target for subsequent data completion, avoiding blind processing and ensuring the accuracy and efficiency of the completion operation. It should be noted that this part of data completion is data-level completion. The table after completion is in the form of a two-dimensional table with a row and column structure, with rows and columns corresponding to different data attributes and records.

[0079] In this embodiment, in the filled table, the non-missing data point closest to the missing location is the first non-missing point, and the data value corresponding to the first non-missing point is the first reference value.

[0080] In this embodiment, in the filled table, the other missing data point that is closest to the missing location point is the nearest missing point, the non-missing data point that is closest to the nearest missing point is the second non-missing point, and the data value corresponding to the second missing point is the second reference value.

[0081] In this embodiment, when the number of the most recent missing points is 0, the missing data at the missing location points is filled in only based on the first reference value.

[0082] When the number of most recent missing points is 1, the data at the missing location is filled by combining the average of the first reference value and the second reference value.

[0083] In this embodiment, the formula incorporates This approach obtains data information from multiple dimensions, avoiding the one-sidedness of relying solely on a single reference value. Multiple reference values ​​can more comprehensively reflect the characteristics and trends of the data surrounding the missing location, making the imputed value more consistent with the overall data distribution. The introduction of slope takes into account the changing trends of neighboring points corresponding to each reference value. Data often exhibits certain patterns of change in space or time, and these trends can be captured through slope. , The changes between points corresponding to different reference values ​​were taken into account, further improving the description of data change trends and making the supplementary values ​​more consistent with the actual changes in the data.

[0084] In this embodiment, according to The magnitude relationship is adjusted to control the sign of subsequent calculations, balancing the impact of the difference between the two second reference values ​​on the results. The absolute value of the difference is less than b1. The value is 1; otherwise, the value is -1, and the value of b1 is 1.

[0085] In this embodiment, the denominator N1 normalizes the calculation result, so that the calculation result fluctuates within a reasonable range and avoids abnormal results due to factors such as data size.

[0086] The beneficial effects of the above technical solution are: different numbers of nearest missing points indicate different situations surrounding the missing data, and adopting a differentiated imputation strategy is more targeted. When the number of nearest missing points is 0, it means that the data around the missing point is relatively complete, and imputation using only the first reference value is simple and efficient; when the number is 1 or 2, imputation using multiple reference values ​​can better consider the changing trends and correlations of the data, and avoid the deviation that may be caused by a single reference value.

[0087] This invention provides a quality information management method based on multimodal AI, which evaluates the quality level of a set target under corresponding work steps, including:

[0088] Convert the data analysis results for each modality into corresponding feature vectors;

[0089] Based on the characteristics and requirements of the set objectives, multiple quality assessment indicators related to each work step are determined, and corresponding weights are assigned to the quality assessment indicators under each work step. The quality assessment indicators include: installation accuracy, operational standardization, and operational stability.

[0090] The feature vectors are matched with each quality assessment index to construct a multimodal result matrix. ,in, 1 represents the installation accuracy, operational standardization, and operational stability under the first operation step, respectively; These represent the installation accuracy, operational standardization, and operational stability at the nth operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability in the first operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability at the nth operation step, respectively.

[0091] Extract the oblique vector of the multimodal result matrix J, and map each element in the oblique vector to the level lookup table to obtain the initial level of the corresponding element;

[0092] The amount of change is determined based on the connection points of the aforementioned work steps;

[0093] The initial grade is adjusted based on the amount of change to obtain the quality grade.

[0094] In this embodiment, the feature vector is a form of numerical and structured representation of data analysis results. For example, in step 1, the result for parameter A1 is U1, the result for parameter A2 is U2, and the result for parameter A3 is U3. At this time, the feature vector obtained is: [U1 U2 U3].

[0095] In this embodiment, the weights are pre-set values ​​assigned based on the importance of each quality assessment indicator in different work steps. The importance of each quality assessment indicator varies in different work steps. By assigning weights, the impact of key indicators on the quality of work steps can be highlighted. For example, in work step 1: the weight of operational stability may be 0.5; the weight of installation accuracy may be 0.3; and the weight of operational standardization may be 0.2.

[0096] In this embodiment, the matching analysis, for example, compares the feature vector of the component installation position deviation under the visual modality with the installation accuracy index to determine the degree of influence of the installation position deviation on the installation accuracy; and compares the feature vectors of pressure, temperature and other parameters under the sensor modality with the set operating stability index to analyze their effect on the operating stability of the equipment.

[0097] In this embodiment, the oblique vector refers to the vector extracted along the main diagonal direction. The oblique vector elements represent the comprehensive performance results of the quality assessment indicators under different work steps. The preliminary quality level corresponding to each element is obtained by mapping the oblique vector elements to the level comparison table. The level comparison table contains the values ​​of different elements and the level that matches the values.

[0098] In this embodiment, the connection point refers to the impact that the instability of the quality of the previous operation step may have on the subsequent operation steps.

[0099] The beneficial effects of the above technical solution are as follows: the data analysis results under each modality are converted into corresponding feature vectors, realizing a unified structured representation of the data, which facilitates subsequent processing and analysis; multiple quality assessment indicators related to each operation step are identified, and each indicator is assigned a corresponding weight, thus constructing a quantitative system for quality assessment of operation steps; a multimodal result matrix is ​​constructed, which presents the matching results of different modal data and quality assessment indicators under each operation step in matrix form, comprehensively reflecting the quality-related information of each operation step; the extraction of oblique vectors can filter out representative key information from the matrix; the data is simplified while retaining core quality characteristics; and the change amount calculated based on the connection position of the operation steps is determined for subsequent adjustment of the initial level.

[0100] This invention provides a quality information management method based on multimodal AI, which determines the amount of change based on the connection position of the work steps, including:

[0101] When the operation step is the first step, the initial level is regarded as the quality level;

[0102] When the operation step is not the first step, obtain the impact of the complete quality of the previous step on the corresponding operation step;

[0103]

[0104] in, This represents the influence coefficient on the i-th operation step; This indicates the sensitivity of the i-th step to the influence of the (i-1)-th step at the connection position; This indicates the degree of correlation between the i-th step and the (i-1)-th step; n represents the total number of steps in the operation. This represents the directional judgment coefficient for the i-th task step. When it is a positive direction, The value is 1, which indicates a negative direction. The value is -1;

[0105] The adjustment level is determined based on the influence coefficient and the preset unit influence, and the initial level is adjusted to obtain the quality level.

[0106] Preferably, the quality grade is obtained by adjusting the initial grade, including:

[0107]

[0108]

[0109] Where ZD represents quality level; CD represents initial level; TD represents adjustment level; This indicates the floor function.

[0110] In this embodiment, The value of is between 0 and 1. The closer the value is to 1, the more sensitive the i-th step is to the influence of the previous step; the closer the value is to 0, the less sensitive it is to the influence of the previous step.

[0111] The value ranges from 0 to 1. The closer the value is to 1, the closer the relationship between the two steps is; the closer the value is to 0, the looser the relationship is.

[0112] CD: Indicates the initial grade, which is the quality grade determined before considering the impact of work steps. The value is a non-negative integer.

[0113] This formula comprehensively and accurately measures the impact of each preceding step on the current step. In actual production, work steps are interconnected, and the quality of one step affects subsequent steps. Ignoring this impact would lead to inaccurate quality assessments. By considering the interconnectedness of steps, this formula more realistically reflects the quality of each work step.

[0114] In this embodiment, the value of d is generally 1.

[0115] The beneficial effects of the above technical solution are: by considering the impact of step connection, it can more realistically reflect the quality of operation steps, the accurate quality level provides a reliable basis for production decision-making, it provides a scientific and reasonable way to determine the quality level, it can handle complex correlations, and it can adapt to the requirements of precise quality management in complex production environments.

[0116] This invention provides a quality information management method based on multimodal AI, which determines the storage address of multimodal data for corresponding work steps in a second database, including:

[0117] The overall grade is obtained by combining the step weight of each task step with the quality level of each task step, and the overall difference between the overall grade and the overall standard is obtained.

[0118] Based on the quality level and set standards, the difference array Cs= is obtained. ,in, This represents the difference in level between the i-th task steps;

[0119] Calculate the symbol combination {E1, E2} for each remaining task step (excluding the nth task step) and the next task step, where E1 is the symbol for the corresponding remaining task step and E2 is the symbol for the next task step.

[0120] If the corresponding remaining steps do not meet the set criteria, then the storage address in database two is obtained by matching the step-symbol combination-double difference-address lookup table based on the step weight, symbol combination, level difference, and comprehensive difference of the corresponding remaining steps.

[0121] In this embodiment, the comprehensive standard is a pre-set standard used to measure the quality of the entire production process or product, such as a quality level of 5. Considering the weight of each step highlights the impact of key operational steps on the overall quality. Combining the quality level with the comprehensive grade calculates the comprehensive grade, comprehensively reflecting the quality status of the production process. The calculation of comprehensive differences provides a clear direction for quality improvement.

[0122] In this embodiment, the symbol combination includes "--" and "-+". "-" indicates that the corresponding work step does not meet the set standard, and "+" indicates that the corresponding work step meets the set standard. The symbol combination can intuitively reflect the changes and correlations in quality between work steps.

[0123] In this embodiment, the step-symbol combination-dual difference-address lookup table is a pre-established mapping table that records the correspondence between different combinations of job step weights, symbol combinations, level differences, and comprehensive differences and their storage addresses in database two. For example, when the job step weight is within a certain range, the symbol combination is in a specific form, and the level differences and comprehensive differences meet certain conditions, it corresponds to a certain storage area address in database two.

[0124] The beneficial effects of the above technical solution are as follows: by obtaining the comprehensive difference between the comprehensive level and the comprehensive standard, the overall quality can be quantitatively measured and compared. The level difference of each operation step is presented in the form of an array, which facilitates the analysis and comparison of the quality of individual operation steps. The symbol combination of each remaining operation step (excluding the nth operation step) and the next operation step is statistically obtained, and the connection relationship of the quality status between operation steps is recorded. For the remaining operation steps that do not meet the set standard, the storage address in database two is obtained by matching their relevant parameters from the reference table, so as to achieve accurate storage and location of the problematic operation step data.

[0125] This invention provides a quality information management system based on multimodal AI, such as... Figure 2 As shown, it includes:

[0126] The data acquisition module is used to determine the work process of the target project, and to perform multimodal detection on the set target of the corresponding work step according to the multimodal detection method under each work step in the work process to obtain multimodal data.

[0127] The rating assessment module is used to perform data analysis on multimodal data under each work step and to assess the quality level of the set target under the corresponding work step.

[0128] The standard storage module is used to store the multimodal data of the corresponding work step into database one when the quality level meets the set standard of the corresponding work step.

[0129] The non-standard storage module is used to determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management when the quality level does not meet the set standard of the corresponding operation step. This is based on the difference between the quality level of the corresponding operation step and the set standard, and combined with the comprehensive level of the target project determined based on all quality levels.

[0130] The beneficial effects of the above technical solution are: on the one hand, it facilitates quality management during the project process; on the other hand, it facilitates the storage and management of qualified and unqualified quality data, thereby facilitating subsequent retrieval and use, and improving the efficiency of problem investigation.

[0131] The present invention provides a storage medium, including: a memory for storing the aforementioned quality information management method based on multimodal AI.

[0132] The present invention provides a computer device, including: a processor, for executing the aforementioned quality information management method based on multimodal AI.

[0133] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A quality information management method based on multimodal AI, characterized in that, include: Step 1: Determine the work process of the target project. Based on the multimodal detection method under each work step in the work process, perform multimodal detection on the set target of the corresponding work step to obtain multimodal data. Step 2: Perform data analysis on the multimodal data for each work step, converting the data analysis results for each modality into corresponding feature vectors; based on the characteristics and requirements of the set objectives, determine multiple quality assessment indicators related to each work step, and assign corresponding weights to the quality assessment indicators for each work step. These quality assessment indicators include: installation accuracy, operational standardization, and operational stability; perform matching analysis between the feature vectors and each quality assessment indicator to construct a multimodal result matrix. in, These represent the installation accuracy, operational standardization, and operational stability under the first operation step, respectively. These represent the installation accuracy, operational standardization, and operational stability at the nth operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability in the first operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability under the nth operation step, respectively; the slope vector of the multimodal result matrix J is extracted, and each element in the slope vector is mapped to a grade lookup table to obtain the initial grade of the corresponding element; the change amount is determined based on the connection position of the operation steps; the initial grade is adjusted based on the change amount to obtain the quality grade, including: Where ZD represents quality level; CD represents initial level; TD represents adjustment level; Indicates the floor function; Step 3: When the quality level meets the set standard of the corresponding work step, store the multimodal data under the corresponding work step into database one; Step 4: When the quality level does not meet the set standard of the corresponding operation step, determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management based on the difference between the quality level of the corresponding operation step and the set standard, and in combination with the comprehensive level of the target project determined based on all quality levels.

2. The quality information management method based on multimodal AI according to claim 1, characterized in that, The data analysis process for multimodal data under each task step includes: Fill each modal data into the modal blank table that matches the corresponding operation step, determine whether there are missing objects in the filled table, and if so, lock the missing location point of the missing object; Based on the format of the filled table, obtain the first reference value of the first non-missing point closest to the missing point, the missing point closest to the missing point, and the second reference value of the second non-missing point closest to the missing point; If the number of the nearest missing points is 0, then the missing data at the missing location points is filled in for the first time based on the first reference value; If the number of the most recent missing points is 1, then the data of the missing points is filled in a second time based on the first reference value and the second reference value; If the number of the most recent missing points is 2, then the data of the missing points is filled in a third time based on the first reference value and two second reference values; The complete data for the corresponding modality is obtained based on the completion results.

3. The quality information management method based on multimodal AI according to claim 2, characterized in that, Based on the first reference value and two second reference values, a third supplementation is performed on the data at the missing location points, including: ； in, These are the first reference value, one second reference value, and another second reference value, respectively. These represent values ​​based on the first reference value. slope of neighboring points of corresponding point Based on a second reference value slope of neighboring points of corresponding point The slope of the neighboring points corresponding to another second reference value C3 ; Indicates the first reference value Corresponding point and a second reference value The slope between corresponding points; Indicates the first reference value Corresponding point and another second reference value The slope between corresponding points; Represents a symbolic function; This represents the total number of non-missing points in the filled table. This indicates the third padding value.

4. The quality information management method based on multimodal AI according to claim 3, characterized in that, The amount of change is determined based on the connection points of the aforementioned work steps, including: When the operation step is the first step, the initial level is regarded as the quality level; When the operation step is not the first step, obtain the impact of the complete quality of the previous step on the corresponding operation step; in, This represents the influence coefficient on the i-th operation step; This indicates the sensitivity of the i-th step to the influence of the (i-1)-th step at the connection position; This indicates the degree of correlation between the i-th step and the (i-1)-th step; n represents the total number of steps in the operation. This represents the directional judgment coefficient for the i-th task step. When it is a positive direction, The value is 1, which indicates a negative direction. The value is -1; The adjustment level is determined based on the influence coefficient and the preset unit influence, and the initial level is adjusted to obtain the quality level.

5. The quality information management method based on multimodal AI according to claim 1, characterized in that, Determine the storage address of the multimodal data for the corresponding task steps in Database 2, including: The overall grade is obtained by combining the step weight of each task step with the quality level of each task step, and the overall difference between the overall grade and the overall standard is obtained. Based on the quality level and set standards, the difference array Cs= is obtained. ,in, This represents the difference in level between the i-th task steps; Calculate the symbol combination {E1, E2} for each remaining task step (excluding the nth task step) and the next task step, where E1 is the symbol for the corresponding remaining task step and E2 is the symbol for the next task step. If the corresponding remaining steps do not meet the set criteria, then the storage address in database two is obtained by matching the step-symbol combination-double difference-address lookup table based on the step weight, symbol combination, level difference, and comprehensive difference of the corresponding remaining steps.

6. A quality information management system based on multimodal AI, characterized in that, include: The data acquisition module is used to determine the work process of the target project, and to perform multimodal detection on the set target of the corresponding work step according to the multimodal detection method under each work step in the work process to obtain multimodal data. The rating assessment module is used to perform multimodal data analysis for each work step, converting the data analysis results for each modality into corresponding feature vectors. Based on the characteristics and requirements of the set objectives, multiple quality assessment indicators related to each work step are determined, and corresponding weights are assigned to each quality assessment indicator for each work step. These quality assessment indicators include: installation accuracy, operational standardization, and operational stability. The feature vectors are then matched with each quality assessment indicator to construct a multimodal result matrix. in, These represent the installation accuracy, operational standardization, and operational stability under the first operation step, respectively. These represent the installation accuracy, operational standardization, and operational stability at the nth operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability in the first operation step, respectively. These represent the weights of installation accuracy, operational standardization, and operational stability under the nth operation step, respectively; the slope vector of the multimodal result matrix J is extracted, and each element in the slope vector is mapped to a grade lookup table to obtain the initial grade of the corresponding element; the change amount is determined based on the connection position of the operation steps; the initial grade is adjusted based on the change amount to obtain the quality grade, including: Where ZD represents quality level; CD represents initial level; TD represents adjustment level; Indicates the floor function; The standard storage module is used to store the multimodal data of the corresponding work step into database one when the quality level meets the set standard of the corresponding work step. The non-standard storage module is used to determine the storage address of the multimodal data of the corresponding operation step in Database 2 and perform storage management when the quality level does not meet the set standard of the corresponding operation step. This is based on the difference between the quality level of the corresponding operation step and the set standard, and combined with the comprehensive level of the target project determined based on all quality levels.

7. A storage medium, characterized in that, include: A memory for storing the quality information management method based on multimodal AI as described in any one of claims 1-5.

8. A computer device, characterized in that, include: A processor for executing the quality information management method based on multimodal AI as described in any one of claims 1-5.

Images