Production process quality intelligent monitoring method and system based on deep learning driving

Abstract

The invention discloses a production process quality intelligent monitoring method and system based on deep learning driving, and relates to the technical field of artificial intelligence, and the method comprises the steps: firstly obtaining a multi-mode quality inspection index state of a target product and a plurality of to-be-matched index state types, and extracting state features to obtain corresponding feature vectors; calculating state confidence distribution according to the multi-modal feature vector; obtaining a relation topological structure of each composite abnormal quality inspection condition; performing embedding processing on the category feature vector by combining confidence distribution and a topological structure to obtain a depth category feature vector; and finally, determining a target index state category as a quality monitoring result according to the matching coefficient, thereby realizing intelligent monitoring on the quality of the production process.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence, and in particular to a method and system for intelligently monitoring production process quality based on deep learning. Background Art

[0002] In the field of production process quality monitoring, traditional methods mostly rely on manual experience or simple data statistics, which are difficult to cope with complex and changeable production conditions and multi-dimensional quality inspection indicators. As the scale of production expands and the complexity of products increases, this monitoring method is inefficient and inaccurate, and it is unable to detect quality problems in a timely and accurate manner. Although deep learning technology has been applied to some quality monitoring scenarios, the comprehensive processing of multimodal quality inspection indicators and the complex relationships between indicators still needs to be improved. The present invention aims to use deep learning technology to fully explore the information of multimodal quality inspection indicators and the relationship between indicator status categories, so as to realize intelligent, efficient and accurate monitoring of production process quality. Summary of the Invention

[0003] The purpose of the present invention is to provide a method and system for intelligent monitoring of production process quality based on deep learning.

[0004] In a first aspect, an embodiment of the present invention provides a method for intelligently monitoring production process quality based on deep learning, comprising:

[0005] Obtaining a multimodal quality inspection indicator state and multiple indicator state categories to be matched for a target product, and extracting state features for the multimodal quality inspection indicator state and each indicator state category to be matched in turn, to obtain a multimodal feature vector for the multimodal quality inspection indicator state and a category feature vector for each indicator state category to be matched;

[0006] Calculating, based on the multimodal feature vector, a state confidence distribution of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition;

[0007] Obtaining a relationship topology structure corresponding to each composite abnormal quality inspection condition, wherein the relationship topology structure reflects the mutual influence relationship between the status categories of each to-be-matched indicator under the composite abnormal quality inspection condition;

[0008] Based on the state confidence distribution and the relationship topological structure corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the category feature vector of each to-be-matched indicator state category to obtain a deep category feature vector corresponding to each to-be-matched indicator state category;

[0009] Determining, based on the depth category feature vector corresponding to each to-be-matched indicator state category, a matching coefficient for each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state;

[0010] According to the matching coefficient, a target indicator state category corresponding to the multimodal quality inspection indicator state is determined from each of the indicator state categories to be matched as a quality monitoring result of the target product.

[0011] In a second aspect, an embodiment of the present invention provides a server system, including a server, wherein the server is configured to execute the method described in the first aspect.

[0012] Compared with the existing technology, the beneficial effects provided by the present invention include: adopting a deep learning-driven intelligent monitoring method and system for production process quality disclosed by the present invention, by obtaining the multimodal quality inspection indicator status of the target product and multiple indicator status categories to be matched, extracting state features to obtain corresponding feature vectors; calculating the state confidence distribution based on the multimodal feature vectors; obtaining the relationship topological structure of each composite abnormal quality inspection condition; combining the confidence distribution and the topological structure to embed the category feature vector to obtain a deep category feature vector; determining the matching coefficient based on this, and finally determining the target indicator status category as the quality monitoring result according to the matching coefficient, thereby realizing intelligent monitoring of the production process quality. BRIEF DESCRIPTION OF THE DRAWINGS

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly describes the drawings required for use in the embodiments. It should be understood that the following drawings illustrate only certain embodiments of the present invention and should not be construed as limiting the scope of the present invention. Those skilled in the art can, without inventive effort, derive other relevant drawings from these drawings.

[0014] Figure 1 A schematic diagram of the steps of the method for intelligent monitoring of production process quality based on deep learning provided by an embodiment of the present invention;

[0015] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. DETAILED DESCRIPTION

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more apparent, the technical solutions of the embodiments of the present invention will be described clearly and completely below in conjunction with the accompanying drawings of the embodiments of the present invention. It should be understood that the described embodiments are only a portion of the embodiments of the present invention, not all of them. Generally, the components of the embodiments of the present invention described and illustrated in the drawings herein may be arranged and designed in a variety of different configurations.

[0017] The specific embodiments of the present invention are described in detail below with reference to the accompanying drawings.

[0018] In order to solve the technical problems in the above background technology, Figure 1This is a flow chart of a method for intelligent monitoring of production process quality based on deep learning driven by an embodiment of the present disclosure. The method for intelligent monitoring of production process quality based on deep learning driven by an embodiment of the present disclosure is introduced in detail below.

[0019] Step S201: Acquire a multimodal quality inspection indicator state and multiple indicator state categories to be matched for a target product, and sequentially extract state features for the multimodal quality inspection indicator state and each indicator state category to be matched, to obtain a multimodal feature vector for the multimodal quality inspection indicator state and a category feature vector for each indicator state category to be matched;

[0020] Step S202, calculating the state confidence distribution of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition based on the multimodal feature vector;

[0021] Step S203: Obtain a relationship topology structure corresponding to each composite abnormal quality inspection condition, wherein the relationship topology structure reflects the mutual influence relationship between the status categories of each to-be-matched indicator under the composite abnormal quality inspection condition;

[0022] Step S204: Based on the state confidence distribution and the relationship topology corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the category feature vector of each to-be-matched indicator state category to obtain a deep category feature vector corresponding to each to-be-matched indicator state category;

[0023] Step S205, determining a matching coefficient of each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state based on the depth category feature vector corresponding to each to-be-matched indicator state category;

[0024] Step S206 : According to the matching coefficient, a target indicator state category corresponding to the multimodal quality inspection indicator state is determined from each of the indicator state categories to be matched as the quality monitoring result of the target product.

[0025] In an embodiment of the present invention, the server illustratively obtains the multimodal quality inspection indicator status of a target product from various data sources, such as sensors and testing equipment on the production line. For example, in an automobile manufacturing company, the multimodal quality inspection indicator status for a target product, such as an automobile engine, may include data from engine temperature sensors, vibration sensors, and fuel injection volume. These data reflect the engine's operating status from different dimensions. The server also obtains multiple indicator status categories to be matched, such as normal engine temperature, excessive engine temperature, abnormal engine vibration, and fuel injection volume deviation. These categories are pre-defined and used to match the actual multimodal quality inspection indicator status. The server performs multimodal state characterization processing on the quality inspection indicator data across multiple quality inspection dimensions. Still using the automobile engine as an example, for the temperature dimension, the server uses a specific algorithm to convert the temperature data collected in real time by the temperature sensor into a vector representation that reflects characteristics such as temperature change trends and fluctuations. For the vibration dimension, the server converts the vibration sensor data into a vector representation that represents characteristics such as vibration frequency and amplitude. In this way, the quality inspection indicator characteristics of the multimodal quality inspection indicator status across each quality inspection dimension are obtained. For each matching indicator state category, the server also performs multimodal state representation processing. For example, for the matching indicator state category "engine temperature is too high," the server combines historical data, industry standards, and other information to construct a feature vector representing this state category. This feature vector may include relevant features such as the temperature exceeding a specific threshold and the temperature rise rate. The server constructs a feature set based on the state category features of each matching indicator state category and the industry prior knowledge vector. This prior knowledge vector may be derived from the automotive manufacturing industry's long-term experience, such as knowledge of the normal engine temperature range and vibration tolerance under different operating conditions. This knowledge is converted into vector form and combined with the state category features of each matching indicator state category to form a rich feature set. The server determines the target quality inspection dimension for this cycle from the quality inspection indicator features of multiple quality inspection dimensions. For this cycle, the quality inspection indicator features of the temperature dimension are selected. The server then performs cross-modal gated interaction between the quality inspection indicator features of this target quality inspection dimension and the feature set. During this process, the server uses complex algorithms to interact with the temperature dimension features and each element in the feature set to extract more representative features. For example, the server analyzes the relationship between temperature features, state category features such as "engine temperature too high," and industry prior knowledge vectors to obtain a graph convolution feature vector for the multimodal quality inspection indicator state and a graph convolution feature vector for each indicator state category to be matched. The server determines the current feature set based on these graph convolution feature vectors.Next, the server iterates, determining the quality inspection indicator features for the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of multiple quality inspection dimensions, until the process completes with all quality inspection indicator features for all quality inspection dimensions (such as temperature, vibration, and fuel injection quantity). Finally, based on the current feature set, it obtains the multimodal feature vector for the multimodal quality inspection indicator state and the category feature vector for each indicator state category to be matched. Another approach is to extract state features using a quality inspection state recognition network. The server sequentially inputs the obtained multimodal quality inspection indicator state and each indicator state category to be matched into the quality inspection state recognition network. This network, trained with extensive data, automatically learns how to extract effective features from the input data. For example, during training, the network has learned the characteristic patterns of various indicator data under different operating states of a vehicle engine. When a new multimodal quality inspection indicator state is input, it can accurately extract its multimodal feature vector and the corresponding category feature vector for each indicator state category to be matched. The server performs calculations using the multimodal feature vectors. For example, consider a composite abnormal quality inspection condition, such as "engine overheating and abnormal vibration." The server inputs the multimodal feature vector into a specific algorithm model, which analyzes the relationships between multimodal features, such as the temperature feature vector and the vibration feature vector. If the temperature feature vector indicates that the engine temperature is approaching or exceeding the overheat threshold, and the vibration feature vector indicates that the vibration amplitude is outside the normal range, the model, based on this information and the patterns learned during training, calculates a confidence value for the multimodal quality inspection indicator state belonging to the composite abnormal quality inspection condition, "engine overheating and abnormal vibration." The server performs similar calculations for all pre-defined composite abnormal quality inspection conditions, such as "engine fuel injection abnormality and unstable temperature," to obtain a state confidence distribution for the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition—that is, a set of confidence values corresponding to each composite abnormal quality inspection condition. The server then obtains the relationship topology corresponding to each composite abnormal quality inspection condition. Taking the compound abnormal quality inspection condition of "engine overheating and abnormal vibration" as an example, its relationship topology includes category objects corresponding to each indicator state category to be matched, such as the "engine temperature is too high" category object and the "engine vibration is abnormal" category object. The path between category objects reflects the mutual influence relationship between the two connected category objects. For example, excessively high engine temperature may cause engine components to expand and contract due to heat and cool, thereby affecting engine vibration. This influence relationship is represented by a path. In the compound abnormal quality inspection condition of "engine fuel injection is abnormal and temperature is unstable", there is also a path between the "fuel injection abnormality" category object and the "temperature unstable" category object, indicating that abnormal fuel injection amount may lead to incomplete engine combustion, thereby causing temperature fluctuations, reflecting the mutual influence between them.For each indicator state category to be matched, the server determines the corresponding target category object in the relational topology corresponding to each composite abnormal quality inspection condition. For example, for the indicator state category to be matched, "engine temperature is too high" is one of the target category objects in the relational topology of "engine overheating and abnormal vibration." For each composite abnormal quality inspection condition, the server determines the associated feature vector of the target category object in that composite abnormal quality inspection condition based on the mutual influence between the target category object and the remaining category objects in the relational topology. For example, in the relational topology of "engine overheating and abnormal vibration," the "engine temperature is too high" target category object is connected to the "engine vibration abnormality" category object via a path. The server comprehensively considers factors such as the impact of excessive temperature on vibration and the feedback of abnormal vibration on temperature, and combines the multimodal feature vector and information in the relational topology to determine the associated feature vector of the "engine temperature is too high" target category object in this composite abnormal quality inspection condition. For each compound abnormal quality inspection condition, the server merges the credible value of the multimodal quality inspection indicator state belonging to the compound abnormal quality inspection condition with the associated feature vector of the target category object under the compound abnormal quality inspection condition. For example, for the compound abnormal quality inspection condition of "engine overheating and abnormal vibration," if the calculated credible value of the multimodal quality inspection indicator state belonging to this condition is 0.8, and the associated feature vector of the target category object of "engine temperature too high" is [0.2, 0.3, 0.4], the server will merge the credible value 0.8 with the associated feature vector [0.2, 0.3, 0.4] using a specific algorithm to obtain the basic merged features of the "engine temperature too high" indicator state category to be matched under this compound abnormal quality inspection condition. Based on the basic merged features of the indicator state category to be matched under each compound abnormal quality inspection condition, the server determines the merged feature vector of the indicator state category to be matched. For example, the "engine temperature is too high" indicator state category to be matched has corresponding basic merged features in all complex abnormal quality inspection conditions such as "engine overheating and abnormal vibration", "engine fuel injection abnormality and unstable temperature", etc. The server comprehensively calculates these basic merged features to obtain the merged feature vector of the "engine temperature is too high" indicator state category to be matched. Based on the merged feature vector, the server optimizes the category feature vector of the indicator state category to be matched. For example, for the "engine temperature is too high" indicator state category to be matched, the server fuses and adjusts its original category feature vector with the merged feature vector, and obtains the deep category feature vector corresponding to the indicator state category to be matched through specific optimization algorithms, such as the back propagation algorithm in the neural network. This deep category feature vector more accurately reflects the relationship between the state category and the multimodal quality inspection indicator state and various complex abnormal quality inspection conditions.For each indicator state category to be matched, the server merges the category feature vector and the deep category feature vector for that category to be matched to obtain a target category feature vector. For example, for the indicator state category "abnormal engine vibration," the server merges the original category feature vector with the deep category feature vector obtained through feature embedding. Using a specific merging algorithm, the information in the two vectors is integrated to produce a more representative target category feature vector. Based on the target category feature vector for each indicator state category to be matched, the server determines the matching coefficient for each indicator state category within the multimodal quality inspection indicator state. The server inputs the target category feature vector into a specific scoring model. The model uses the feature information in the vector and the criteria learned during training to generate a numerical matching coefficient. For example, if the target category feature vector for the indicator state category "abnormal engine vibration" closely matches the vibration characteristics of the multimodal quality inspection indicator state, the model will assign a high matching coefficient, indicating a high degree of match between the target category and the current multimodal quality inspection indicator state. Based on the matching coefficients of each indicator state category, the server determines the target indicator state category corresponding to the multimodal quality inspection indicator state from all the indicator state categories to be matched, which serves as the quality monitoring result for the target product. For example, in the quality monitoring of automobile engines, the matching coefficient for "excessive engine temperature" is 0.7, the matching coefficient for "abnormal engine vibration" is 0.8, and the matching coefficient for "fuel injection quantity deviation" is 0.3. The server compares these matching coefficients and finds that "abnormal engine vibration" has the highest matching coefficient. It then determines "abnormal engine vibration" as the target indicator state category, indicating that the current automobile engine quality monitoring result indicates an abnormality in engine vibration. Through the detailed steps described above, the deep learning-driven intelligent production process quality monitoring method can help enterprise servers accurately and efficiently monitor the quality of target products, promptly identify potential problems in the production process, and provide strong support for enterprises to improve product quality and reduce production costs.

[0026] In a possible implementation, the state feature extraction is performed on the multimodal quality inspection indicator state and each indicator state category to be matched in turn to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each indicator state category to be matched, which can be implemented through the following example.

[0027] Performing multimodal state characterization processing on the quality inspection indicator data of the multimodal quality inspection indicator state in at least one quality inspection dimension to obtain a quality inspection indicator feature of the multimodal quality inspection indicator state in at least one quality inspection dimension;

[0028] Perform multimodal state representation processing on each state category of the indicator to be matched in turn to obtain the state category features of each state category of the indicator to be matched;

[0029] Construct a feature set based on the state category features of each indicator state category to be matched and the industry prior knowledge vector;

[0030] The quality inspection indicator feature and the feature set are subjected to cross-modal gated interaction to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each indicator state category to be matched.

[0031] In an embodiment of the present invention, for example, a server is responsible for monitoring the quality of the mobile phone production process. Regarding multimodal quality inspection indicator status, mobile phone production involves multiple quality inspection dimensions, such as appearance and performance. In the appearance dimension, the server obtains quality inspection indicator data such as the color, scratches, and flatness of the mobile phone casing, and performs multimodal state characterization on this data. For example, a specific algorithm is used to convert color data into feature values representing the degree of color deviation, and scratch data into feature representations such as scratch length and depth, thereby obtaining quality inspection indicator features for the appearance dimension. In the performance dimension, the server processes data such as the mobile phone processor performance and battery life, converting processor operating frequency and heat generation into corresponding features, thereby obtaining quality inspection indicator features for at least one quality inspection dimension of the multimodal quality inspection indicator status. For each indicator status category to be matched, such as "minor appearance defects" and "unstable performance," the server performs multimodal state characterization on each of them. Taking "minor appearance defects" as an example, by combining historical data on similar appearance issues and industry standards, features such as minor scratches and subtle color differences are quantified to obtain the status category features for this status category. Next, the server constructs a feature set based on the state category features of each indicator state category to be matched and the industry prior knowledge vector. The industry prior knowledge vector contains knowledge conversion vectors of common appearance and performance standards in the mobile phone industry, such as the allowable range of color deviation for mobile phone cases and the normal heating range for processors. These vectors are integrated with the state category features to form a feature set. Finally, the server performs cross-modal gated interaction between the quality inspection indicator features and the feature set. For example, the quality inspection indicator features of the appearance dimension are first selected and interacted with the elements in the feature set. By analyzing the relationship between the appearance features and the state category features such as "minor appearance defects" and the industry prior knowledge vector, a complex algorithm is applied to obtain the graph convolution feature vectors of the multimodal quality inspection indicator state of the mobile phone appearance, as well as the graph convolution feature vectors for each indicator state category to be matched, such as "minor appearance defects." After multiple cycles through different quality inspection dimensions, based on the graph convolution feature vectors obtained each time, the multimodal feature vectors of the multimodal quality inspection indicator state and the category feature vectors for each indicator state category to be matched are ultimately obtained.

[0032] In a possible implementation, the cross-modal gating interaction between the quality inspection indicator feature and the feature set to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched can be implemented through the following examples.

[0033] Determine the quality inspection indicator characteristics of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator characteristics of the at least one quality inspection dimension;

[0034] Performing cross-modal gated interaction on the quality inspection indicator features of the target quality inspection dimension and the feature set to obtain a graph convolution feature vector of the multimodal quality inspection indicator state and a graph convolution feature vector of each indicator state category to be matched;

[0035] Determine a current feature set based on the graph convolution feature vector of the multimodal quality inspection indicator state and the graph convolution feature vector of each indicator state category to be matched;

[0036] The step of determining the quality inspection indicator features of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of the at least one quality inspection dimension is cyclically executed until the quality inspection indicator features of all quality inspection dimensions are completed, and based on the current feature set, the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched are obtained.

[0037] In an embodiment of the present invention, for example, the server performs quality monitoring on the production process of smart watches. The quality inspection dimensions of smart watches include hardware performance, appearance, battery life, etc. The server starts to perform cross-modal gated interaction operations. First, the quality inspection indicator characteristics of the target quality inspection dimension corresponding to this cycle are determined from the quality inspection indicator characteristics of multiple quality inspection dimensions. For example, the quality inspection indicator characteristics of the hardware performance dimension are selected in the first cycle, which may include quantized feature values such as processor operation speed and sensor accuracy. Then, the server performs cross-modal gated interaction on the quality inspection indicator characteristics of the target quality inspection dimension (hardware performance dimension) and the feature set constructed previously. The feature set includes state category features such as "hardware performance lag" and "sensor data deviation" waiting to match indicator state categories, as well as prior knowledge vectors of the smart watch industry about hardware performance standards. Using a specific algorithm, the relationship between hardware performance characteristics and each element in the set is analyzed, such as the correlation between processor speed and the "hardware performance lag" status category. This yields a graph convolutional feature vector for the smartwatch's multimodal quality inspection indicator status in the hardware performance dimension, as well as a graph convolutional feature vector for each of the target indicator status categories, such as "hardware performance lag" and "sensor data deviation." The current feature set is then determined based on the graph convolutional feature vector for the multimodal quality inspection indicator status and the graph convolutional feature vector for each target indicator status category. This current feature set incorporates the key feature information generated by this cross-modal gating interaction in the hardware performance dimension. The server then iterates through the steps for determining the target quality inspection dimension. The second cycle selects the appearance dimension, whose quality inspection indicator features may include dial flatness and strap color. Similarly, a cross-modal gating interaction is performed on the quality inspection indicator features of the appearance dimension and the feature set, resulting in a graph convolutional feature vector for the multimodal quality inspection indicator status in the appearance dimension and a graph convolutional feature vector for each target indicator status category. The current feature set is then updated. This cycle continues until all quality inspection indicator features for all quality inspection dimensions, including battery life, have completed the above process. Ultimately, based on the continuously updated current feature set, the server obtains a multimodal feature vector that fully reflects the status of the smartwatch's multimodal quality inspection indicators, as well as a category feature vector corresponding to each indicator status category to be matched, providing key data support for subsequent judgment of the smartwatch's quality status.

[0038] In an embodiment of the present invention, the relationship topology structure includes a category object corresponding to each indicator state category to be matched, and a path between category objects, wherein the path reflects the mutual influence relationship between two connected category objects;

[0039] Based on the state confidence distribution and the relational topological structure corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the category feature vector of each indicator state category to be matched to obtain a deep category feature vector corresponding to each indicator state category to be matched, which can be implemented through the following examples.

[0040] For each to-be-matched indicator status category, determine the target category object corresponding to the to-be-matched indicator status category in the relationship topology structure corresponding to each composite abnormal quality inspection condition;

[0041] For each compound abnormal quality inspection condition corresponding to the relationship topology structure, based on the mutual influence relationship between the target category object and the other category objects in the relationship topology structure, determine the associated feature vector of the target category object under the compound abnormal quality inspection condition;

[0042] Based on the state confidence distribution, the associated feature vectors of the target category objects under each composite abnormal quality inspection condition are merged to obtain a merged feature vector of the state category of the indicator to be matched;

[0043] Based on the combined feature vector, the category feature vector of the to-be-matched indicator state category is optimized to obtain a depth category feature vector corresponding to the to-be-matched indicator state category.

[0044] In an embodiment of the present invention, let's take the example of a server performing quality monitoring during the laptop production process. In this scenario, the indicator status categories to be matched include "abnormal screen display," "short battery life," and "poor heat dissipation," among others. A composite abnormal quality inspection condition might be "abnormal screen display and poor heat dissipation." In the relational topology, each indicator status category to be matched corresponds to a category object. For example, "abnormal screen display" corresponds to a category object, and "poor heat dissipation" also corresponds to a category object. The presence of a path between these categories indicates a mutual influence. For example, poor heat dissipation can affect the screen display because high temperatures can cause unstable performance of the screen display components. First, for each indicator status category to be matched, the server determines the corresponding target category object in the relational topology corresponding to each composite abnormal quality inspection condition. For example, for the indicator status category to be matched, "abnormal screen display" corresponds to the target category object in the relational topology corresponding to the composite abnormal quality inspection condition "abnormal screen display and poor heat dissipation." Next, for each composite abnormal quality inspection condition, the server determines the associated feature vector for the target category object based on the mutual influence between the target category object and the remaining category objects. In the relationship topology for "abnormal screen display and poor heat dissipation," "abnormal screen display" is the target class object. The server considers the impact of poor heat dissipation on screen display, such as the potential for screen flickering and color deviation caused by high temperatures. Combining relevant features from the multimodal quality inspection indicator states, the server determines the associated feature vector for "abnormal screen display" in this composite abnormal quality inspection condition. This vector may include feature values associated with screen issues caused by high temperatures. Then, based on the state confidence distribution, the server merges the associated feature vectors for the target class objects in each composite abnormal quality inspection condition to obtain a merged feature vector for the indicator state category to be matched. For the composite abnormal quality inspection condition "abnormal screen display and short battery life," the server similarly determines the associated feature vector for "abnormal screen display" in this condition. This is then combined with the associated feature vector for "abnormal screen display and poor heat dissipation." Based on the state confidence distribution (for example, the confidence value for "abnormal screen display and poor heat dissipation" is 0.7, while the confidence value for "abnormal screen display and short battery life" is 0.3), the server uses a specific algorithm to proportionally merge the two associated feature vectors to obtain the merged feature vector for the indicator state category to be matched. Finally, based on the merged feature vector, the server optimizes the category feature vector of the "screen display abnormality" indicator status category to be matched to obtain a deep category feature vector.The server combines the original "screen display abnormality" category feature vector with the merged feature vector, and adjusts it using an optimization algorithm so that the resulting deep category feature vector more accurately reflects the relationship between "screen display abnormality" and different composite abnormal quality inspection conditions, providing a more accurate feature basis for the subsequent determination of the degree of matching with the multimodal quality inspection indicator status.

[0045] In an embodiment of the present invention, the state confidence distribution includes a credible value that the state of the multimodal quality inspection indicator belongs to each composite abnormal quality inspection condition;

[0046] Based on the state confidence distribution, the associated feature vectors of the target category objects under each compound abnormal quality inspection condition are feature merged to obtain the merged feature vector of the state category of the indicator to be matched, which can be implemented through the following examples.

[0047] For each composite abnormal quality inspection condition, the credible value of the multimodal quality inspection indicator state belonging to the composite abnormal quality inspection condition is merged with the associated feature vector of the target category object under the composite abnormal quality inspection condition to obtain the basic merged feature of the indicator state category to be matched under the composite abnormal quality inspection condition;

[0048] Based on the basic combined features of the to-be-matched indicator state category under each compound abnormal quality inspection condition, a combined feature vector of the to-be-matched indicator state category is determined.

[0049] In an embodiment of the present invention, for example, taking the server monitoring the production quality of smartphones as an example, there are currently multiple composite abnormal quality inspection conditions, such as "camera imaging is blurry and the battery is depleted quickly" and "body is overheated and the signal is unstable". The indicator state categories to be matched include "camera imaging is blurry", "battery is depleted quickly", "body is overheated", "signal is unstable", etc. First, the state confidence distribution is clarified, which represents the credible value of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition. The server has calculated in the early stage that the credible value of the multimodal quality inspection indicator state belonging to "camera imaging is blurry and the battery is depleted quickly" is 0.8, and the credible value of the state belonging to "body is overheated and the signal is unstable" is 0.6. Next, operations are performed for each composite abnormal quality inspection condition. Taking the "camera imaging is blurry" indicator state category to be matched as an example, under the composite abnormal quality inspection condition of "camera imaging is blurry and the battery is depleted quickly", the server has determined that the associated feature vector of the "camera imaging is blurry" target category object is [0.2, 0.3, 0.1]. The server merges the confidence value 0.8 for this composite abnormal quality inspection condition with the associated feature vector [0.2, 0.3, 0.1]. For example, using a specific weighting algorithm, the confidence value 0.8 is multiplied by each value in the associated feature vector to obtain the basic merged feature [0.16, 0.24, 0.08] for "camera blur" in this composite abnormal quality inspection condition. Similarly, for the composite abnormal quality inspection condition "body overheating and unstable signal," if the associated feature vector for the "camera blur" target class object is [0.1, 0.2, 0.3], the server merges the confidence value 0.6 with this associated feature vector to obtain the basic merged feature [0.06, 0.12, 0.18] for "camera blur" in this condition. Finally, based on the basic merged features of "camera blur" in each composite abnormal quality inspection condition, the server determines its merged feature vector. The server uses a comprehensive calculation method, such as adding the values of the corresponding positions of the two basic merged features and taking the average, namely [(0.16+0.06) / 2, (0.24+0.12) / 2, (0.08+0.18) / 2], to obtain the final merged feature vector [0.11, 0.18, 0.13]. This merged feature vector comprehensively considers the characteristic information of "camera imaging blur" under different composite abnormal quality inspection conditions, providing a basis for subsequent optimization of the "camera imaging blur" category feature vector and ultimately obtaining the depth category feature vector.

[0050] In the embodiment of the present invention, the determination of the matching coefficient of each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state based on the depth category feature vector corresponding to each to-be-matched indicator state category can be implemented through the following examples.

[0051] For each indicator state category to be matched, the category feature vector and the depth category feature vector of the indicator state category to be matched are merged to obtain the target category feature vector of the indicator state category to be matched;

[0052] Based on the target category feature vector of each to-be-matched indicator state category, a matching coefficient of each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state is determined.

[0053] In an embodiment of the present invention, illustratively, taking the example of a server monitoring the quality of a tablet computer production process, the tablet computer's to-be-matched indicator status categories include "screen touch insensitivity", "battery life is poor", "audio output abnormality", etc. The server has obtained the deep category feature vector of each to-be-matched indicator status category through the previous steps. For the to-be-matched indicator status category of "screen touch insensitivity", the server first merges its original category feature vector and deep category feature vector. The original category feature vector is obtained by performing multimodal state characterization processing on the status category, and may include relevant feature values such as touch response time and false touch rate. The deep category feature vector is obtained after comprehensively considering various complex abnormal quality inspection conditions, such as further refining the "screen touch insensitivity" feature under complex abnormal quality inspection conditions such as "screen touch insensitivity and poor battery life" and "screen touch insensitivity and audio output abnormality", which may include feature values after interaction with other abnormal conditions. The server uses a specific merging algorithm to merge these two vectors, such as element-wise addition or weighted fusion, to obtain the target category feature vector for the "screen touch insensitivity" indicator status category to be matched. The same process applies to other indicator status categories to be matched, such as "poor battery life" and "abnormal audio output." The server merges these respective category feature vectors with the depth category feature vector to obtain the corresponding target category feature vector. Next, based on the target category feature vector for each indicator status category to be matched, the server determines the matching coefficient for each indicator status category to be matched within the multimodal quality inspection indicator status. The server inputs the target category feature vector for "screen touch insensitivity" into a pre-trained matching coefficient calculation model. This model, trained using extensive tablet production data, understands the relationships between various tablet indicators under normal and abnormal conditions. Based on the feature values in the target category feature vector, such as the degree of change in touch response time and the degree of correlation with other abnormal conditions, the model outputs a value as the matching coefficient for "screen touch insensitivity" within the current multimodal quality inspection indicator status according to specific calculation rules. For example, if the target category feature vector indicates that the touch response time exceeds the normal range and is strongly correlated with abnormal conditions such as poor battery life, the model output may have a high matching coefficient, indicating that the "screen touch is not sensitive" state has a high degree of match with the current multimodal quality inspection indicator state. Following the same process, the server processes the target category feature vectors for other indicator state categories to be matched, such as "poor battery life" and "abnormal audio output," and determines their matching coefficients for the multimodal quality inspection indicator states. These matching coefficients will provide key evidence for subsequently determining the actual quality issues of the tablet.

[0054] In an embodiment of the present invention, the state feature extraction of the multimodal quality inspection indicator state and each indicator state category to be matched is performed in sequence to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched, which can be implemented through the following examples.

[0055] Through the quality inspection state recognition network, state feature extraction is performed on the multimodal quality inspection indicator state and each indicator state category to be matched in turn to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each indicator state category to be matched.

[0056] In an embodiment of the present invention, illustratively, the server is responsible for quality monitoring of the production process of smart headphones. The production of smart headphones involves a variety of quality inspection indicators. The multimodal quality inspection indicator status covers data on multiple aspects such as acoustic performance, connection stability, and battery life, while the indicator status categories to be matched include "sound quality noise", "unstable Bluetooth connection", "too short battery life", etc. The server uses a quality inspection status recognition network to extract features. First, the multimodal quality inspection indicator status data of the smart headphones is input into the quality inspection status recognition network. For example, frequency response curve data and harmonic distortion data in terms of acoustic performance, Bluetooth connection success rate and signal strength fluctuation data in terms of connection stability, battery capacity and charging speed data in terms of battery life, etc. The quality inspection status recognition network is a deep neural network model that has been trained with a large amount of smart headphone production data. It can automatically learn patterns and features in the data. Each layer of neurons in the network will process the input data layer by layer. For example, the convolution layer extracts local feature patterns in the acoustic data, and the recurrent neural network layer analyzes the time series characteristics of the connection stability data. After a series of complex calculations and transformations, the final output is a multimodal feature vector representing the multimodal quality inspection indicator status. This vector embodies the key characteristic information of the smart earphones across various quality inspection dimensions. Next, for the "noise" indicator status category to be matched, the server also inputs relevant descriptive data into the quality inspection status recognition network. This data may include acoustic parameters of the earphones when noise issues have occurred in the past, production batch information, and feedback from the usage environment. Based on the knowledge learned from its training, the network analyzes and processes this data, discovering features related to "noise" from different perspectives. Ultimately, it outputs a category feature vector for the "noise" indicator status category to be matched. This vector accurately characterizes the "noise" status. Similarly, the server inputs relevant data for each category to be matched, such as "unstable Bluetooth connection" and "short battery life," into the quality inspection status recognition network to obtain the corresponding category feature vector. In this way, the server uses the quality inspection status recognition network to efficiently and accurately complete the state feature extraction of multimodal quality inspection indicator status and each indicator status category to be matched, providing a key data basis for the subsequent analysis and judgment of the production quality of smart headphones.

[0057] In an embodiment of the present invention, the quality inspection state recognition network is used to extract state features of the multimodal quality inspection indicator state and each indicator state category to be matched in turn, and before obtaining the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched, the following implementation method is also provided.

[0058] Acquire a training sample and a relationship topology structure corresponding to each composite abnormal quality inspection condition, wherein the training sample includes a plurality of preset quality inspection indicator states and preset indicator state categories of the preset quality inspection indicator states;

[0059] By using a quality inspection state recognition network, state feature extraction is performed on the preset quality inspection indicator state and each to-be-matched indicator state category in turn to obtain a preset multimodal feature vector of the preset quality inspection indicator state and a preset category feature vector of each to-be-matched indicator state category;

[0060] Calculating, based on the preset multimodal feature vector, a state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition;

[0061] Based on the state confidence distribution and the relationship topological structure corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the preset category feature vector of each to-be-matched indicator state category to obtain a preset depth category feature vector corresponding to each to-be-matched indicator state category;

[0062] Determining a matching coefficient for each to-be-matched indicator state category belonging to the preset quality inspection indicator state based on a preset depth category feature vector corresponding to each to-be-matched indicator state category;

[0063] According to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network are updated to obtain a trained quality inspection state recognition network.

[0064] In an embodiment of the present invention, for example, a server monitors the quality of a smart bracelet production process. First, the server obtains training samples and the corresponding relationship topology for each compound abnormal quality inspection condition. The preset quality inspection indicator states in the training samples cover various types of smart bracelet data, such as heart rate monitoring accuracy, step count deviation, and screen brightness adjustment stability. Furthermore, these preset quality inspection indicator states have corresponding preset indicator state categories, such as "heart rate monitoring inaccurate," "step count abnormal," and "screen brightness adjustment failure." For a compound abnormal quality inspection condition, such as "heart rate monitoring inaccurate and screen brightness adjustment failure," its relationship topology clearly defines the mutual influence between the two categories of objects, "heart rate monitoring inaccurate" and "screen brightness adjustment failure." For example, a motherboard power supply issue could affect both functions simultaneously. Next, the server extracts state features for the preset quality inspection indicator states and each to-be-matched indicator state category through the quality inspection state recognition network. When data on a preset quality inspection indicator state, such as heart rate monitoring accuracy, is input into the network, the network's internal neuron layers perform computations. Convolutional layers capture local feature patterns in the data, while recurrent layers analyze the data's time series characteristics. Ultimately, this results in a preset multimodal feature vector for the preset quality inspection indicator state. For each matching indicator state category, such as "inaccurate heart rate monitoring," the network processes the relevant data in a similar manner, outputting the corresponding preset category feature vector. Subsequently, based on the preset multimodal feature vectors, the server calculates the state confidence distribution for each compound abnormal quality inspection condition. For example, for a compound abnormality such as "inaccurate heart rate monitoring and screen brightness adjustment failure," the server comprehensively analyzes the features related to heart rate and screen brightness in the preset multimodal feature vector to derive a confidence value for the preset quality inspection indicator state belonging to this compound abnormality. This calculation is repeated for all compound abnormal quality inspection conditions to form a state confidence distribution. Based on the state confidence distribution and the relationship topology, the server then performs feature embedding on the preset category feature vector for each matching indicator state category. Taking "inaccurate heart rate monitoring" as an example, in the relationship topology of "inaccurate heart rate monitoring and screen brightness adjustment failure", it is determined as the target category object, and combined with the state confidence distribution, such as the credible value of the composite anomaly, and the mutual influence relationship with "screen brightness adjustment failure", the associated feature vector is determined, and then the features are merged to obtain the preset depth category feature vector. Next, based on the preset depth category feature vector corresponding to each indicator state category to be matched, the server determines the matching coefficient of each indicator state category to be matched belonging to the preset quality inspection indicator state. The preset depth category feature vector of "inaccurate heart rate monitoring" is combined with the relevant features of the preset quality inspection indicator state, and the matching coefficient is calculated through a specific algorithm to reflect the degree of matching between "inaccurate heart rate monitoring" and the current preset quality inspection indicator state.Finally, based on the matching coefficient and the preset indicator status category, the server updates the network parameters in the quality inspection status recognition network. If the matching coefficient does not match the preset indicator status category (for example, "inaccurate heart rate monitoring"), it indicates a bias in the network prediction. The server then adjusts the network's weights, biases, and other parameters using techniques such as backpropagation. By repeating these steps multiple times and continuously optimizing the network until the matching coefficient output by the network closely matches the preset indicator status category, the trained quality inspection status recognition network is obtained, enabling it to more accurately monitor the quality of smart bracelet production.

[0065] In an embodiment of the present invention, updating the network parameters in the quality inspection state recognition network according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state to obtain the trained quality inspection state recognition network can be implemented through the following examples.

[0066] determining a cost parameter of the quality inspection state identification network according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state;

[0067] When the cost parameter does not reach a convergence state, updating the network parameters in the quality inspection state identification network based on the cost parameter;

[0068] According to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition, each preset quality inspection indicator state is divided to obtain a preset quality inspection indicator state group corresponding to each composite abnormal quality inspection condition;

[0069] For each compound abnormal quality inspection condition, based on the preset indicator state categories of the preset quality inspection indicator states in the preset quality inspection indicator state group corresponding to the compound abnormal quality inspection condition, the mutual influence relationship of each to-be-matched indicator state category in the relationship topology structure corresponding to the compound abnormal quality inspection condition is optimized to obtain the optimized relationship topology structure corresponding to the compound abnormal quality inspection condition;

[0070] Based on the updated quality inspection state recognition network and the optimized relational topology corresponding to each composite abnormal quality inspection condition, the steps of extracting state features of the preset quality inspection indicator state and each indicator state category to be matched are executed cyclically through the quality inspection state recognition network until the cost parameter of the quality inspection state recognition network reaches a convergence state, thereby obtaining a trained quality inspection state recognition network.

[0071] In an embodiment of the present invention, the server monitors the quality of the smart bracelet production process as an example. The server first determines the cost parameter of the quality inspection status identification network based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status. For example, if the actual preset indicator status category of a preset quality inspection indicator status is "abnormal exercise step record", and the matching coefficient of "abnormal exercise step record" calculated by the current network is low, it indicates that the network prediction deviates from the actual situation. The server uses a specific loss function (such as the cross-entropy loss function) to calculate the cost parameter based on this deviation. It measures the degree of difference between the network's current prediction result and the actual situation. When the cost parameter does not reach a convergence state, it means that the deviation between the network prediction result and the actual situation is still large. The server updates the network parameters in the quality inspection status identification network based on the cost parameter. For example, using the gradient descent algorithm, the gradient of the parameters of each layer of the network (such as weights and biases) is calculated based on the cost parameter, and these parameters are adjusted in a direction that reduces the cost parameter, thereby improving the network's prediction ability. Next, the server divides each preset quality inspection indicator state according to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition. There are two composite abnormal quality inspection conditions: "abnormal exercise step recording and inaccurate heart rate monitoring" and "abnormal exercise step recording and screen brightness adjustment failure". For a certain preset quality inspection indicator state, if its confidence in "abnormal exercise step recording and inaccurate heart rate monitoring" is high, it will be divided into the preset quality inspection indicator state group corresponding to the composite abnormal quality inspection condition. For each composite abnormal quality inspection condition, the server optimizes the mutual influence relationship of each to-be-matched indicator state category in the relationship topology structure based on the preset indicator state category of the preset quality inspection indicator state in its corresponding preset quality inspection indicator state group. For example, in the preset quality inspection indicator state group of "abnormal exercise step recording and inaccurate heart rate monitoring", if more preset quality inspection indicator states simultaneously show abnormalities in exercise step and heart rate monitoring, and there is a specific correlation pattern, such as when the exercise step number fluctuates greatly, the heart rate monitoring often deviates, the server will adjust the mutual influence relationship between the two indicator state categories to be matched in the relationship topology structure accordingly, and obtain an optimized relationship topology structure. Afterwards, based on the updated quality inspection state recognition network and the optimized relationship topology structure, the server cyclically executes the steps of extracting state features for the preset quality inspection indicator state and each indicator state category to be matched in turn through the quality inspection state recognition network. The preset multimodal feature vector of the preset quality inspection indicator state and the preset category feature vector of the indicator state category to be matched are re-extracted, and then the state confidence distribution, feature embedding processing, and matching coefficient determination are calculated.These steps are repeated continuously, each time calculating the cost parameters based on the new matching coefficient and preset indicator state category, updating the network parameters, and optimizing the relationship topology structure until the cost parameters of the quality inspection state recognition network reach a convergence state. At this time, the trained quality inspection state recognition network is obtained, which can more accurately monitor and judge the quality status of the smart bracelet production process.

[0072] In an embodiment of the present invention, each preset quality inspection indicator state is divided according to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition to obtain a preset quality inspection indicator state group corresponding to each composite abnormal quality inspection condition, which can be implemented through the following examples.

[0073] According to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection state, determining the preset composite abnormal quality inspection state corresponding to the preset quality inspection indicator state from each composite abnormal quality inspection state;

[0074] Based on the preset composite abnormal quality inspection status corresponding to each preset quality inspection indicator status, each preset quality inspection indicator status is classified and processed to obtain a preset quality inspection indicator status group corresponding to each composite abnormal quality inspection status.

[0075] In an embodiment of the present invention, for example, in the production process of smart watches, there are a variety of composite abnormal quality inspection conditions, such as "abnormal screen display and short battery life", "inaccurate heart rate monitoring and unstable Bluetooth connection", etc. The server has obtained the state confidence distribution of each preset quality inspection indicator state belonging to each composite abnormal quality inspection condition. The first step is to determine the preset composite abnormal quality inspection condition corresponding to the preset quality inspection indicator state from each composite abnormal quality inspection condition based on the state confidence distribution. One of the preset quality inspection indicator states includes screen brightness fluctuation data, battery power consumption rate data, heart rate monitoring deviation data, and Bluetooth connection interruption number data. For the composite abnormal quality inspection condition of "abnormal screen display and short battery life", the server calculates the state confidence distribution based on the screen and battery related data in the preset quality inspection indicator state, and obtains a confidence value of 0.7 that the preset quality inspection indicator state belongs to this composite abnormal quality inspection condition; for "inaccurate heart rate monitoring and unstable Bluetooth connection", the confidence value is 0.3. Since 0.7 is greater than 0.3, the server determines that "abnormal screen display and short battery life" is the preset composite abnormal quality inspection condition corresponding to this preset quality inspection indicator state. In the second step, each preset quality inspection indicator state is classified based on the preset composite abnormal quality inspection condition corresponding to each preset quality inspection indicator state, obtaining a preset quality inspection indicator state group corresponding to each composite abnormal quality inspection condition. The server processed 100 preset quality inspection indicator states and determined the corresponding preset composite abnormal quality inspection condition for each preset quality inspection indicator state using the method in the first step. For example, if there are 30 preset quality inspection indicator states corresponding to the preset composite abnormal quality inspection condition of "abnormal screen display and short battery life," the server will group these 30 preset quality inspection indicator states together to form a preset quality inspection indicator state group corresponding to the composite abnormal quality inspection condition of "abnormal screen display and short battery life." Another 20 preset quality inspection indicator states correspond to the preset composite abnormal quality inspection condition of "inaccurate heart rate monitoring and unstable Bluetooth connection." These 20 preset quality inspection indicator states are grouped together to form a preset quality inspection indicator state group corresponding to "inaccurate heart rate monitoring and unstable Bluetooth connection." Through this classification process, the server constructs a corresponding preset quality inspection indicator state group for each composite abnormal quality inspection condition. These groups provide the data foundation for the subsequent optimization of the relationship topology structure for different composite abnormal quality inspection conditions, helping to improve the accuracy of the quality inspection state recognition network in monitoring the production quality of smart watches.

[0076] In an embodiment of the present invention, the preset indicator state categories of the preset quality inspection indicator states in the preset quality inspection indicator state group corresponding to the compound abnormal quality inspection condition are optimized to optimize the mutual influence relationship of each indicator state category to be matched in the relationship topology structure corresponding to the compound abnormal quality inspection condition, and obtain the optimized relationship topology structure corresponding to the compound abnormal quality inspection condition, which can be implemented through the following examples.

[0077] Based on the preset indicator state categories of the preset quality inspection indicator states in the preset quality inspection indicator state group corresponding to the compound abnormal quality inspection condition, determining a preset quality inspection indicator state group for each indicator state category to be matched in the preset quality inspection indicator state group corresponding to the compound abnormal quality inspection condition, wherein the preset quality inspection indicator state group for each indicator state category to be matched includes the preset quality inspection indicator state corresponding to the corresponding indicator state category to be matched;

[0078] For each indicator state category to be matched, the preset quality inspection indicator states in the preset quality inspection indicator state group of the indicator state category to be matched are analyzed to determine that when the preset quality inspection indicator states include the indicator state category to be matched under the compound abnormal quality inspection condition, the preset quality inspection indicator states also include target credibility values of the remaining indicator state categories to be matched, and the target credibility values reflect the mutual influence relationship between the indicator state category to be matched and the remaining indicator state categories to be matched;

[0079] Based on the target credibility value, the mutual influence relationship of each to-be-matched indicator state category in the relationship topology structure corresponding to the composite abnormal quality inspection situation is optimized to obtain an optimized relationship topology structure corresponding to the composite abnormal quality inspection situation.

[0080] In an embodiment of the present invention, for example, a server monitors the quality of a smart speaker during production. Consider a composite abnormal quality inspection condition, characterized by "abnormal sound quality and unstable connection." First, the server determines a preset quality inspection indicator status group for each indicator status category to be matched, based on the preset indicator status categories of the preset quality inspection indicator statuses in the preset quality inspection indicator status group corresponding to this composite abnormal quality inspection condition. The preset quality inspection indicator status group contains actual test data from multiple smart speakers, and its preset indicator status categories include "noisy sound quality," "distorted volume," "disconnected Bluetooth connection," and "slow Wi-Fi connection." The server groups all preset quality inspection indicator statuses belonging to the "noisy sound quality" category into a group, representing the preset quality inspection indicator status group for the "noisy sound quality" indicator status category to be matched. Similarly, the server groups the preset quality inspection indicator statuses corresponding to each indicator status category to be matched, such as "distorted volume," "disconnected Bluetooth connection," and "slow Wi-Fi connection," into corresponding groups. Next, for each indicator status category to be matched, the server analyzes the preset quality inspection indicator status group and determines a target credibility value. Taking "noise" as an example, the server examined the status of the pre-set quality control indicators in this group and found that when "noise" occurred on a smart speaker, "Bluetooth connection interruption" also occurred in 70% of the data. Therefore, for the combined abnormal quality control condition of "abnormal sound quality and unstable connection," the target credibility value between "noise" and "Bluetooth connection interruption" is 0.7, reflecting the mutual influence between the two. Similarly, for "noise" and "slow Wi-Fi connection," if the probability of simultaneous occurrence is 30%, the target credibility value is 0.3. Finally, based on these target credibility values, the server optimizes the relationship topology corresponding to the combined abnormal quality control condition of "abnormal sound quality and unstable connection." In the original relationship topology, the path weight between "noise" and "Bluetooth connection interruption" is 0.5, and the path weight between "noise" and "slow Wi-Fi connection" is 0.2. Now, based on the target credibility value, the path weights for "noisy sound quality" and "interrupted Bluetooth connection" are adjusted to 0.7, and the path weights for "noisy sound quality" and "slow Wi-Fi connection" are adjusted to 0.3. By adjusting the path weights (i.e., the mutual influence relationships) between the various indicator status categories to be matched, an optimized relationship topology is obtained. This optimized relationship topology can more accurately reflect the true mutual influence relationships between the various indicator status categories to be matched for the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection," helping the server to subsequently use the quality inspection status identification network to more accurately determine quality issues in the smart speaker production process.

[0081] In an embodiment of the present invention, for each indicator state category to be matched, the preset quality inspection indicator state in the preset quality inspection indicator state group of the indicator state category to be matched is analyzed to determine that when the preset quality inspection indicator state includes the indicator state category to be matched under the composite abnormal quality inspection condition, the preset quality inspection indicator state also includes the target credible values of the remaining indicator state categories to be matched, which can be implemented through the following examples.

[0082] For each indicator status category to be matched, counting the basic number of preset quality inspection indicator statuses in the preset quality inspection indicator status group of the indicator status category to be matched;

[0083] In the preset quality inspection indicator state group of the indicator state category to be matched, determining the number of advanced preset quality inspection indicator states that also include the remaining indicator state categories to be matched;

[0084] Based on the basic quantity and the advanced quantity, when it is determined that the preset quality inspection indicator state includes the indicator state category to be matched under the compound abnormal quality inspection condition, the preset quality inspection indicator state also includes target credible values of other indicator state categories to be matched.

[0085] In this embodiment of the present invention, taking the server's quality monitoring of the smart speaker production process as an example, we focus on the compound abnormal quality inspection condition of "abnormal sound quality and unstable connection." The preset quality inspection indicator status group corresponding to this compound abnormal quality inspection condition contains test data from numerous smart speakers, where the indicator status categories to be matched include "noisy sound quality," "distorted volume," "disconnected Bluetooth connection," and "slow Wi-Fi connection." First, for each indicator status category to be matched, the server counts the base number of pre-set quality inspection indicator statuses in its pre-set quality inspection indicator status group. For the "noisy sound quality" indicator status category to be matched, the server finds all data records in the pre-set quality inspection indicator status group that are determined to contain "noisy sound quality." The count shows a total of 100 such records, which constitute the base number of pre-set quality inspection indicator statuses for "noisy sound quality." Next, within the pre-set quality inspection indicator status group for "noisy sound quality," the server determines the advanced number of pre-set quality inspection indicator statuses that also contain the remaining pre-set quality inspection indicator status categories to be matched. For example, among the 100 records with "noisy sound quality," the server further screened for records that also included "Bluetooth connection interruption," finding 40 such records. These 40 records represent the advanced count for the remaining indicator status category that also included "Bluetooth connection interruption." Similarly, for "slow Wi-Fi connection," the server also searched for records that also included "slow Wi-Fi connection" among the 100 records with "noisy sound quality," finding 20 such records. These 20 records represent the advanced count for the remaining indicator status category that also included "slow Wi-Fi connection." Finally, based on the base count and the advanced count, the server determined the target credibility value. For "sound quality noise" and "Bluetooth connection interruption", the target credibility value is calculated by dividing the advanced number by the basic number, that is, (40÷100=0.4). This 0.4 is the target credibility value of "Bluetooth connection interruption" when the preset quality inspection indicator status includes "sound quality noise" under the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection", reflecting the degree of mutual influence between "sound quality noise" and "Bluetooth connection interruption". For "sound quality noise" and "slow Wi-Fi connection", the target credibility value is (20÷100=0.2), indicating the degree of mutual influence between these two indicator status categories to be matched. In this way, the server determines the corresponding target credibility value for each indicator status category to be matched and the remaining indicator status categories to be matched, providing accurate data support for the subsequent optimization of the relationship topology structure, so that the relationship topology structure can more accurately reflect the true degree of correlation between the various indicator status categories to be matched under the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection" of the smart speaker.

[0086] In the embodiment of the present invention, the acquisition of training samples and the relationship topology structure corresponding to each composite abnormal quality inspection condition can be implemented through the following examples.

[0087] Acquire a training sample, and determine a preset quality inspection indicator state group for each indicator state category to be matched in the training sample based on the preset indicator state category of the preset quality inspection indicator state in the training sample, wherein the preset quality inspection indicator state group for each indicator state category to be matched includes the preset quality inspection indicator state of the corresponding indicator state category to be matched in the training sample;

[0088] For each indicator state category to be matched, analyzing the preset quality inspection indicator states in the preset quality inspection indicator state group of the indicator state category to be matched to determine that when the preset quality inspection indicator states include the indicator state category to be matched, the preset quality inspection indicator state also includes the credibility values of the remaining indicator state categories to be matched, and the credibility values reflect the mutual influence relationship between the indicator state category to be matched and the remaining indicator state categories to be matched;

[0089] Based on the credible value, the mutual influence relationship between the category objects corresponding to the indicator status category to be matched in the relationship topology structure of each composite abnormal quality inspection situation is set to obtain the relationship topology structure corresponding to each composite abnormal quality inspection situation after setting.

[0090] In an embodiment of the present invention, these steps are described in detail using the example of a server performing quality monitoring on the production process of an intelligent sweeping robot. First, the server obtains a training sample. This training sample contains a large amount of production data for intelligent sweeping robots, covering preset quality inspection indicator states for various aspects such as cleaning performance, navigation accuracy, and battery life, as well as corresponding preset indicator state categories, such as "poor cleaning," "large navigation deviation," and "short battery life." Next, based on the preset indicator state categories, the server determines a preset quality inspection indicator state group for each indicator state category to be matched from the training sample. For example, for the indicator state category to be matched, the server filters all intelligent sweeping robot production data related to "poor cleaning" from the training sample to form a preset quality inspection indicator state group for "poor cleaning." Similarly, corresponding preset quality inspection indicator state groups are determined for other indicator state categories to be matched, such as "large navigation deviation" and "short battery life." Next, for each indicator status category to be matched, the server analyzes the preset quality inspection indicator statuses in its preset quality inspection indicator status group to determine a credibility value. Taking "poor cleaning" as an example, the server counts the number of samples in the preset quality inspection indicator status group for "poor cleaning" that also exhibit "large navigation deviation." If there are 100 samples in the preset quality inspection indicator status group for "poor cleaning," and 30 of them also exhibit "large navigation deviation," the credibility value between "poor cleaning" and "large navigation deviation" is 30 ÷ 100 = 0.3. This indicates that when the smart sweeping robot exhibits "poor cleaning," there is a 30% probability that "large navigation deviation" will also occur, reflecting the mutual influence relationship between these two indicator status categories to be matched. Similarly, the server calculates the credibility value between "poor cleaning" and "short battery life," as well as the credibility values between the other indicator status categories to be matched. Finally, based on these credibility values, the server sets the mutual influence relationship between the category objects corresponding to the indicator status categories to be matched in the relationship topology structure for each composite abnormal quality inspection condition. There is a compound abnormal quality inspection condition called "poor cleaning and large navigation deviation." In its relational topology, "poor cleaning" and "large navigation deviation" are two category objects. Based on the previously calculated credibility value of 0.3, the server sets the mutual influence relationship between these two category objects. For example, in the relational topology graph, the edge connecting the two nodes "poor cleaning" and "large navigation deviation" is assigned a weight of 0.3, indicating the degree of mutual influence between them. For other compound abnormal quality inspection conditions, such as "poor cleaning and short battery life" and "large navigation deviation and short battery life," the server also follows a similar method to set the mutual influence relationship between the category objects in the relational topology based on the corresponding credibility values, thereby obtaining the relational topology structure corresponding to each compound abnormal quality inspection condition after the setting.These relationship topologies will provide an important basis for the subsequent training of the quality inspection status recognition network, helping the server to more accurately monitor the production quality of the intelligent sweeping robot.

[0091] In an embodiment of the present invention, for each indicator state category to be matched, the preset quality inspection indicator state in the preset quality inspection indicator state group of the indicator state category to be matched is analyzed to determine that when the preset quality inspection indicator state includes the indicator state category to be matched, the preset quality inspection indicator state also includes the credible values of the remaining indicator state categories to be matched, which can be implemented through the following examples.

[0092] For each indicator state category to be matched, counting the basic number of preset quality inspection indicator states in the preset quality inspection indicator state group of the indicator state category to be matched;

[0093] In the preset quality inspection indicator state group of the indicator state category to be matched, determining the number of advanced preset quality inspection indicator states that also include the remaining indicator state categories to be matched;

[0094] Based on the basic quantity and the advanced quantity, it is determined that when the preset quality inspection indicator state includes the indicator state category to be matched, the preset quality inspection indicator state also includes credible values of other indicator state categories to be matched.

[0095] In this embodiment of the present invention, the server's quality monitoring of drone production is used as an example to illustrate this process. The server obtains a large number of training samples for drones. These samples cover preset quality inspection indicator states for various aspects, such as flight stability, image quality, and battery life, as well as corresponding preset indicator state categories, such as "flight jitter," "blurry image," and "inadequate battery life." For each indicator state category to be matched, the server first counts the base number of preset quality inspection indicator states in its preset quality inspection indicator state group. Taking the "flight jitter" indicator state category as an example, the server selects all drone data records indicating "flight jitter" from the training samples. A careful count reveals a total of 200 such records, which constitute the base number of preset quality inspection indicator state groups for "flight jitter." Next, within the preset quality inspection indicator state group for "flight jitter," the server determines the advanced number of preset quality inspection indicator states that also include the remaining indicator state categories to be matched. For example, the server searches for records with a "blurred image" issue among the 200 "flight jitter" records. After checking each one, it finds 50 records with the "blurred image" issue. These 50 records represent the number of advanced levels that also include "blurred image" in the other indicator status category to be matched. Similarly, for "insufficient battery life," the server counts 30 records with the "inadequate battery life" issue among the 200 "flight jitter" records. These 30 records represent the number of advanced levels that also include "inadequate battery life." Finally, based on the base number and the number of advanced levels, the server determines a confidence value. For "flight jitter" and "blurred image," the confidence value is calculated by dividing the number of advanced levels by the base number: 50 ÷ 200 = 0.25. This means that when the drone experiences "flight jitter," there is a 25% chance that "blurred image" will also occur. This 0.25 represents the confidence value between the two indicator status categories to be matched, reflecting the degree of mutual influence between "flight jitter" and "blurred image." For "flight jitter" and "inadequate battery life," the confidence value is 30 ÷ 200 = 0.15, indicating the degree of mutual influence between these two to-be-matched indicator status categories. This allows the server to determine the corresponding confidence value for each to-be-matched indicator status category relative to the remaining to-be-matched indicator status categories. This provides accurate data support for the subsequent establishment of the mutual influence relationships between category objects in the composite abnormal quality inspection condition relationship topology, helping the server to more effectively monitor and analyze drone production quality.

[0096] In a possible implementation, determining the cost parameter of the quality inspection status identification network according to the matching coefficient and the preset indicator status category of the preset quality inspection indicator status can be implemented through the following example.

[0097] Calculating a category cost parameter according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state;

[0098] Calculating the distribution uncertainty of the preset quality inspection indicator state based on the state confidence distribution that the preset quality inspection indicator state belongs to each composite abnormal quality inspection condition;

[0099] Determining, based on the state confidence distribution that the preset quality inspection indicator state belongs to each composite abnormal quality inspection state, the preset composite abnormal quality inspection state corresponding to the preset quality inspection indicator state, and determining, based on the preset composite abnormal quality inspection state corresponding to each preset quality inspection indicator state, the number of training samples corresponding to each composite abnormal quality inspection state;

[0100] Calculate the distribution cost parameter based on the distribution uncertainty of each preset quality inspection indicator state and the number of training samples corresponding to each composite abnormal quality inspection condition;

[0101] Based on the category cost parameter and the distribution cost parameter, a cost parameter of the quality inspection state identification network is determined.

[0102] In an embodiment of the present invention, illustratively, taking the server's quality monitoring of the production process of smart cameras as an example, the process of determining the cost parameters of the quality inspection status identification network is described in detail. The server is processing the production data of the smart camera. The preset quality inspection index status includes data on image clarity, night vision effect, network connection stability, etc. The preset index status categories include "image blur", "night vision function abnormality", "network connection interruption", etc. The server has obtained the matching coefficients of each preset quality inspection index state and each index state category to be matched through the quality inspection status identification network. For each preset quality inspection index state, the server calculates the category cost parameter based on its matching coefficient and the preset index state category. For example, the actual preset index state category of a preset quality inspection index state is "image blur", and the "image blur" matching coefficient given by the quality inspection state identification network is 0.6. The server uses a common loss function, such as the cross-entropy loss function, and substitutes the actual category ("image blur") and the predicted matching coefficient (0.6) into the calculation. The formula for the cross-entropy loss function is: Among them, y i represents the actual category (1 represents “image blur”, 0 represents other categories), p i Represents the predicted matching coefficient. Here, y "图像模糊” =1, p "图像模糊”=0.6, then the cost parameter of the preset quality inspection indicator state with respect to the "image blur" category is calculated. This calculation is performed for all preset quality inspection indicator states, and then the total category cost parameter is summarized. The server calculates the distribution uncertainty based on the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition. For example, there are composite abnormal quality inspection conditions such as "image blur and abnormal night vision function" and "image blur and network connection interruption". For a certain preset quality inspection indicator state, the confidence that it belongs to "image blur and abnormal night vision function" is 0.7, and the confidence that it belongs to "image blur and network connection interruption" is 0.3. The server uses methods such as information entropy to measure the uncertainty of this distribution. This calculation is performed for all preset quality inspection indicator states and summarized. Based on the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition, the server determines the preset composite abnormal quality inspection condition corresponding to each preset quality inspection indicator state. For example, if the confidence level of a preset quality inspection indicator state belonging to "image blur and abnormal night vision function" is the highest, then "image blur and abnormal night vision function" is its corresponding preset composite abnormal quality inspection state. Then, based on the preset composite abnormal quality inspection state corresponding to each preset quality inspection indicator state, the server counts the number of training samples corresponding to each composite abnormal quality inspection state. Among all the training samples, there are 100 corresponding to "image blur and abnormal night vision function" and 80 corresponding to "image blur and network connection interrupted". The server calculates the distribution cost parameter based on the distribution uncertainty of each preset quality inspection indicator state and the number of training samples corresponding to each composite abnormal quality inspection state. For example, the sum of the distribution uncertainty corresponding to "image blur and abnormal night vision function" is 1.2 (obtained by summing the distribution uncertainty of all preset quality inspection indicator states under the composite abnormal quality inspection state), and the number of training samples is 100; the sum of the distribution uncertainty corresponding to "image blur and network connection interrupted" is 1.0, and the number of training samples is 80. The server may use weighted average and other methods to calculate the distribution cost parameter, and the calculation formula is Among them H k is the sum of the distribution uncertainties of each composite abnormal quality inspection condition, n k is the number of training samples for each composite abnormal quality inspection condition. Substitute the data to calculate the distribution cost parameter. Finally, the server determines the cost parameter of the quality inspection state recognition network based on the category cost parameter and the distribution cost parameter. For example, if the category cost parameter is 2.5 and the distribution cost parameter is 1.8, the server can use a weighted addition method (the weighting coefficients are w1=0.6 and w2=0.4 respectively), that is, the cost parameter C=w1×C c +w2×C d =0.6×2.5+0.4×1.8=2.02, where 2.02 is the final cost parameter of the quality inspection status recognition network, which is used for subsequent updating and optimization of network parameters.

[0103] In a possible implementation, the quality inspection state recognition network is used to extract state features of the preset quality inspection indicator state and each indicator state category to be matched in turn to obtain a preset multimodal feature vector of the preset quality inspection indicator state and a preset category feature vector of each indicator state category to be matched. This can be implemented through the following examples.

[0104] Performing multimodal state characterization processing on the preset quality inspection indicator state through a quality inspection state recognition network to obtain a preset quality inspection indicator state quality inspection indicator feature of the preset quality inspection indicator state;

[0105] Through the quality inspection state recognition network, each state category of the indicator to be matched is subjected to multimodal state representation processing in turn to obtain the sample state category features of each state category of the indicator to be matched; based on the sample state category features of each state category of the indicator to be matched and the adaptive and plastic industry prior knowledge vector, a feature set is constructed;

[0106] Performing cross-modal gating interaction on the preset quality inspection indicator state quality inspection indicator feature and the feature set to obtain a preset multimodal feature vector of the preset quality inspection indicator state and a preset category feature vector of each indicator state category to be matched;

[0107] The updating of network parameters in the quality inspection state recognition network according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state to obtain the trained quality inspection state recognition network can be implemented through the following examples.

[0108] According to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network and the industry prior knowledge vector are updated to obtain a trained quality inspection state recognition network.

[0109] In an embodiment of the present invention, the server's quality monitoring of a smartwatch's production process is used as an example to illustrate this process. The smartwatch's preset quality inspection indicator states cover multiple aspects of data, including heart rate monitoring accuracy, motion trajectory recording accuracy, and screen touch sensitivity. The indicator state categories to be matched include "large heart rate monitoring deviation," "inaccurate motion trajectory recording," and "unresponsive screen touch." The server utilizes a quality inspection state recognition network to perform multimodal state representation processing on the preset quality inspection indicator states. For example, the network converts heart rate monitoring accuracy data into a vector representation that reflects characteristics such as the heart rate fluctuation range and the degree of deviation from the standard value, thereby obtaining the quality inspection indicator characteristics of the preset quality inspection indicator state in the heart rate monitoring dimension. For motion trajectory recording accuracy data, the network converts it into a vector representation that reflects characteristics such as the degree of trajectory deviation and positioning error. Similarly, by integrating the data processing results from each dimension, the server obtains the quality inspection indicator characteristics of the preset quality inspection indicator state. Furthermore, the server utilizes adaptive and malleable industry prior knowledge vectors. For example, standards and experience in the smartwatch industry regarding heart rate monitoring, motion trajectory recording, and screen touch are converted into vectors. Then, a feature set is constructed based on the sample state category features of each indicator state category to be matched and the industry prior knowledge vector. The server performs cross-modal gated interaction between the preset quality inspection indicator state features and the feature set. First, the quality inspection indicator features of the heart rate monitoring dimension are selected and interacted with the elements in the feature set. By analyzing the relationship between the heart rate features and state category features such as "large heart rate monitoring deviation" and the industry prior knowledge vector, a complex algorithm is applied to obtain the graph convolution feature vector of the preset quality inspection indicator state in the heart rate monitoring dimension, as well as the graph convolution feature vector for each indicator state category to be matched, such as "large heart rate monitoring deviation." The steps of selecting and interacting quality inspection indicator features from different dimensions are repeated until all dimensions are processed. Based on the graph convolution feature vectors obtained each time, the preset multimodal feature vector of the preset quality inspection indicator state and the preset category feature vector for each indicator state category to be matched are finally obtained. The server updates the network parameters and industry prior knowledge vector in the quality inspection state recognition network based on the matching coefficient and the preset indicator state category of the preset quality inspection indicator state. For example, if a preset quality inspection indicator status is actually "large heart rate monitoring deviation," but the network's matching coefficient is low, this indicates an inaccurate prediction. The server uses techniques such as backpropagation to adjust network parameters such as weights and biases to make the network's prediction more accurate. Furthermore, if the correlation between heart rate monitoring deviation and inaccurate motion trajectory recording in actual production differs from that reflected in the original industry prior knowledge vector, the server will adjust the industry prior knowledge vector based on the data in the current training sample to better reflect actual production conditions. After multiple such updates, a trained quality inspection status recognition network is ultimately obtained, enabling more accurate quality monitoring of smartwatch production.

[0110] The embodiment of the present invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned production process quality intelligent monitoring method based on deep learning. Figure 2 As shown, Figure 2 This is a block diagram of the structure of a computer device 100 provided in an embodiment of the present invention. Computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or exchange, memory 111, processor 112, and communication unit 113 are electrically connected to each other, directly or indirectly. For example, these components can be electrically connected via one or more communication buses or signal lines.

[0111] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the above illustrative discussion is not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in light of the above teachings. These embodiments have been selected and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the present disclosure and to utilize various embodiments with various modifications as appropriate for the specific application contemplated.

Claims

1. A method for intelligent monitoring of production process quality based on deep learning, characterized by: include: Obtaining a multimodal quality inspection indicator state and multiple indicator state categories to be matched for a target product, and extracting state features for the multimodal quality inspection indicator state and each indicator state category to be matched in turn, to obtain a multimodal feature vector for the multimodal quality inspection indicator state and a category feature vector for each indicator state category to be matched; Calculating, based on the multimodal feature vector, a state confidence distribution of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition; Obtaining a relationship topology structure corresponding to each composite abnormal quality inspection condition, wherein the relationship topology structure reflects the mutual influence relationship between the status categories of each to-be-matched indicator under the composite abnormal quality inspection condition; Based on the state confidence distribution and the relationship topological structure corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the category feature vector of each to-be-matched indicator state category to obtain a deep category feature vector corresponding to each to-be-matched indicator state category; Determining, based on the depth category feature vector corresponding to each to-be-matched indicator state category, a matching coefficient for each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state; According to the matching coefficient, a target indicator state category corresponding to the multimodal quality inspection indicator state is determined from each of the indicator state categories to be matched as a quality monitoring result of the target product.

2. The method according to claim 1, characterized in that The state feature extraction is performed on the multimodal quality inspection indicator state and each to-be-matched indicator state category in sequence to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each to-be-matched indicator state category, including: Performing multimodal state characterization processing on the quality inspection indicator data of the multimodal quality inspection indicator state in at least one quality inspection dimension to obtain a quality inspection indicator feature of the multimodal quality inspection indicator state in at least one quality inspection dimension; Perform multimodal state representation processing on each state category of the indicator to be matched in turn to obtain the state category features of each state category of the indicator to be matched; Construct a feature set based on the state category features of each indicator state category to be matched and the industry prior knowledge vector; The quality inspection indicator feature and the feature set are subjected to cross-modal gated interaction to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each indicator state category to be matched.

3. The method according to claim 2, characterized in that The cross-modal gating interaction between the quality inspection indicator feature and the feature set to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched includes: Determine the quality inspection indicator characteristics of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator characteristics of the at least one quality inspection dimension; Performing cross-modal gated interaction on the quality inspection indicator features of the target quality inspection dimension and the feature set to obtain a graph convolution feature vector of the multimodal quality inspection indicator state and a graph convolution feature vector of each indicator state category to be matched; Determine a current feature set based on the graph convolution feature vector of the multimodal quality inspection indicator state and the graph convolution feature vector of each indicator state category to be matched; The step of determining the quality inspection indicator features of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of the at least one quality inspection dimension is cyclically executed until the quality inspection indicator features of all quality inspection dimensions are completed, and based on the current feature set, the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched are obtained.

4. The method according to claim 1, wherein The relationship topology structure includes a category object corresponding to each indicator state category to be matched, and a path between category objects, wherein the path reflects the mutual influence relationship between two connected category objects; The feature embedding process is performed on the category feature vector of each to-be-matched indicator state category based on the relationship topological structure corresponding to the state confidence distribution and each composite abnormal quality inspection condition to obtain a deep category feature vector corresponding to each to-be-matched indicator state category, including: For each to-be-matched indicator status category, determine the target category object corresponding to the to-be-matched indicator status category in the relationship topology structure corresponding to each composite abnormal quality inspection condition; For each compound abnormal quality inspection condition corresponding to the relationship topology structure, based on the mutual influence relationship between the target category object and the other category objects in the relationship topology structure, determine the associated feature vector of the target category object under the compound abnormal quality inspection condition; Based on the state confidence distribution, the associated feature vectors of the target category objects under each composite abnormal quality inspection condition are merged to obtain a merged feature vector of the state category of the indicator to be matched; Based on the combined feature vector, the category feature vector of the to-be-matched indicator state category is optimized to obtain a depth category feature vector corresponding to the to-be-matched indicator state category.

5. The method according to claim 4, characterized in that The state confidence distribution includes a credible value that the state of the multimodal quality inspection indicator belongs to each composite abnormal quality inspection condition; Based on the state confidence distribution, the associated feature vectors of the target category objects under each compound abnormal quality inspection condition are merged to obtain the merged feature vector of the state category of the indicator to be matched, including: For each composite abnormal quality inspection condition, the credible value of the multimodal quality inspection indicator state belonging to the composite abnormal quality inspection condition is merged with the associated feature vector of the target category object under the composite abnormal quality inspection condition to obtain the basic merged feature of the indicator state category to be matched under the composite abnormal quality inspection condition; Based on the basic combined features of the to-be-matched indicator state category under each compound abnormal quality inspection condition, a combined feature vector of the to-be-matched indicator state category is determined.

6. The method according to claim 1, characterized in that The determining, based on the depth category feature vector corresponding to each to-be-matched indicator state category, that each to-be-matched indicator state category belongs to the multimodal quality inspection indicator state, includes: For each indicator state category to be matched, the category feature vector and the depth category feature vector of the indicator state category to be matched are merged to obtain the target category feature vector of the indicator state category to be matched; Based on the target category feature vector of each to-be-matched indicator state category, a matching coefficient of each to-be-matched indicator state category belonging to the multimodal quality inspection indicator state is determined.

7. The method according to claim 1, characterized in that The state feature extraction is performed on the multimodal quality inspection indicator state and each to-be-matched indicator state category in sequence to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each to-be-matched indicator state category, including: Through the quality inspection state recognition network, state feature extraction is performed on the multimodal quality inspection indicator state and each indicator state category to be matched in turn to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each indicator state category to be matched.

8. The method according to claim 7, characterized in that Before extracting state features of the multimodal quality inspection indicator state and each to-be-matched indicator state category in turn through the quality inspection state recognition network to obtain a multimodal feature vector of the multimodal quality inspection indicator state and a category feature vector of each to-be-matched indicator state category, the method further includes: Acquire a training sample and a relationship topology structure corresponding to each composite abnormal quality inspection condition, wherein the training sample includes a plurality of preset quality inspection indicator states and preset indicator state categories of the preset quality inspection indicator states; By using a quality inspection state recognition network, state feature extraction is performed on the preset quality inspection indicator state and each to-be-matched indicator state category in turn to obtain a preset multimodal feature vector of the preset quality inspection indicator state and a preset category feature vector of each to-be-matched indicator state category; Calculating, based on the preset multimodal feature vector, a state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition; Based on the state confidence distribution and the relationship topological structure corresponding to each composite abnormal quality inspection condition, feature embedding processing is performed on the preset category feature vector of each to-be-matched indicator state category to obtain a preset depth category feature vector corresponding to each to-be-matched indicator state category; Determining a matching coefficient for each to-be-matched indicator state category belonging to the preset quality inspection indicator state based on a preset depth category feature vector corresponding to each to-be-matched indicator state category; According to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network are updated to obtain a trained quality inspection state recognition network.

9. The method according to claim 8, characterized in that The updating of network parameters in the quality inspection state recognition network according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state to obtain a trained quality inspection state recognition network includes: determining a cost parameter of the quality inspection state identification network according to the matching coefficient and the preset indicator state category of the preset quality inspection indicator state; When the cost parameter does not reach a convergence state, updating the network parameters in the quality inspection state identification network based on the cost parameter; According to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition, each preset quality inspection indicator state is divided to obtain a preset quality inspection indicator state group corresponding to each composite abnormal quality inspection condition; For each compound abnormal quality inspection condition, based on the preset indicator state categories of the preset quality inspection indicator states in the preset quality inspection indicator state group corresponding to the compound abnormal quality inspection condition, the mutual influence relationship of each to-be-matched indicator state category in the relationship topology structure corresponding to the compound abnormal quality inspection condition is optimized to obtain the optimized relationship topology structure corresponding to the compound abnormal quality inspection condition; Based on the updated quality inspection state recognition network and the optimized relational topology corresponding to each composite abnormal quality inspection condition, the steps of extracting state features of the preset quality inspection indicator state and each indicator state category to be matched are executed cyclically through the quality inspection state recognition network until the cost parameter of the quality inspection state recognition network reaches a convergence state, thereby obtaining a trained quality inspection state recognition network.

10. A server system, characterized in that: The method comprises a server, wherein the server is configured to execute the method according to any one of claims 1 to 9.

Images