Deep Learning-Driven Intelligent Monitoring Method and System for Production Process Quality

Abstract

This invention discloses a deep learning-driven intelligent monitoring method and system for production process quality, relating to the field of artificial intelligence technology. The method includes: firstly, acquiring the multimodal quality inspection indicator status of the target product and multiple matching indicator status categories, and extracting state features to obtain corresponding feature vectors; calculating the state confidence distribution based on the multimodal feature vectors; acquiring the relational topology of each composite abnormal quality inspection status; combining the confidence distribution and topology to embed the category feature vectors to obtain deep category feature vectors; determining the matching coefficients accordingly; and finally determining the target indicator status category as the quality monitoring result based on the matching coefficients, thereby achieving intelligent monitoring of production process quality.

Description

Technical Field

[0001] This invention relates to the field of artificial intelligence, and more specifically, to a method and system for intelligent monitoring of production process quality based on deep learning. Background Technology

[0002] In the field of production process quality monitoring, traditional methods rely heavily on manual experience or simple data statistics, which are insufficient to handle complex and ever-changing production conditions and multi-dimensional quality inspection indicators. With the expansion of production scale and the increase in product complexity, this monitoring method is inefficient and inaccurate, failing to detect quality problems in a timely and precise manner. Although deep learning technology has been applied to some quality monitoring scenarios, its comprehensive processing of multimodal quality inspection indicators and the complex relationships between them still needs improvement. This invention aims to utilize deep learning technology to fully explore multimodal quality inspection indicator information and the relationships between indicator status categories, achieving intelligent, efficient, and accurate monitoring of production process quality. Summary of the Invention

[0003] The purpose of this invention is to provide a method and system for intelligent monitoring of production process quality based on deep learning.

[0004] In a first aspect, embodiments of the present invention provide a deep learning-driven intelligent monitoring method for production process quality, comprising:

[0005] The multimodal quality inspection index status and multiple matching index status categories of the target product are obtained, and the status features of the multimodal quality inspection index status and each matching index status category are extracted sequentially to obtain the multimodal feature vector of the multimodal quality inspection index status and the category feature vector of each matching index status category.

[0006] Based on the multimodal feature vector, calculate the state confidence distribution of the multimodal quality inspection index state belonging to each composite abnormal quality inspection condition;

[0007] Obtain the relational topology structure corresponding to each composite abnormal quality inspection status, wherein the relational topology structure reflects the mutual influence relationship between the status categories of each indicator to be matched under the composite abnormal quality inspection status.

[0008] Based on the state confidence distribution and the relational topology corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the category feature vector of each target indicator state category to obtain the deep category feature vector corresponding to each target indicator state category.

[0009] Based on the deep category feature vector corresponding to each state category of the indicator to be matched, the matching coefficient of each state category of the indicator to be matched to the state of the multimodal quality inspection indicator is determined.

[0010] Based on the matching coefficient, the target indicator state category corresponding to the multimodal quality inspection indicator state is determined from each of the indicator state categories to be matched as the quality monitoring result of the target product.

[0011] In a second aspect, embodiments of the present invention provide a server-based system, including a server, the server being configured to perform the method described in the first aspect.

[0012] Compared with existing technologies, the beneficial effects provided by this invention include: The method and system for intelligent monitoring of production process quality based on deep learning, as disclosed in this invention, acquires the multimodal quality inspection index status of the target product and multiple matching index status categories, extracts state features to obtain corresponding feature vectors; calculates the state confidence distribution based on the multimodal feature vectors; obtains the relational topology of each composite abnormal quality inspection status; combines the confidence distribution and topology to embed the category feature vectors to obtain deep category feature vectors; determines the matching coefficients accordingly; and finally determines the target index status category as the quality monitoring result based on the matching coefficients, thereby achieving intelligent monitoring of production process quality. Attached Figure Description

[0013] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the embodiments will be briefly described below. It should be understood that the following drawings only show some embodiments of the present invention and should not be considered as limiting the scope. For those skilled in the art, other related drawings can be obtained based on these drawings without creative effort.

[0014] Figure 1 A flowchart illustrating the steps of a deep learning-driven intelligent monitoring method for production process quality provided in an embodiment of the present invention.

[0015] Figure 2 A schematic block diagram of the structure of a computer device provided in an embodiment of the present invention. Detailed Implementation

[0016] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some, not all, of the embodiments of the present invention. The components of the embodiments of the present invention described and shown in the accompanying drawings can generally be arranged and designed in various different configurations.

[0017] The specific embodiments of the present invention will now be described in detail with reference to the accompanying drawings.

[0018] In order to solve the technical problems mentioned in the background art Figure 1This is a flowchart illustrating the intelligent monitoring method for production process quality based on deep learning, provided in an embodiment of this disclosure. The following is a detailed description of this intelligent monitoring method for production process quality based on deep learning.

[0019] Step S201: Obtain the multimodal quality inspection index status and multiple matching index status categories of the target product, and sequentially extract the status features of the multimodal quality inspection index status and each matching index status category to obtain the multimodal feature vector of the multimodal quality inspection index status and the category feature vector of each matching index status category.

[0020] Step S202: Based on the multimodal feature vector, calculate the state confidence distribution of the multimodal quality inspection index state belonging to each composite abnormal quality inspection condition;

[0021] Step S203: Obtain the relational topology structure corresponding to each composite abnormal quality inspection status. The relational topology structure reflects the mutual influence relationship between the status categories of each indicator to be matched under the composite abnormal quality inspection status.

[0022] Step S204: Based on the state confidence distribution and the relational topology structure corresponding to each composite abnormal quality inspection status, perform feature embedding processing on the category feature vector of each target indicator state category to obtain the deep category feature vector corresponding to each target indicator state category.

[0023] Step S205: Based on the deep category feature vector corresponding to each index state category to be matched, determine the matching coefficient of each index state category to be matched belonging to the multimodal quality inspection index state.

[0024] Step S206: Based on the matching coefficient, determine the target indicator state category corresponding to the multimodal quality inspection indicator state from each target indicator state category as the quality monitoring result of the target product.

[0025] In this embodiment of the invention, for example, the server obtains the multimodal quality inspection indicator status of the target product from various data sources such as sensors and testing equipment on the production line. For example, in an automobile manufacturing company, for the target product of an automobile engine, the multimodal quality inspection indicator status may cover engine temperature sensor data, vibration sensor data, fuel injection quantity data, etc., which reflect the engine's operating status from different dimensions. Simultaneously, the server also obtains multiple categories of indicator status to be matched, such as normal engine temperature, excessively high engine temperature, abnormal engine vibration, and fuel injection quantity deviation. These categories are pre-defined and used to match the actually obtained multimodal quality inspection indicator status. The server performs multimodal state characterization processing on the quality inspection indicator data across multiple quality inspection dimensions. Taking the automobile engine as an example again, for the temperature dimension, the server uses a specific algorithm to convert the temperature data collected in real time by the temperature sensor into a vector representation that reflects the temperature change trend and fluctuation; for the vibration dimension, it converts the vibration sensor data into a vector representing the vibration frequency, amplitude, and other characteristics. In this way, the quality inspection indicator characteristics of the multimodal quality inspection indicator status across each quality inspection dimension are obtained. For each category of indicators to be matched, the server also performs multimodal state representation processing. For example, for the category of "engine temperature too high," the server combines historical data and industry standards to construct a feature vector representing this category. This feature vector may include features such as temperature exceeding a specific threshold and the rate of temperature rise. The server constructs a feature set based on the state category features of each category of indicators to be matched and industry prior knowledge vectors. Industry prior knowledge vectors may come from long-term experience accumulated in the automotive manufacturing industry, such as the normal temperature range and permissible vibration range of engines under different operating conditions. This knowledge is transformed into vector form and combined with the state category features of each category of indicators to be matched to form a rich feature set. The server determines the quality inspection indicator features for the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of multiple quality inspection dimensions. In this cycle, the temperature dimension quality inspection indicator features are selected. Then, the server performs cross-modal gating interaction between the quality inspection indicator features of the target quality inspection dimension and the feature set. In this process, the server uses complex algorithms to allow the temperature dimension features to interact with each element in the feature set to extract more representative features. For example, the server analyzes the relationship between temperature features and state category features such as "engine temperature too high," as well as industry prior knowledge vectors, to obtain graph convolutional feature vectors for the multimodal quality inspection indicator states and graph convolutional feature vectors for each indicator state category to be matched. The server determines the current feature set based on these graph convolutional feature vectors.Next, the server iteratively executes the step of determining the quality inspection indicator features for the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of multiple quality inspection dimensions, until the process ends after all quality inspection dimensions (such as temperature, vibration, fuel injection quantity, etc.) have had their quality inspection indicator features processed. Finally, based on the current feature set, it obtains the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched. Another approach is to extract state features using a quality inspection state recognition network. The server sequentially inputs the obtained multimodal quality inspection indicator states and each indicator state category to be matched into the quality inspection state recognition network. This network, trained on a large amount of data, can automatically learn how to extract effective features from the input data. For example, during training, the network has learned the feature patterns of various indicator data under different operating states of a car engine. When a new multimodal quality inspection indicator state is input, it can accurately extract its multimodal feature vector, and for each indicator state category to be matched, it can also extract the corresponding category feature vector. The server uses the multimodal feature vectors for calculation. Taking a car engine as an example, there exists a composite abnormal quality inspection condition: "engine overheating and abnormal vibration." The server inputs multimodal feature vectors into a specific algorithm model, which analyzes the relationships between multimodal features such as temperature and vibration feature vectors. If the temperature feature vector indicates that the engine temperature is close to or exceeds the overheating threshold, and the vibration feature vector also indicates that the vibration amplitude exceeds the normal range, the model calculates the confidence value of this multimodal quality inspection indicator state belonging to the composite abnormal quality inspection condition of "engine overheating and abnormal vibration" based on this information and the patterns learned during training. For all preset composite abnormal quality inspection conditions, such as "abnormal engine fuel injection and unstable temperature," the server performs similar calculations to obtain the state confidence distribution of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition, i.e., the set of confidence values ​​corresponding to each composite abnormal quality inspection condition. The server obtains the relational topology structure corresponding to each composite abnormal quality inspection condition. Taking the composite anomaly quality inspection condition of "engine overheating and abnormal vibration" as an example, its relational topology includes category objects corresponding to each state category of the indicators to be matched, such as the category object of "engine overheating" and the category object of "engine abnormal vibration." The paths between category objects reflect the mutual influence relationship between two connected category objects. For example, overheating of the engine may cause thermal expansion and contraction of engine parts, which in turn affects the engine vibration; this influence relationship is represented by a path. In the composite anomaly quality inspection condition of "abnormal engine fuel injection and unstable temperature," there is also a path between the category objects of "abnormal fuel injection" and "unstable temperature," indicating that abnormal fuel injection may cause incomplete combustion of the engine, thereby causing temperature fluctuations, reflecting their mutual influence.For each category of indicators to be matched, the server determines the corresponding target category object in the relational topology of each composite anomaly quality inspection condition. For example, for the indicator category "engine temperature too high," it is one of the target category objects in the relational topology of "engine overheating and abnormal vibration." For each relational topology of a composite anomaly quality inspection condition, the server determines the associated feature vector of the target category object under that composite anomaly quality inspection condition based on the mutual influence relationship between the target category object and other category objects in the relational topology. For example, in the relational topology of "engine overheating and abnormal vibration," the target category object "engine temperature too high" and the category object "engine vibration abnormality" are connected by a path. The server will comprehensively consider the impact of excessive temperature on vibration, as well as the feedback of abnormal vibration on temperature, and combine multimodal feature vectors and information in the relational topology to determine the associated feature vector of the target category object "engine temperature too high" under this composite anomaly quality inspection condition. For each composite anomaly quality inspection condition, the server merges the confidence value of the multimodal quality inspection indicator state belonging to that composite anomaly quality inspection condition with the associated feature vector of the target category object under that composite anomaly quality inspection condition. For example, for the composite anomaly quality inspection condition of "engine overheating and abnormal vibration," if the calculated confidence value of this multimodal quality inspection indicator state belonging to this condition is 0.8, and the associated feature vector of the target category object "engine temperature too high" is [0.2, 0.3, 0.4], the server will use a specific algorithm to merge the confidence value 0.8 with the associated feature vector [0.2, 0.3, 0.4] to obtain the basic merged feature of the target indicator state category "engine temperature too high" under this composite anomaly quality inspection condition. Based on the basic merged feature of the target indicator state category under each composite anomaly quality inspection condition, the server determines the merged feature vector of that target indicator state category. For example, the "engine temperature too high" indicator state category has corresponding basic merged features under all composite abnormal quality inspection conditions such as "engine overheating and abnormal vibration" and "engine fuel injection abnormality and unstable temperature". The server comprehensively calculates these basic merged features to obtain the merged feature vector of the "engine temperature too high" indicator state category. Based on the merged feature vector, the server optimizes the category feature vector of the indicator state category to be matched. For example, for the "engine temperature too high" indicator state category, the server fuses and adjusts its original category feature vector with the merged feature vector, and obtains the deep category feature vector corresponding to the indicator state category through specific optimization algorithms, such as the backpropagation algorithm in neural networks. This deep category feature vector more accurately reflects the relationship between the state category and multimodal quality inspection indicator states as well as various composite abnormal quality inspection conditions.For each category of indicator state to be matched, the server merges the category feature vector and the deep category feature vector of that category to obtain the target category feature vector. For example, for the category of "abnormal engine vibration," the server merges its original category feature vector with the deep category feature vector obtained through feature embedding. A specific merging algorithm integrates the information from the two vectors to obtain a more representative target category feature vector. Based on the target category feature vector of each category, the server determines the matching coefficient for each category within the multimodal quality inspection indicator state. The server inputs the target category feature vector into a specific scoring model, which, based on the feature information in the vector and the standards learned during training, provides a numerical value as the matching coefficient. For example, if the target category feature vector of the "abnormal engine vibration" category highly matches the vibration features in the multimodal quality inspection indicator state, the matching coefficient given by the model will be high, indicating a high degree of matching between this category and the current multimodal quality inspection indicator state. Based on the matching coefficients of each pending indicator state category, the server determines the target indicator state category corresponding to the multimodal quality inspection indicator state as the quality monitoring result for the target product. For example, in the quality monitoring of automotive engines, the matching coefficient for "engine overheating" is 0.7, for "abnormal engine vibration" it is 0.8, and for "fuel injection deviation" it is 0.3. The server compares these matching coefficients and finds that "abnormal engine vibration" has the highest matching coefficient. Therefore, it determines "abnormal engine vibration" as the target indicator state category, meaning that the current quality monitoring result for the automotive engine indicates an abnormality in engine vibration. Through these detailed steps, the deep learning-driven intelligent monitoring method for production process quality can help enterprise servers accurately and efficiently monitor the quality of target products, promptly identify potential problems in the production process, and provide strong support for enterprises to improve product quality and reduce production costs.

[0026] In one possible implementation, the step of sequentially extracting state features from the multimodal quality inspection index states and each index state category to be matched, to obtain the multimodal feature vector of the multimodal quality inspection index states and the category feature vector of each index state category to be matched, can be implemented through the following example.

[0027] The multimodal quality inspection index status is subjected to multimodal status characterization processing on the quality inspection index data in at least one quality inspection dimension to obtain the quality inspection index features of the multimodal quality inspection index status in at least one quality inspection dimension.

[0028] Multimodal state representation processing is performed on each state category of the indicator to be matched in turn to obtain the state category features of each state category of the indicator to be matched.

[0029] A feature set is constructed based on the state category features of each indicator state category to be matched and the industry prior knowledge vector;

[0030] The quality inspection indicator features and the feature set are subjected to cross-modal gating interaction to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched.

[0031] In this embodiment of the invention, for example, the server is responsible for monitoring the quality of the mobile phone production process. For multimodal quality inspection indicators, mobile phone production involves multiple quality inspection dimensions, such as appearance and performance. In the appearance dimension, the server acquires quality inspection indicator data such as the color, scratches, and flatness of the mobile phone casing, and performs multimodal state characterization processing on this data. For example, a specific algorithm is used to convert color data into feature values ​​representing the degree of color deviation, and scratch data is converted into feature representations such as scratch length and depth, thus obtaining the quality inspection indicator features for the appearance dimension. In the performance dimension, the server processes data such as the mobile phone processor performance and battery life, converting processor operating frequency and heat generation into corresponding features, thereby obtaining the quality inspection indicator features for the multimodal quality inspection indicator state in at least one quality inspection dimension. For the indicator state category to be matched, such as "minor appearance defects" or "unstable performance," the server sequentially performs multimodal state characterization processing on them. Taking "minor appearance defects" as an example, by combining past data on similar appearance problems and industry standards, features such as minor scratches and slight color differences are quantified to obtain the state category features for this state category. Next, the server constructs a feature set based on the state category features of each to-be-matched indicator state category and industry prior knowledge vectors. The industry prior knowledge vectors include vectors derived from knowledge of common mobile phone industry standards for appearance and performance, such as the allowable range of color deviation for phone casings and the normal heat dissipation range of processors. These vectors are integrated with the state category features to form the feature set. Finally, the server performs cross-modal gating interaction between the quality inspection indicator features and the feature set. For example, it first selects the quality inspection indicator features of the appearance dimension and allows them to interact with elements in the feature set. By analyzing the relationship between appearance features and state category features such as "minor appearance defects" and industry prior knowledge vectors, a complex algorithm is used to obtain the graph convolutional feature vectors of the multimodal quality inspection indicator states for mobile phone appearance, as well as the graph convolutional feature vectors of each to-be-matched indicator state category, such as "minor appearance defects." After multiple iterations across different quality inspection dimensions, based on the graph convolutional feature vectors obtained each time, the multimodal feature vectors of the multimodal quality inspection indicator states and the category feature vectors of each to-be-matched indicator state category are finally obtained.

[0032] In one possible implementation, the step of performing cross-modal gating interaction between the quality inspection indicator features and the feature set to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched can be implemented through the following example.

[0033] The quality inspection indicator characteristics of the target quality inspection dimension corresponding to this cycle are determined from the quality inspection indicator characteristics of the at least one quality inspection dimension.

[0034] Cross-modal gating interaction is performed on the quality inspection index features of the target quality inspection dimension and the feature set to obtain the graph convolution feature vector of the multimodal quality inspection index state and the graph convolution feature vector of each index state category to be matched.

[0035] Based on the graph convolutional feature vectors of the multimodal quality inspection index states and the graph convolutional feature vectors of each index state category to be matched, the current feature set is determined.

[0036] The process of determining the quality inspection indicator features of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of at least one quality inspection dimension is repeated until the process ends after all quality inspection indicator features of all quality inspection dimensions are obtained. Based on the current feature set, the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched are obtained.

[0037] In this embodiment of the invention, an exemplary example is taken: a server monitoring the quality of a smartwatch production process. The quality inspection dimensions of a smartwatch include hardware performance, appearance, and battery life. The server begins cross-modal gating interaction. First, it determines the quality inspection indicator features of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of multiple quality inspection dimensions. For example, the first cycle selects the quality inspection indicator features of the hardware performance dimension, which may include quantified feature values ​​such as processor speed and sensor accuracy. Next, the server performs cross-modal gating interaction between the quality inspection indicator features of the target quality inspection dimension (hardware performance dimension) and the previously constructed feature set. The feature set includes state category features such as "hardware performance lag" and "sensor data deviation" waiting to be matched with indicator state categories, as well as prior knowledge vectors regarding hardware performance standards in the smartwatch industry. Using a specific algorithm, the relationship between hardware performance characteristics and elements within the set is analyzed, such as the correlation between processor speed and the "hardware performance lag" state category. This yields the graph convolutional feature vectors of the smartwatch's multimodal quality inspection indicators in the hardware performance dimension, as well as the graph convolutional feature vectors of each target indicator state category, such as "hardware performance lag" and "sensor data deviation." Then, based on the graph convolutional feature vectors of the multimodal quality inspection indicators and each target indicator state category, the current feature set is determined. This current feature set incorporates key feature information generated from the cross-modal gating interaction in the hardware performance dimension. Afterward, the server iteratively executes the steps to determine the target quality inspection dimension. The second cycle selects the appearance dimension, whose quality inspection indicators may include dial flatness, strap color, etc. Similarly, cross-modal gating interaction is performed on the appearance dimension's quality inspection indicator features and the feature set to obtain the graph convolutional feature vectors of the appearance dimension's multimodal quality inspection indicators and each target indicator state category, and then the current feature set is updated. This process is repeated until all quality inspection indicators, including battery life, have completed the aforementioned steps. Finally, based on the continuously updated feature set, the server obtains a multimodal feature vector that comprehensively reflects the multimodal quality inspection indicator status of the smartwatch, as well as a category feature vector corresponding to each indicator status category to be matched, providing crucial data support for subsequent judgments on the smartwatch's quality.

[0038] In this embodiment of the invention, the relational topology includes a category object corresponding to each category of the state of the index to be matched, and a path between the category objects, wherein the path reflects the mutual influence relationship between two connected category objects;

[0039] Based on the state confidence distribution and the relational topology structure corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the category feature vector of each target indicator state category to obtain the deep category feature vector corresponding to each target indicator state category. This can be implemented through the following example.

[0040] For each category of the indicator status to be matched, determine the target category object corresponding to the category of the indicator status in the relational topology structure corresponding to each composite abnormal quality inspection status;

[0041] For each composite abnormal quality inspection situation, based on the relational topology structure corresponding to the target category object and other category objects in the relational topology structure, the associated feature vector of the target category object under the composite abnormal quality inspection situation is determined.

[0042] Based on the state confidence distribution, the associated feature vectors of the target category objects under each composite abnormal quality inspection condition are merged to obtain the merged feature vector of the state category of the indicator to be matched.

[0043] Based on the merged feature vector, the category feature vector of the state category of the indicator to be matched is optimized to obtain the deep category feature vector corresponding to the state category of the indicator to be matched.

[0044] In this embodiment of the invention, an exemplary case is taken of a server monitoring the quality of a laptop manufacturing process. In this scenario, the categories of indicators to be matched include "abnormal screen display," "short battery life," and "poor heat dissipation," while a composite abnormal quality inspection condition is "abnormal screen display and poor heat dissipation." In the relational topology, each category of indicator to be matched corresponds to a category object. For example, "abnormal screen display" corresponds to one category object, and "poor heat dissipation" also corresponds to another. If a path exists between them, it indicates a mutual influence relationship. For instance, poor heat dissipation may affect screen display because high temperatures can cause instability in the performance of screen display components. First, for each category of indicator to be matched, the server determines its corresponding target category object in the relational topology for each composite abnormal quality inspection condition. Taking "abnormal screen display" as an example, in the relational topology for the composite abnormal quality inspection condition "abnormal screen display and poor heat dissipation," "abnormal screen display" is the corresponding target category object. Next, for each relational topology for each composite abnormal quality inspection condition, the server determines the associated feature vector of the target category object based on the mutual influence relationship between the target category object and other category objects. In the relational topology of "screen display abnormality and poor heat dissipation," "screen display abnormality" is the target category object. The server considers the impact of poor heat dissipation on screen display, such as high temperature causing screen flickering and color deviation. Combining relevant features from the multimodal quality inspection indicator states, the server determines the associated feature vector of "screen display abnormality" under this composite abnormal quality inspection condition. This vector may contain feature values ​​related to screen problems caused by high temperature. Then, based on the state confidence distribution, the server merges the associated feature vectors of the target category object under each composite abnormal quality inspection condition to obtain the merged feature vector of the indicator state category to be matched. Similarly, for the composite abnormal quality inspection condition of "screen display abnormality and short battery life," the server also determines the associated feature vector of "screen display abnormality" under this condition. Combined with the associated feature vector under the condition of "screen display abnormality and poor heat dissipation," and based on the state confidence distribution (e.g., the confidence value of "screen display abnormality and poor heat dissipation" is 0.7, and the confidence value of "screen display abnormality and short battery life" is 0.3), a specific algorithm is used to merge the two associated feature vectors proportionally to obtain the merged feature vector of the indicator state category to be matched, "screen display abnormality." Finally, based on the merged feature vector, the server optimizes the category feature vector of the "screen display abnormality" indicator status category to be matched, and obtains the deep category feature vector.The server combines the original "screen display anomaly" category feature vector with the merged feature vector and uses an optimization algorithm to adjust it, so that the resulting deep category feature vector more accurately reflects the relationship between "screen display anomaly" and different composite anomaly quality inspection conditions, providing more accurate feature basis for subsequent determination of the degree of matching with the multimodal quality inspection indicator status.

[0045] In this embodiment of the invention, the state confidence distribution includes the confidence value of the multimodal quality inspection index state belonging to each composite abnormal quality inspection condition;

[0046] The process of merging the associated feature vectors of target category objects under each composite abnormal quality inspection condition based on the state confidence distribution to obtain the merged feature vector of the state category of the indicator to be matched can be implemented through the following example.

[0047] For each composite abnormal quality inspection situation, the confidence value of the multimodal quality inspection index state belonging to the composite abnormal quality inspection situation is merged with the associated feature vector of the target category object under the composite abnormal quality inspection situation to obtain the basic merged feature of the index state category to be matched under the composite abnormal quality inspection situation.

[0048] Based on the basic merging features of the state category of the indicator to be matched under each composite abnormal quality inspection condition, the merging feature vector of the state category of the indicator to be matched is determined.

[0049] In this embodiment of the invention, taking the server monitoring the production quality of smartphones as an example, there are currently multiple composite abnormal quality inspection conditions, such as "blurry camera image and rapid battery drain" and "overheating and unstable signal". The categories of the indicators to be matched include "blurry camera image", "rapid battery drain", "overheating", and "unstable signal". First, the state confidence distribution is defined, which represents the confidence value of the multimodal quality inspection indicator state belonging to each composite abnormal quality inspection condition. The server calculates in advance that the confidence value of the multimodal quality inspection indicator state belonging to "blurry camera image and rapid battery drain" is 0.8, and the confidence value of belonging to "overheating and unstable signal" is 0.6. Next, operations are performed for each composite abnormal quality inspection condition. Taking the indicator state category to be matched, "blurry camera image", as an example, under the composite abnormal quality inspection condition of "blurry camera image and rapid battery drain", the server has determined that the associated feature vector of the target category object of "blurry camera image" is [0.2, 0.3, 0.1]. The server merges the confidence value 0.8 belonging to the composite anomaly quality inspection condition with the associated feature vector [0.2, 0.3, 0.1]. For example, using a specific weighted algorithm, the confidence value 0.8 is multiplied by each value in the associated feature vector to obtain the basic merged feature [0.16, 0.24, 0.08] for "blurred camera image" under the composite anomaly quality inspection condition. Similarly, under the composite anomaly quality inspection condition of "overheating and unstable signal", if the associated feature vector of the "blurred camera image" target category is [0.1, 0.2, 0.3], the server merges the confidence value 0.6 with this associated feature vector to obtain the basic merged feature [0.06, 0.12, 0.18] for "blurred camera image" under this condition. Finally, based on the basic merged feature of "blurred camera image" under each composite anomaly quality inspection condition, the server determines its merged feature vector. The server employs a comprehensive calculation method, such as adding the values ​​at corresponding positions of two basic merged features and then averaging them, i.e., [(0.16+0.06) / 2,(0.24+0.12) / 2,(0.08+0.18) / 2], to obtain the final merged feature vector [0.11,0.18,0.13]. This merged feature vector comprehensively considers the feature information of "camera image blur" under different composite anomaly quality inspection conditions, providing a basis for subsequent optimization of the "camera image blur" category feature vector, and thus obtaining the depth category feature vector.

[0050] In this embodiment of the invention, the step of determining the matching coefficient of each target indicator state category belonging to the multimodal quality inspection indicator state based on the deep category feature vector corresponding to each target indicator state category can be implemented through the following example.

[0051] For each category of indicator state to be matched, the category feature vector and deep category feature vector of the category of indicator state to be matched are merged to obtain the target category feature vector of the category of indicator state to be matched.

[0052] Based on the target category feature vector of each target indicator state category, determine the matching coefficient of each target indicator state category belonging to the multimodal quality inspection indicator state.

[0053] In this embodiment of the invention, taking the server monitoring the quality of a tablet computer's production process as an example, the categories of indicators to be matched for the tablet computer include "screen touch insensitivity," "poor battery life," and "abnormal audio output." The server has already obtained the deep category feature vector for each category of indicator to be matched through the previous steps. For the category of "screen touch insensitivity," the server first merges its original category feature vector and deep category feature vector. The original category feature vector is obtained by performing multimodal state representation processing on this category, and may include relevant feature values ​​such as touch response time and false touch rate. The deep category feature vector is obtained by comprehensively considering various composite abnormal quality inspection conditions, such as further refinement of the "screen touch insensitivity" feature under composite abnormal quality inspection conditions such as "screen touch insensitivity and poor battery life" and "screen touch insensitivity and abnormal audio output," and may include feature values ​​after interaction with other abnormal conditions. The server uses a specific merging algorithm to combine the two vectors, such as element-wise addition or weighted fusion, to obtain the target category feature vector for the "screen touch insensitivity" indicator state category. The same operation applies to other indicator state categories such as "poor battery life" and "abnormal audio output." The server merges their respective category feature vectors with the deep category feature vectors to obtain the corresponding target category feature vectors. Next, based on the target category feature vector for each indicator state category, the server determines the matching coefficient for each category to belong to the multimodal quality inspection indicator state. The server inputs the target category feature vector for "screen touch insensitivity" into a pre-trained matching coefficient calculation model. This model is trained on a large amount of tablet computer production data and understands the relationship between various indicators of tablet computers under normal and abnormal conditions. The model outputs a value as the matching coefficient for "screen touch insensitivity" belonging to the current multimodal quality inspection indicator state, based on the feature values ​​in the target category feature vector, such as the degree of change in touch response time and the degree of correlation with other abnormal conditions, according to specific calculation rules. For example, if the target category feature vector shows that the touch response time is outside the normal range and is strongly correlated with abnormal conditions such as poor battery life, the matching coefficient output by the model may be high, indicating that the state of "unresponsive screen touch" matches the current multimodal quality inspection indicator state to a high degree. Following the same process, the server processes the target category feature vectors of other indicator states to be matched, such as "poor battery life" and "abnormal audio output," and determines their matching coefficients for each multimodal quality inspection indicator state. These matching coefficients will provide crucial evidence for subsequently determining the actual quality problems of the tablet computer.

[0054] In this embodiment of the invention, the step of sequentially extracting state features from the multimodal quality inspection index state and each index state category to be matched, to obtain the multimodal feature vector of the multimodal quality inspection index state and the category feature vector of each index state category to be matched, can be implemented through the following example.

[0055] The quality inspection status recognition network sequentially extracts status features from the multimodal quality inspection index status and each category of the index to be matched, thereby obtaining the multimodal feature vector of the multimodal quality inspection index status and the category feature vector of each category of the index to be matched.

[0056] In this embodiment of the invention, for example, the server is responsible for quality monitoring of the smart earphone production process. The production of smart earphones involves various quality inspection indicators. These multimodal quality inspection indicators cover data on acoustic performance, connection stability, battery life, and other aspects, while the categories of indicators to be matched include "sound quality noise," "unstable Bluetooth connection," and "too short battery life." The server uses a quality inspection status recognition network to extract features. First, the multimodal quality inspection indicator status data of the smart earphones is input into the quality inspection status recognition network. For example, this includes frequency response curve data and harmonic distortion data for acoustic performance; Bluetooth connection success rate and signal strength fluctuation data for connection stability; and battery capacity and charging speed data for battery life. The quality inspection status recognition network is a deep neural network model trained on a large amount of smart earphone production data. It can automatically learn patterns and features in the data. The neurons within the network process the input data layer by layer. For example, convolutional layers extract local feature patterns from the acoustic data, and recurrent neural network layers analyze the time-series features of the connection stability data. After a series of complex calculations and transformations, the final output is a multimodal feature vector representing the state of multimodal quality inspection indicators. This vector encapsulates the key feature information of the smart headphones across various quality inspection dimensions. Next, for the "sound quality noise" category, the server inputs relevant descriptive data into the quality inspection state recognition network. This data may include acoustic parameters of the headphones, production batch information, and usage environment feedback from past instances of sound quality noise issues. Based on its training knowledge, the network analyzes and processes this data, mining features related to "sound quality noise" from different angles, and ultimately outputs a category feature vector for the "sound quality noise" category, accurately characterizing the features of this state. Following the same method, the server sequentially inputs relevant data for each category of "unstable Bluetooth connection" and "short battery life" into the quality inspection state recognition network to obtain the corresponding category feature vectors. In this way, the server, with the help of the quality inspection status recognition network, efficiently and accurately extracts the status features of multimodal quality inspection indicators and each category of indicator status to be matched, providing a key data foundation for the subsequent analysis and judgment of the production quality of smart headphones.

[0057] In this embodiment of the invention, before the step of extracting state features from the multimodal quality inspection index state and each matching index state category sequentially through the quality inspection state recognition network to obtain the multimodal feature vector of the multimodal quality inspection index state and the category feature vector of each matching index state category, the following implementation method is also provided.

[0058] Obtain training samples and the relational topology structure corresponding to each composite abnormal quality inspection status. The training samples include multiple preset quality inspection index states and preset index state categories of the preset quality inspection index states.

[0059] Through the quality inspection status recognition network, the status features of the preset quality inspection index status and each index status category to be matched are extracted sequentially to obtain the preset multimodal feature vector of the preset quality inspection index status and the preset category feature vector of each index status category to be matched.

[0060] Based on the preset multimodal feature vector, calculate the state confidence distribution of the preset quality inspection index state belonging to each composite abnormal quality inspection state;

[0061] Based on the state confidence distribution and the relational topology corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the preset category feature vector of each target indicator state category to obtain the preset deep category feature vector corresponding to each target indicator state category.

[0062] Based on the preset deep category feature vector corresponding to each state category of the indicator to be matched, determine the matching coefficient of each state category of the indicator to be matched belonging to the preset quality inspection indicator state.

[0063] Based on the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network are updated to obtain the trained quality inspection state recognition network.

[0064] In this embodiment of the invention, taking the server's quality monitoring of the smart bracelet production process as an example, firstly, the server acquires training samples and the relational topology corresponding to each composite abnormal quality inspection condition. The preset quality inspection indicator states in the training samples cover various types of data from the smart bracelet, such as accuracy data of heart rate monitoring, deviation data of step counting, and stability data of screen brightness adjustment. Simultaneously, these preset quality inspection indicator states all have corresponding preset indicator state categories, such as "inaccurate heart rate monitoring," "abnormal step counting," and "screen brightness adjustment failure." For composite abnormal quality inspection conditions, such as "inaccurate heart rate monitoring and screen brightness adjustment failure," the relational topology clearly defines the mutual influence relationship between the two categories of "inaccurate heart rate monitoring" and "screen brightness adjustment failure," for example, a motherboard power supply problem might simultaneously affect both functions. Next, the server uses a quality inspection state recognition network to sequentially extract state features from the preset quality inspection indicator states and each category of indicator state to be matched. For pre-defined quality inspection indicator states, such as the accuracy data of heart rate monitoring, after inputting into the network, the neural layers within the network perform calculations. Convolutional layers capture local feature patterns in the data, and recurrent layers analyze the time-series features of the data, ultimately obtaining a pre-defined multimodal feature vector for the pre-defined quality inspection indicator state. For each target indicator state category, such as "inaccurate heart rate monitoring," the network processes the relevant data in a similar manner, outputting a corresponding pre-defined category feature vector. Then, based on the pre-defined multimodal feature vectors, the server calculates the state confidence distribution for each composite abnormal quality inspection condition. For example, for a composite abnormality like "inaccurate heart rate monitoring and screen brightness adjustment malfunction," the server, based on a comprehensive analysis of the features related to heart rate and screen brightness in the pre-defined multimodal feature vector, derives a confidence value for the pre-defined quality inspection indicator state belonging to this composite abnormality. This calculation is performed for all composite abnormal quality inspection conditions, forming a state confidence distribution. Finally, based on the state confidence distribution and relational topology, the server performs feature embedding processing on the pre-defined category feature vector for each target indicator state category. Taking "inaccurate heart rate monitoring" as an example, in the relational topology of "inaccurate heart rate monitoring and screen brightness adjustment failure," it is identified as the target category object. Combining the state confidence distribution, such as the confidence value of this composite anomaly, and its interaction with "screen brightness adjustment failure," an associated feature vector is determined. Then, feature merging is performed to obtain a preset deep category feature vector. Next, based on the preset deep category feature vector corresponding to each state category of the indicator to be matched, the server determines the matching coefficient of each state category belonging to the preset quality inspection indicator state. The preset deep category feature vector of "inaccurate heart rate monitoring" is combined with the relevant features of the preset quality inspection indicator state, and a specific algorithm is used to calculate the matching coefficient, reflecting the degree of matching between "inaccurate heart rate monitoring" and the current preset quality inspection indicator state.Finally, based on the matching coefficients and preset indicator state categories, the server updates the network parameters in the quality inspection state recognition network. If the matching coefficients do not match the preset indicator state categories (e.g., the actual state is "inaccurate heart rate monitoring"), it indicates a bias in the network prediction. The server then adjusts the network's weights, biases, and other parameters using techniques such as backpropagation. This process is repeated multiple times to continuously optimize the network until the network's output matching coefficients closely match the preset indicator state categories. This results in the trained quality inspection state recognition network, enabling more accurate quality monitoring of smart bracelet production.

[0065] In this embodiment of the invention, the step of updating the network parameters in the quality inspection status recognition network according to the matching coefficient and the preset indicator status category of the preset quality inspection indicator status to obtain the trained quality inspection status recognition network can be implemented through the following example.

[0066] The cost parameters of the quality inspection status identification network are determined based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status.

[0067] When the cost parameter does not reach a convergence state, the network parameters in the quality inspection status identification network are updated based on the cost parameter.

[0068] Based on the confidence distribution of the state of each composite abnormal quality inspection condition, the state of each pre-set quality inspection indicator is divided to obtain the pre-set quality inspection indicator state group corresponding to each composite abnormal quality inspection condition.

[0069] For each composite abnormal quality inspection situation, based on the preset indicator state category of the preset quality inspection indicator state in the preset quality inspection indicator state group corresponding to the composite abnormal quality inspection situation, the mutual influence relationship of each indicator state category to be matched in the relation topology structure corresponding to the composite abnormal quality inspection situation is optimized to obtain the optimized relation topology structure corresponding to the composite abnormal quality inspection situation.

[0070] Based on the updated quality inspection status recognition network and the optimized relational topology corresponding to each composite abnormal quality inspection status, the step of extracting state features from the preset quality inspection index status and each index status category to be matched is executed cyclically through the quality inspection status recognition network until the cost parameter of the quality inspection status recognition network reaches a convergence state, thus obtaining the trained quality inspection status recognition network.

[0071] In this embodiment of the invention, the exemplary case of server-side quality monitoring of the smart bracelet production process is still taken. The server first determines the cost parameter of the quality inspection status identification network based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status. For example, the actual preset indicator status category of a certain preset quality inspection indicator status is "abnormal step count recording," but the matching coefficient for "abnormal step count recording" calculated by the current network is low, indicating a deviation between the network prediction and the actual situation. The server uses a specific loss function (such as the cross-entropy loss function) to calculate the cost parameter based on this deviation. This parameter measures the degree of difference between the network's current prediction result and the actual situation. When the cost parameter does not reach convergence, it means that the deviation between the network prediction result and the actual situation is still large. The server updates the network parameters in the quality inspection status identification network based on the cost parameter. For example, using the gradient descent algorithm, the gradient of each layer parameter (such as weights and biases) of the network is calculated based on the cost parameter, and these parameters are adjusted in the direction that reduces the cost parameter, thereby improving the network's prediction ability. Next, the server categorizes each pre-set quality inspection indicator state based on the state confidence distribution of each composite abnormal quality inspection condition. There are two composite abnormal quality inspection conditions: "abnormal step count recording and inaccurate heart rate monitoring" and "abnormal step count recording and screen brightness adjustment malfunction." For a given pre-set quality inspection indicator state, if its confidence in belonging to "abnormal step count recording and inaccurate heart rate monitoring" is high, it is assigned to the pre-set quality inspection indicator state group corresponding to that composite abnormal quality inspection condition. For each composite abnormal quality inspection condition, the server optimizes the mutual influence relationships of each unmatched indicator state category in the relational topology based on the pre-set indicator state category in its corresponding pre-set quality inspection indicator state group. For example, in the pre-defined quality inspection indicator state group of "abnormal step count and inaccurate heart rate monitoring," if many pre-defined quality inspection indicator states simultaneously exhibit abnormalities in both step count and heart rate monitoring, and there is a specific correlation pattern, such as significant fluctuations in step count often accompanied by deviations in heart rate monitoring, the server will adjust the mutual influence relationship between these two unmatched indicator state categories in the relational topology, resulting in an optimized relational topology. Then, based on the updated quality inspection state recognition network and the optimized relational topology, the server iteratively executes the step of extracting state features from the pre-defined quality inspection indicator states and each unmatched indicator state category through the quality inspection state recognition network. This involves re-extracting the pre-defined multimodal feature vectors of the pre-defined quality inspection indicator states and the pre-defined category feature vectors of the unmatched indicator state categories, calculating the state confidence distribution, performing feature embedding processing, and determining the matching coefficients.These steps are repeated continuously, each time calculating the cost parameter based on the new matching coefficient and preset indicator state category, updating the network parameters, and optimizing the relational topology, until the cost parameter of the quality inspection state recognition network reaches a convergence state. At this point, the trained quality inspection state recognition network is obtained, which can more accurately monitor and judge the quality status in the smart bracelet production process.

[0072] In this embodiment of the invention, the step of dividing each preset quality inspection indicator state according to the state confidence distribution of the preset quality inspection indicator state belonging to each composite abnormal quality inspection condition to obtain the preset quality inspection indicator state group corresponding to each composite abnormal quality inspection condition can be implemented through the following example.

[0073] Based on the confidence distribution of the state of the preset quality inspection index state belonging to each composite abnormal quality inspection state, the preset composite abnormal quality inspection state corresponding to the preset quality inspection index state is determined from each composite abnormal quality inspection state.

[0074] Based on the preset composite abnormal quality inspection status corresponding to each preset quality inspection indicator status, the preset quality inspection indicator status is classified and processed to obtain the preset quality inspection indicator status group corresponding to each composite abnormal quality inspection status.

[0075] In this embodiment of the invention, for example, during the smartwatch manufacturing process, there are multiple composite abnormal quality inspection conditions, such as "abnormal screen display and short battery life" and "inaccurate heart rate monitoring and unstable Bluetooth connection." The server has obtained the state confidence distribution of each preset quality inspection indicator state belonging to each composite abnormal quality inspection condition. The first step is to determine the preset composite abnormal quality inspection condition corresponding to the preset quality inspection indicator state from each composite abnormal quality inspection condition based on the state confidence distribution. One of the preset quality inspection indicator states includes screen brightness fluctuation data, battery power consumption rate data, heart rate monitoring deviation data, and Bluetooth connection interruption count data, etc. For the composite abnormal quality inspection condition of "abnormal screen display and short battery life," the server calculates the state confidence distribution based on the screen and battery-related data in the preset quality inspection indicator state, obtaining a confidence value of 0.7 for the preset quality inspection indicator state belonging to this composite abnormal quality inspection condition; for "inaccurate heart rate monitoring and unstable Bluetooth connection," the confidence value is 0.3. Since 0.7 is greater than 0.3, the server determines "abnormal screen display and short battery life" as the pre-set composite abnormal quality inspection condition corresponding to this pre-set quality inspection indicator state. The second step involves classifying each pre-set quality inspection indicator state based on its corresponding pre-set composite abnormal quality inspection condition, resulting in a pre-set quality inspection indicator state group for each composite abnormal quality inspection condition. The server processed 100 pre-set quality inspection indicator states, determining the corresponding pre-set composite abnormal quality inspection condition for each state using the method described in the first step. For example, if there are 30 pre-set quality inspection indicator states corresponding to the pre-set composite abnormal quality inspection condition of "abnormal screen display and short battery life," the server will group these 30 pre-set quality inspection indicator states into one group, forming the pre-set quality inspection indicator state group corresponding to the composite abnormal quality inspection condition of "abnormal screen display and short battery life." Similarly, if there are 20 pre-set quality inspection indicator states corresponding to the pre-set composite abnormal quality inspection condition of "inaccurate heart rate monitoring and unstable Bluetooth connection," these 20 pre-set quality inspection indicator states will be grouped into another group, forming the pre-set quality inspection indicator state group corresponding to "inaccurate heart rate monitoring and unstable Bluetooth connection." Through this classification process, the server constructs a corresponding pre-set quality inspection indicator state group for each composite abnormal quality inspection condition. These groups provide a data foundation for subsequent optimization of the relationship topology structure for different composite abnormal quality inspection conditions, helping to improve the accuracy of the quality inspection status identification network in monitoring the production quality of smartwatches.

[0076] In this embodiment of the invention, the mutual influence relationship of each matching indicator state category in the relational topology structure corresponding to the composite abnormal quality inspection status is optimized based on the preset indicator state category in the preset quality inspection indicator state group corresponding to the composite abnormal quality inspection status, so as to obtain the optimized relational topology structure corresponding to the composite abnormal quality inspection status. This can be implemented through the following example.

[0077] Based on the preset indicator status category of the preset quality inspection indicator status in the preset quality inspection indicator status group corresponding to the composite abnormal quality inspection status, a preset quality inspection indicator status group for each indicator status category to be matched is determined in the preset quality inspection indicator status group corresponding to the composite abnormal quality inspection status. Each preset quality inspection indicator status group for an indicator status category to be matched includes the preset quality inspection indicator status corresponding to the corresponding indicator status category to be matched.

[0078] For each category of indicator status to be matched, the preset quality inspection indicator status in the preset quality inspection indicator status group of the category of indicator status to be matched is analyzed to determine the target confidence value of the preset quality inspection indicator status that includes the category of indicator status to be matched when the preset quality inspection indicator status includes the category of indicator status to be matched under the composite abnormal quality inspection situation. The target confidence value reflects the mutual influence relationship between the category of indicator status to be matched and the other categories of indicator status to be matched.

[0079] Based on the target confidence value, the mutual influence relationship of each index state category to be matched in the relational topology structure corresponding to the composite abnormal quality inspection status is optimized to obtain the optimized relational topology structure corresponding to the composite abnormal quality inspection status.

[0080] In this embodiment of the invention, taking the server's quality monitoring of the smart speaker production process as an example, there exists a composite abnormal quality inspection situation of "abnormal sound quality and unstable connection". First, based on the preset indicator state categories in the preset quality inspection indicator state group corresponding to this composite abnormal quality inspection situation, the server determines the preset quality inspection indicator state group for each target indicator state category. The preset quality inspection indicator state group contains actual test data from multiple smart speakers, and its preset indicator state categories include "sound quality noise", "volume distortion", "Bluetooth connection interruption", "slow Wi-Fi connection", etc. The server groups all preset quality inspection indicator states belonging to "sound quality noise" into one group, which is the preset quality inspection indicator state group for the target indicator state category of "sound quality noise"; similarly, the preset quality inspection indicator states corresponding to each target indicator state category such as "volume distortion", "Bluetooth connection interruption", and "slow Wi-Fi connection" are respectively grouped into corresponding groups. Next, for each target indicator state category, the server analyzes its preset quality inspection indicator state group to determine the target confidence value. Taking "sound quality noise" as an example, the server checks the status of preset quality inspection indicators in this group and finds that when a smart speaker exhibits "sound quality noise," "Bluetooth connection interruption" also occurs in 70% of these cases. Therefore, under the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection," the target confidence value between "sound quality noise" and "Bluetooth connection interruption" is 0.7, reflecting their mutual influence. Similarly, for "sound quality noise" and "slow Wi-Fi connection," if there is a 30% probability of them occurring simultaneously, the target confidence value is 0.3. Finally, based on these target confidence values, the server optimizes the relational topology corresponding to the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection." In the original relational topology, the path weight between "sound quality noise" and "Bluetooth connection interruption" is 0.5, and the path weight between "sound quality noise" and "slow Wi-Fi connection" is 0.2. Now, based on the target confidence value, the path weights for "sound quality noise" and "Bluetooth connection interruption" are adjusted to 0.7, and the path weights for "sound quality noise" and "slow Wi-Fi connection" are adjusted to 0.3. By adjusting the path weights (i.e., the mutual influence relationships) between the various categories of indicators to be matched, an optimized relationship topology is obtained. This optimized relationship topology can more accurately reflect the true mutual influence relationships between the various categories of indicators to be matched in the smart speaker under the combined abnormal quality inspection condition of "abnormal sound quality and unstable connection." This helps the server to more accurately determine quality problems in the smart speaker production process using the quality inspection status identification network.

[0081] In this embodiment of the invention, the step of analyzing the preset quality inspection indicator states in the preset quality inspection indicator state group of each target indicator state category to determine the target confidence value of the preset quality inspection indicator state when the target indicator state includes the target indicator state category under the composite abnormal quality inspection condition can be implemented through the following example.

[0082] For each category of indicator status to be matched, the basic number of preset quality inspection indicator statuses in the preset quality inspection indicator status group of the category of indicator status to be matched is counted.

[0083] In the preset quality inspection indicator state group of the indicator state category to be matched, determine the number of preset quality inspection indicator states that simultaneously include the other indicator state categories to be matched.

[0084] Based on the basic quantity and the advanced quantity, when the preset quality inspection indicator status includes the category of the indicator status to be matched under the composite abnormal quality inspection condition, the target confidence value of the preset quality inspection indicator status also includes the other categories of indicator status to be matched.

[0085] In this embodiment of the invention, exemplarily, the quality monitoring of the smart speaker production process by the server is still taken as an example, focusing on the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection". The preset quality inspection indicator state group corresponding to the composite abnormal quality inspection condition of "abnormal sound quality and unstable connection" contains a large amount of smart speaker detection data, among which the categories of indicator states to be matched include "sound quality noise", "volume distortion", "Bluetooth connection interruption", "slow Wi-Fi connection", etc. First, for each category of indicator state to be matched, the server counts the basic number of preset quality inspection indicator states in its preset quality inspection indicator state group. Taking the category of indicator state to be matched, "sound quality noise", as an example, the server finds all data records judged to have "sound quality noise" in the preset quality inspection indicator state group. After counting, there are 100 such records, which is the basic number of the preset quality inspection indicator state group for "sound quality noise". Next, in the preset quality inspection indicator state group for "sound quality noise", the server determines the advanced number of preset quality inspection indicator states that simultaneously contain other categories of indicator states to be matched. For example, among the 100 records containing "audio noise," the server further filtered out records that also showed "Bluetooth connection interruption," finding 40 such records. These 40 records represent the advanced number of records simultaneously containing "Bluetooth connection interruption," another metric to be matched. Similarly, for "slow Wi-Fi connection," the server also searched among the 100 records with "audio noise" for records also showing "slow Wi-Fi connection," finding 20 such records. These 20 records represent the advanced number of records simultaneously containing "slow Wi-Fi connection." Finally, based on the base number and the advanced number of records, the server determined the target confidence value. For "sound quality noise" and "Bluetooth connection interruption," the target confidence value is calculated by dividing the advanced quantity by the basic quantity, i.e., (40 ÷ 100 = 0.4). This 0.4 is the target confidence value for "sound quality noise" and "Bluetooth connection interruption" when the preset quality inspection indicator states include both "sound quality noise" and "Bluetooth connection interruption" under the combined abnormal quality inspection condition of "abnormal sound quality and unstable connection." This reflects the degree of mutual influence between "sound quality noise" and "Bluetooth connection interruption." For "sound quality noise" and "slow Wi-Fi connection," the target confidence value is (20 ÷ 100 = 0.2), indicating the degree of mutual influence between these two indicator state categories to be matched. In this way, the server determines the corresponding target confidence value between each indicator state category to be matched and the other indicator state categories to be matched, providing accurate data support for subsequent optimization of the relationship topology structure. This allows the relationship topology structure to more accurately reflect the true correlation between each indicator state category to be matched under the combined abnormal quality inspection condition of "abnormal sound quality and unstable connection."

[0086] In this embodiment of the invention, the acquisition of training samples and the relational topology structure corresponding to each composite abnormal quality inspection status can be implemented through the following example.

[0087] Acquire training samples, and determine a set of preset quality inspection indicators for each category of indicators to be matched in the training samples based on the preset indicator status categories of the preset quality inspection indicators in the training samples. Each set of preset quality inspection indicators for each category of indicators to be matched includes the preset quality inspection indicators for the corresponding category of indicators to be matched in the training samples.

[0088] For each category of indicator status to be matched, the preset quality inspection indicator status in the preset quality inspection indicator status group of the category of indicator status to be matched is analyzed to determine the confidence value that the preset quality inspection indicator status also contains other categories of indicator status when the preset quality inspection indicator status contains the category of indicator status to be matched. The confidence value reflects the mutual influence relationship between the category of indicator status to be matched and the other categories of indicator status to be matched.

[0089] Based on the confidence value, the mutual influence relationship between the category objects corresponding to the state category of the indicator to be matched in the relational topology structure of each composite abnormal quality inspection status is set to obtain the relational topology structure corresponding to each composite abnormal quality inspection status after setting.

[0090] In this embodiment of the invention, the steps are illustrated in detail by taking the server's quality monitoring of the intelligent robotic vacuum cleaner's production process as an example. First, the server acquires training samples. These training samples contain a large amount of production data for intelligent robotic vacuum cleaners, covering preset quality inspection indicator states in various aspects such as cleaning effect, navigation accuracy, and battery life, as well as corresponding preset indicator state categories, such as "not cleaning thoroughly," "large navigation deviation," and "short battery life." Next, based on the preset indicator state categories, the server determines the preset quality inspection indicator state group for each target indicator state category from the training samples. For example, for the target indicator state category of "not cleaning thoroughly," the server filters all intelligent robotic vacuum cleaner production data related to "not cleaning thoroughly" from the training samples to form the preset quality inspection indicator state group for "not cleaning thoroughly." Similarly, for other target indicator state categories such as "large navigation deviation" and "short battery life," corresponding preset quality inspection indicator state groups are also determined. Then, for each category of indicator status to be matched, the server analyzes the pre-set quality inspection indicator statuses in its pre-set quality inspection indicator status group to determine the confidence value. Taking "not cleaning properly" as an example, the server counts the number of samples that also show "large navigation deviation" in the pre-set quality inspection indicator status group for "not cleaning properly". There are 100 samples in the pre-set quality inspection indicator status group for "not cleaning properly", and 30 of them also show "large navigation deviation". Therefore, the confidence value between "not cleaning properly" and "large navigation deviation" is 30 ÷ 100 = 0.3. This indicates that when the intelligent robot vacuum cleaner experiences "not cleaning properly", there is a 30% probability that "large navigation deviation" will also occur, reflecting the mutual influence relationship between these two categories of indicator status to be matched. Similarly, the server calculates the confidence value between "not cleaning properly" and "short battery life", as well as the confidence values ​​between other categories of indicator status to be matched. Finally, based on these confidence values, the server sets the mutual influence relationships between the category objects corresponding to the categories of indicator status to be matched in the relational topology structure of each composite abnormal quality inspection situation. There exists a composite anomaly quality inspection condition: "Inadequate cleaning and large navigation deviation." In its relational topology, "Inadequate cleaning" and "Large navigation deviation" are two category objects. The server, based on a previously calculated confidence value of 0.3, sets the mutual influence relationship between these two category objects. For example, in the relational topology graph, the edge connecting the nodes "Inadequate cleaning" and "Large navigation deviation" is assigned a weight of 0.3, representing the degree of mutual influence between them. For other composite anomaly quality inspection conditions, such as "Inadequate cleaning and short battery life" or "Large navigation deviation and short battery life," the server uses a similar method, setting the mutual influence relationship between category objects in the relational topology based on the corresponding confidence value, thus obtaining the relational topology structure corresponding to each composite anomaly quality inspection condition.These relational topologies will provide an important basis for the subsequent training of the quality inspection status recognition network, helping the server to more accurately monitor the production quality of the intelligent sweeping robot.

[0091] In this embodiment of the invention, the step of analyzing the preset quality inspection indicator states in the preset quality inspection indicator state group of each target indicator state category to determine the confidence value of the preset quality inspection indicator state containing the target indicator state category when the preset quality inspection indicator state contains the target indicator state category can be implemented through the following example.

[0092] For each category of indicator status to be matched, the basic number of preset quality inspection indicator statuses in the preset quality inspection indicator status group of the category of indicator status to be matched is counted.

[0093] In the preset quality inspection indicator state group of the indicator state category to be matched, determine the number of preset quality inspection indicator states that simultaneously include the other indicator state categories to be matched.

[0094] Based on the basic quantity and the advanced quantity, it is determined that when the preset quality inspection indicator status includes the category of the indicator status to be matched, the preset quality inspection indicator status also includes the confidence value of the other categories of indicator status to be matched.

[0095] In this embodiment of the invention, the process of server-side quality monitoring of drone production is illustrated in detail below. The server acquires a large number of training samples related to drones. These samples cover various preset quality inspection indicator states, such as flight stability, image quality, and battery life, as well as corresponding preset indicator state categories, such as "flight jitter," "blurry image," and "insufficient battery life." For each indicator state category to be matched, the server first counts the basic number of preset quality inspection indicator states in its preset quality inspection indicator state group. Taking "flight jitter" as an example, the server filters all drone data records showing "flight jitter" issues from the training samples. After careful counting, there are 200 such records, which constitute the basic number of the "flight jitter" preset quality inspection indicator state group. Next, within the "flight jitter" preset quality inspection indicator state group, the server determines the advanced number of preset quality inspection indicator states that simultaneously include other indicator state categories to be matched. For example, the server searches for records in the 200 "flight jitter" records that also exhibit "blurred shooting." After checking each record, it finds 50 records that also have "blurred shooting." These 50 records represent the advanced number of records that simultaneously contain "blurred shooting," another unmatched indicator category. Similarly, for "insufficient battery life," the server counts 30 records in the 200 "flight jitter" records that also exhibit "insufficient battery life." These 30 records represent the advanced number of records that simultaneously contain "insufficient battery life." Finally, based on the base number and the advanced number, the server determines the confidence value. For "flight jitter" and "blurred shooting," the confidence value is calculated by dividing the advanced number by the base number, i.e., 50 ÷ 200 = 0.25. This means that when a drone experiences "flight jitter," there is a 25% probability that "blurred shooting" will also occur. This 0.25 is the confidence value between them, reflecting the degree of mutual influence between these two unmatched indicator categories. For "flight jitter" and "insufficient battery life," the confidence value is 30 ÷ 200 = 0.15, indicating the degree of mutual influence between these two categories of indicators to be matched. In this way, the server can determine the corresponding confidence value between each category of indicator to be matched and the other categories, providing accurate data support for setting the mutual influence relationships between category objects in the subsequent composite anomaly quality inspection relationship topology. This helps the server more effectively monitor and analyze the production quality of drones.

[0096] In one possible implementation, the step of determining the cost parameter of the quality inspection status identification network based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status can be performed through the following example.

[0097] Calculate the category cost parameter based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status;

[0098] Based on the state confidence distribution of the preset quality inspection index state to each composite abnormal quality inspection condition, the distribution uncertainty of the preset quality inspection index state is calculated.

[0099] Based on the confidence distribution of the state of the preset quality inspection index state belonging to each composite abnormal quality inspection state, the preset composite abnormal quality inspection state corresponding to the preset quality inspection index state is determined, and based on the preset composite abnormal quality inspection state corresponding to each preset quality inspection index state, the number of training samples corresponding to each composite abnormal quality inspection state is determined.

[0100] The distribution cost parameter is calculated based on the distribution uncertainty of each preset quality inspection indicator state and the number of training samples corresponding to each composite abnormal quality inspection state.

[0101] Based on the category cost parameter and the distribution cost parameter, the cost parameters of the quality inspection status identification network are determined.

[0102] In this embodiment of the invention, taking the server's quality monitoring of the smart camera production process as an example, the process of determining the network cost parameter for quality inspection status recognition is described in detail. The server is processing the smart camera's production data. Preset quality inspection status indicators include data on image clarity, night vision effect, network connection stability, etc., and preset indicator status categories include "image blur," "night vision function abnormality," and "network connection interruption." The server has obtained the matching coefficients between each preset quality inspection status and each indicator status category to be matched through the quality inspection status recognition network. For each preset quality inspection status, the server calculates the category cost parameter based on its matching coefficient and the preset indicator status category. For example, the actual preset indicator status category of a certain preset quality inspection status is "image blur," while the matching coefficient for "image blur" given by the quality inspection status recognition network is 0.6. The server uses a common loss function, such as the cross-entropy loss function, to substitute the actual category ("image blur") and the predicted matching coefficient (0.6) into the calculation. The formula for the cross-entropy loss function is: Among them, y i Indicates the actual category (1 for "image blurry", 0 for other categories), p i This represents the predicted matching coefficient. Here, y "图像模糊” =1, p "图像模糊”=0.6, then the cost parameter of the preset quality inspection index state with respect to the "image blurry" category is calculated. This calculation is performed for all preset quality inspection index states, and then the total category cost parameter is obtained. The server calculates the distribution uncertainty based on the state confidence distribution of the preset quality inspection index state belonging to each composite abnormal quality inspection condition. For example, there are composite abnormal quality inspection conditions such as "image blurry and night vision function abnormal" and "image blurry and network connection interruption". For a certain preset quality inspection index state, its confidence in belonging to "image blurry and night vision function abnormal" is 0.7, and its confidence in belonging to "image blurry and network connection interruption" is 0.3. The server uses methods such as information entropy to measure the uncertainty of this distribution. This calculation is performed for all preset quality inspection index states, and then the results are summarized. Based on the state confidence distribution of the preset quality inspection index state belonging to each composite abnormal quality inspection condition, the server determines the preset composite abnormal quality inspection condition corresponding to each preset quality inspection index state. For example, if the confidence level of a pre-defined quality inspection indicator state is "image blurry and night vision function abnormal," then "image blurry and night vision function abnormal" is its corresponding pre-defined composite abnormal quality inspection state. Then, based on the pre-defined composite abnormal quality inspection state corresponding to each pre-defined quality inspection indicator state, the server counts the number of training samples corresponding to each composite abnormal quality inspection state. Among all training samples, there are 100 corresponding to "image blurry and night vision function abnormal" and 80 corresponding to "image blurry and network connection interruption." The server calculates the distribution cost parameter based on the distribution uncertainty of each pre-defined quality inspection indicator state and the number of training samples corresponding to each composite abnormal quality inspection state. For example, the total distribution uncertainty corresponding to "image blurry and night vision function abnormal" is 1.2 (obtained by summing the distribution uncertainties of all pre-defined quality inspection indicator states under this composite abnormal quality inspection state), and the number of training samples is 100; the total distribution uncertainty corresponding to "image blurry and network connection interruption" is 1.0, and the number of training samples is 80. The server may use weighted average or other methods to calculate the distribution cost parameter, and the calculation formula is as follows: Where H k It is the sum of the distribution uncertainties of various composite abnormal quality inspection conditions, n k This represents the number of training samples for each composite anomaly quality inspection condition. Substituting the data, the distribution cost parameter is calculated. Finally, the server determines the cost parameter of the quality inspection status recognition network based on the category cost parameter and the distribution cost parameter. For example, if the category cost parameter is 2.5 and the distribution cost parameter is 1.8, the server can use a weighted summation method (weighting coefficients w1 = 0.6 and w2 = 0.4), i.e., cost parameter C = w1 × C. c +w2×C d =0.6×2.5+0.4×1.8=2.02. This 2.02 is the final cost parameter of the quality inspection status identification network, which is used for subsequent updates and optimizations of the network parameters.

[0103] In one possible implementation, the step of extracting state features from the preset quality inspection index state and each index state category to be matched through the quality inspection state recognition network to obtain the preset multimodal feature vector of the preset quality inspection index state and the preset category feature vector of each index state category to be matched can be implemented through the following example.

[0104] The preset quality inspection index status is represented by a quality inspection status identification network to obtain the preset quality inspection index status quality inspection index features.

[0105] The quality inspection status recognition network sequentially performs multimodal status representation processing on each state category of the indicator to be matched, and obtains the sample state category features of each state category of the indicator to be matched; based on the sample state category features of each state category of the indicator to be matched and the adaptive and malleable industry prior knowledge vector, a feature set is constructed.

[0106] The preset quality inspection index status quality inspection index features and the feature set are subjected to cross-modal gating interaction to obtain the preset multimodal feature vector of the preset quality inspection index status and the preset category feature vector of each index status category to be matched.

[0107] The step of updating the network parameters in the quality inspection status recognition network according to the matching coefficient and the preset indicator status category of the preset quality inspection indicator status to obtain the trained quality inspection status recognition network can be implemented through the following example.

[0108] Based on the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network and the industry prior knowledge vector are updated to obtain the trained quality inspection state recognition network.

[0109] In this embodiment of the invention, the process of server-side quality monitoring of smartwatch production is illustrated in detail, for example. The preset quality inspection indicators for smartwatches encompass various aspects, including heart rate monitoring accuracy, motion trajectory recording precision, and screen touch sensitivity. The categories of indicators to be matched include "large heart rate monitoring deviation," "inaccurate motion trajectory recording," and "unresponsive screen touch." The server utilizes a quality inspection status recognition network to perform multimodal status representation processing on the preset quality inspection indicator statuses. For example, for heart rate monitoring accuracy data, the network transforms it into a vector form that reflects the range of heart rate fluctuations and the degree of deviation from the standard value, obtaining the quality inspection indicator features of the preset quality inspection indicator status in the heart rate monitoring dimension. For motion trajectory recording precision data, it transforms it into a vector reflecting the degree of trajectory deviation and positioning error, and so on. By integrating the data processing results from various dimensions, the preset quality inspection indicator status quality inspection indicator features are obtained. Simultaneously, the server utilizes adaptive and malleable industry prior knowledge vectors. For example, industry standards and experience regarding heart rate monitoring, motion trajectory recording, and screen touch in smartwatches are transformed into vector form. Then, based on the sample state category features of each target indicator state category and the industry prior knowledge vector, a feature set is constructed. The server performs cross-modal gating interaction between the pre-set quality inspection indicator state features and the feature set. First, the quality inspection indicator features of the heart rate monitoring dimension are selected, allowing them to interact with elements in the feature set. By analyzing the relationship between heart rate features and state category features such as "large heart rate monitoring deviation" and the industry prior knowledge vector, a complex algorithm is used to obtain the graph convolution feature vector of the pre-set quality inspection indicator state in the heart rate monitoring dimension and the graph convolution feature vector of each target indicator state category such as "large heart rate monitoring deviation". The step of selecting quality inspection indicator features of different dimensions for interaction is executed iteratively until all dimensions are processed. Based on the graph convolution feature vector obtained each time, the pre-set multimodal feature vector of the pre-set quality inspection indicator state and the pre-set category feature vector of each target indicator state category are finally obtained. The server updates the network parameters and industry prior knowledge vector in the quality inspection state recognition network according to the matching coefficient and the pre-set indicator state category of the pre-set quality inspection indicator state. For example, if a pre-defined quality inspection indicator is actually "large heart rate monitoring deviation," but the network gives a low matching coefficient, it indicates that the network prediction is inaccurate. The server uses techniques such as backpropagation algorithms to adjust the network's weights, biases, and other parameters to make the network prediction more accurate. Simultaneously, if it is found that the correlation between heart rate monitoring deviation and inaccurate motion trajectory recording in actual production differs from the original industry prior knowledge vector, the server will adjust the industry prior knowledge vector based on the data in the current training samples to better reflect actual production conditions. After multiple such update processes, the trained quality inspection status recognition network is finally obtained, which can more accurately monitor the quality of the smartwatch production process.

[0110] This invention provides a computer device 100, which includes a processor and a non-volatile memory storing computer instructions. When the computer instructions are executed by the processor, the computer device 100 executes the aforementioned intelligent monitoring method for production process quality based on deep learning. Figure 2 As shown, Figure 2 This is a structural block diagram of a computer device 100 provided in an embodiment of the present invention. The computer device 100 includes a memory 111, a processor 112, and a communication unit 113. To enable data transmission or interaction, the memory 111, processor 112, and communication unit 113 are electrically connected to each other directly or indirectly. For example, these components can be electrically connected to each other through one or more communication buses or signal lines.

[0111] For illustrative purposes, the foregoing description has been made with reference to specific embodiments. However, the foregoing illustrative discussions are not intended to be exhaustive or to limit the present disclosure to the precise forms disclosed. Numerous modifications and variations are possible in accordance with the foregoing teachings. These embodiments were chosen and described in order to best illustrate the principles of the present disclosure and its practical application, thereby enabling those skilled in the art to best utilize the disclosure and to employ various embodiments with different modifications to suit a particular intended application.

Claims

1. A deep learning-driven intelligent monitoring method for production process quality, characterized in that, include: The multimodal quality inspection index status and multiple matching index status categories of the target product are obtained, and the status features of the multimodal quality inspection index status and each matching index status category are extracted sequentially to obtain the multimodal feature vector of the multimodal quality inspection index status and the category feature vector of each matching index status category. Based on the multimodal feature vector, calculate the state confidence distribution of the multimodal quality inspection index state belonging to each composite abnormal quality inspection condition; Obtain the relational topology structure corresponding to each composite abnormal quality inspection status, wherein the relational topology structure reflects the mutual influence relationship between the status categories of each indicator to be matched under the composite abnormal quality inspection status. Based on the state confidence distribution and the relational topology corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the category feature vector of each target indicator state category to obtain the deep category feature vector corresponding to each target indicator state category. Based on the deep category feature vector corresponding to each state category of the indicator to be matched, the matching coefficient of each state category of the indicator to be matched to the state of the multimodal quality inspection indicator is determined. Based on the matching coefficient, the target indicator state category corresponding to the multimodal quality inspection indicator state is determined from each of the indicator state categories to be matched as the quality monitoring result of the target product.

2. The method according to claim 1, characterized in that, The step of sequentially extracting state features from the multimodal quality inspection index states and each category of index state to be matched, to obtain the multimodal feature vector of the multimodal quality inspection index states and the category feature vector of each category of index state to be matched, includes: The multimodal quality inspection index status is subjected to multimodal status characterization processing on the quality inspection index data in at least one quality inspection dimension to obtain the quality inspection index features of the multimodal quality inspection index status in at least one quality inspection dimension. Multimodal state representation processing is performed on each state category of the indicator to be matched in turn to obtain the state category features of each state category of the indicator to be matched. A feature set is constructed based on the state category features of each indicator state category to be matched and the industry prior knowledge vector; The quality inspection indicator features and the feature set are subjected to cross-modal gating interaction to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched.

3. The method according to claim 2, characterized in that, The step of performing cross-modal gating interaction between the quality inspection indicator features and the feature set to obtain the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched includes: The quality inspection indicator characteristics of the target quality inspection dimension corresponding to this cycle are determined from the quality inspection indicator characteristics of the at least one quality inspection dimension. Cross-modal gating interaction is performed on the quality inspection index features of the target quality inspection dimension and the feature set to obtain the graph convolution feature vector of the multimodal quality inspection index state and the graph convolution feature vector of each index state category to be matched. Based on the graph convolutional feature vectors of the multimodal quality inspection index states and the graph convolutional feature vectors of each index state category to be matched, the current feature set is determined. The process of determining the quality inspection indicator features of the target quality inspection dimension corresponding to the current cycle from the quality inspection indicator features of at least one quality inspection dimension is repeated until the process ends after all quality inspection indicator features of all quality inspection dimensions are obtained. Based on the current feature set, the multimodal feature vector of the multimodal quality inspection indicator state and the category feature vector of each indicator state category to be matched are obtained.

4. The method according to claim 1, characterized in that, The relational topology includes the category object corresponding to each state category of the indicator to be matched, and the path between the category objects, wherein the path reflects the mutual influence relationship between two connected category objects. Based on the state confidence distribution and the relational topology corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the category feature vector of each to-be-matched indicator state category to obtain the deep category feature vector corresponding to each to-be-matched indicator state category, including: For each category of the indicator status to be matched, determine the target category object corresponding to the category of the indicator status in the relational topology structure corresponding to each composite abnormal quality inspection status; For each composite abnormal quality inspection situation, based on the relational topology structure corresponding to the target category object and other category objects in the relational topology structure, the associated feature vector of the target category object under the composite abnormal quality inspection situation is determined. Based on the state confidence distribution, the associated feature vectors of the target category objects under each composite abnormal quality inspection condition are merged to obtain the merged feature vector of the state category of the indicator to be matched. Based on the merged feature vector, the category feature vector of the state category of the indicator to be matched is optimized to obtain the deep category feature vector corresponding to the state category of the indicator to be matched.

5. The method according to claim 4, characterized in that, The state confidence distribution includes the confidence value of the multimodal quality inspection index state belonging to each composite abnormal quality inspection condition; Based on the state confidence distribution, the associated feature vectors of target category objects under each composite abnormal quality inspection condition are merged to obtain the merged feature vector of the state category of the indicator to be matched, including: For each composite abnormal quality inspection situation, the confidence value of the multimodal quality inspection index state belonging to the composite abnormal quality inspection situation is merged with the associated feature vector of the target category object under the composite abnormal quality inspection situation to obtain the basic merged feature of the index state category to be matched under the composite abnormal quality inspection situation. Based on the basic merging features of the state category of the indicator to be matched under each composite abnormal quality inspection condition, the merging feature vector of the state category of the indicator to be matched is determined.

6. The method according to claim 1, characterized in that, The step of determining the matching coefficient of each target indicator state category to the multimodal quality inspection indicator state based on the deep category feature vector corresponding to each target indicator state category includes: For each category of indicator state to be matched, the category feature vector and deep category feature vector of the category of indicator state to be matched are merged to obtain the target category feature vector of the category of indicator state to be matched. Based on the target category feature vector of each target indicator state category, determine the matching coefficient of each target indicator state category belonging to the multimodal quality inspection indicator state.

7. The method according to claim 1, characterized in that, The step of sequentially extracting state features from the multimodal quality inspection index states and each category of index state to be matched, to obtain the multimodal feature vector of the multimodal quality inspection index states and the category feature vector of each category of index state to be matched, includes: The quality inspection status recognition network sequentially extracts status features from the multimodal quality inspection index status and each category of the index to be matched, thereby obtaining the multimodal feature vector of the multimodal quality inspection index status and the category feature vector of each category of the index to be matched.

8. The method according to claim 7, characterized in that, Before the step of extracting state features from the multimodal quality inspection indicator states and each matching indicator state category sequentially through the quality inspection state recognition network to obtain the multimodal feature vector of the multimodal quality inspection indicator states and the category feature vector of each matching indicator state category, the method further includes: Obtain training samples and the relational topology structure corresponding to each composite abnormal quality inspection status. The training samples include multiple preset quality inspection index states and preset index state categories of the preset quality inspection index states. Through the quality inspection status recognition network, the status features of the preset quality inspection index status and each index status category to be matched are extracted sequentially to obtain the preset multimodal feature vector of the preset quality inspection index status and the preset category feature vector of each index status category to be matched. Based on the preset multimodal feature vector, calculate the state confidence distribution of the preset quality inspection index state belonging to each composite abnormal quality inspection state; Based on the state confidence distribution and the relational topology corresponding to each composite abnormal quality inspection status, feature embedding processing is performed on the preset category feature vector of each target indicator state category to obtain the preset deep category feature vector corresponding to each target indicator state category. Based on the preset deep category feature vector corresponding to each state category of the indicator to be matched, determine the matching coefficient of each state category of the indicator to be matched belonging to the preset quality inspection indicator state. Based on the matching coefficient and the preset indicator state category of the preset quality inspection indicator state, the network parameters in the quality inspection state recognition network are updated to obtain the trained quality inspection state recognition network.

9. The method according to claim 8, characterized in that, The step of updating the network parameters in the quality inspection status recognition network according to the matching coefficient and the preset indicator status category of the preset quality inspection indicator status to obtain the trained quality inspection status recognition network includes: The cost parameters of the quality inspection status identification network are determined based on the matching coefficient and the preset indicator status category of the preset quality inspection indicator status. When the cost parameter does not reach a convergence state, the network parameters in the quality inspection status identification network are updated based on the cost parameter. Based on the confidence distribution of the state of each composite abnormal quality inspection condition, each pre-set quality inspection indicator state is divided to obtain the pre-set quality inspection indicator state group corresponding to each composite abnormal quality inspection condition. For each composite abnormal quality inspection situation, based on the preset indicator state category of the preset quality inspection indicator state in the preset quality inspection indicator state group corresponding to the composite abnormal quality inspection situation, the mutual influence relationship of each indicator state category to be matched in the relation topology structure corresponding to the composite abnormal quality inspection situation is optimized to obtain the optimized relation topology structure corresponding to the composite abnormal quality inspection situation. Based on the updated quality inspection status recognition network and the optimized relational topology corresponding to each composite abnormal quality inspection status, the step of extracting state features from the preset quality inspection index status and each index status category to be matched is executed cyclically through the quality inspection status recognition network until the cost parameter of the quality inspection status recognition network reaches a convergence state, thus obtaining the trained quality inspection status recognition network.

10. A server-based system, characterized in that, Includes a server, the server being used to perform the method according to any one of claims 1-9.

Images