Intelligent perception and collaborative optimization platform of industrial internet of things

Abstract

The invention discloses an industrial Internet of Things intelligent perception and collaborative optimization platform, which relates to the technical field of industrial Internet of Things and comprises an industrial visual intelligent perception unit, a wide-area knowledge sharing system unit, a multi-modal collaborative optimization unit and an edge cloud collaborative architecture unit. According to the method, a cross-industry knowledge sharing mechanism is constructed, so that rapid migration and adaptation of a lightweight detection model are realized, and the repeated development and deployment cost of the model is greatly reduced; a digital twinning technology is applied, and accurate regulation and optimization of process parameters are realized through virtual-real linkage; an edge cloud collaborative architecture is adopted, so that the real-time response requirement of a production site is ensured, and continuous evolution of the system is realized through a data closed loop, thereby effectively solving the problems of high cost, poor adaptability, optimization lag and the like faced by intelligent upgrading in traditional industrial production; and a complete and reliable technical support system is provided for digital transformation of the manufacturing industry.

Description

Technical Field

[0001] This invention relates to the field of industrial Internet of Things (IoT) technology, and more specifically, to an industrial IoT intelligent sensing and collaborative optimization platform. Background Technology

[0002] The Industrial Internet of Things (IIoT) intelligent sensing and collaborative optimization platform constructs an integrated "sensing-analysis-decision" architecture for multiple industrial scenarios. Through high-precision visual inspection and multi-source data fusion, it achieves intelligent management of the entire production process. Based on an industrial camera network and various sensors, the platform establishes a comprehensive data acquisition system to accurately sense product defects, equipment status, and environmental risks. Relying on a distributed architecture and knowledge-sharing mechanism, it achieves dynamic optimization of production parameters and intelligent resource scheduling. The system effectively overcomes bottlenecks such as industry data silos and response lags, forming a closed-loop control system with "edge-cloud" three-level collaboration, and is widely used in discrete manufacturing, process industries, and other fields. Through continuous technological iteration and functional optimization, the platform significantly improves production quality and operational efficiency, promotes the innovative development of the IIoT towards generalization and intelligence, and provides strong technical support for the digital transformation of industrial enterprises.

[0003] However, existing technologies lack cross-scenario knowledge sharing mechanisms, resulting in redundant model development and poor adaptability; data perception and decision control are isolated from each other, making it difficult to achieve accurate optimization of process parameters; and the efficiency of edge and cloud collaboration is low, failing to support real-time closed-loop control, thereby reducing the improvement of the level of intelligent industrial production.

[0004] There are currently no effective solutions to the problems in the relevant technologies. Summary of the Invention

[0005] To address the shortcomings of existing technologies, this invention proposes an industrial IoT intelligent sensing and collaborative optimization platform. This platform solves the problems mentioned in the background, such as the lack of a cross-scenario knowledge sharing mechanism, leading to repetitive model development and poor adaptability; the isolation between data sensing and decision control, making it difficult to achieve accurate process parameter optimization; and the low efficiency of edge and cloud collaboration, which cannot support real-time closed-loop control, thereby reducing the improvement of the level of intelligent industrial production.

[0006] To achieve the above objectives, the present invention provides the following technical solution: Industrial IoT intelligent sensing and collaborative optimization platform, including: The industrial vision intelligent sensing unit is used to acquire product image data from industrial production lines. It uses multi-scale attention networks and zero-shot detection technology to identify product image data and obtain product visual defect information and equipment operation information. The wide-area knowledge sharing system unit is used to build a cross-industry industrial vision database. It uses domain adaptive transfer and knowledge distillation technology to fuse and analyze product visual defect information, equipment operation information and cross-industry industrial vision databases to build a lightweight detection model and generate optimized detection solutions. The multimodal collaborative optimization unit is used to correlate and locate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme, and dynamically adjust process parameters in a digital twin environment based on the correlation and location results to generate optimization strategies. The edge-cloud collaborative architecture unit is used to deploy optimization strategies and lightweight detection models to the edge, utilize dynamic model pruning and multi-level caching mechanisms for real-time control, and feed back edge execution data to the cloud to form a closed-loop optimization.

[0007] Furthermore, the industrial vision intelligent sensing unit includes: The multi-scale attention network module is used to identify the acquired product image data using a multi-scale attention network, and to locate product defects and obtain initial visual defect information through a transformer convolutional neural network architecture and dynamic receptive field technology. The zero-shot defect detection module is used to identify unknown defects in product image data using zero-shot detection technology. Based on cross-modal alignment technology, it generates virtual samples through text descriptions for inference, obtains classification results of new defect types, and adds the classification results to the visual defect information to obtain the final visual defect information. The adversarial sample defense module is used to associate visual defect information with the device identifier and timestamp at the time of image acquisition using adversarial defense technology to obtain product visual defect information and equipment operation information; Among them, the industrial vision intelligent perception unit is connected through the wide-area knowledge sharing system unit and the multimodal collaborative optimization unit, the multimodal collaborative optimization unit is connected through the edge-cloud collaborative architecture unit, and the multi-scale attention network module is connected through the zero-shot defect detection module and the adversarial sample defense module.

[0008] Furthermore, a multi-scale attention network is used to identify the acquired product image data, and product defects are located through a transformer convolutional neural network architecture and dynamic receptive field technology, yielding initial visual defect information including: The acquired product image data is standardized, denoised, and normalized in size to obtain the processed product image data. The processed product image data is input into the converter convolutional neural network architecture, and the convolutional neural backbone network is used to extract product image features at different resolutions in parallel to generate multi-scale feature sequences. The multi-scale feature sequence is input into the transformer encoder, the feature weight distribution is dynamically calculated through the self-attention mechanism, and the feature weight distribution is searched using a search algorithm to locate the key areas of potential defects in the product image. Based on the key regions of potential defects, dynamic receptive field technology is used to adaptively adjust the perception range of the convolution kernel according to the product material characteristics, thereby obtaining optimized product image features with multi-granularity defect features. By using a detection head network to perform bounding box regression and classification on optimized product image features, the coordinates, category, and confidence level of product defects are obtained. The coordinates, categories, and confidence information of all defects are merged and analyzed to generate structured initial visual defect information containing defect type, location, and confidence level.

[0009] Furthermore, the multi-scale feature sequence is input into the transformer encoder, and the feature weight distribution is dynamically calculated through a self-attention mechanism. A search algorithm is then used to perform a region search on the feature weight distribution to locate potential defective key regions in the product image, including: Initialize the parameters of the search algorithm and set the maximum number of iterations; The multi-scale feature sequence is input into the transformer encoder, an initial response value is assigned to each feature region, and the global feature weight probability distribution is dynamically calculated through a self-attention mechanism. Based on the calculated feature weight distribution, the influence of all feature regions is evaluated using an evaluation algorithm, and the response sequence of key regions to be processed is generated in descending order of influence. Select the feature region with the highest current weight from the key region response sequence, and sample a set of semantically adjacent feature blocks from the neighborhood of the feature region with the highest weight based on feature similarity; The weighted response of the current region is optimized based on the sampled feature blocks, and the weighted feature center of the sampled feature blocks is calculated. The feature block that best matches the center is identified as the defect focal region and marked. Repeat the neighborhood feature sampling and weight response optimization process. If the maximum number of iterations is reached, output the key areas of potential defects in the located product image.

[0010] Furthermore, based on the calculated feature weight distribution, an evaluation algorithm is used to assess the influence of all feature regions, and the response sequences of the key regions to be processed are generated in descending order of influence, including: Initialize the parameters of the evaluation algorithm and set the maximum number of iterations; An evaluation queue is initialized in the feature weight distribution using a spatial uniform sampling method to ensure full coverage of the feature space; Calculate the initial influence of all feature regions in the evaluation queue, sort them, and identify the feature region with the highest current global influence. In the coarse search phase, an alternating selection algorithm is used to perform a global and local search on the evaluation queue to discover the region with the highest influence. During the detailed search phase, the evaluation queue assesses the highest-impact areas discovered, using finer analysis to improve the accuracy of regional impact assessments. The process of repeating influence identification, coarse search, and fine search, if the maximum number of iterations is reached, outputs a sequence of key region responses to be processed in descending order of influence.

[0011] Furthermore, in the coarse search phase, an alternating selection algorithm is used to perform global and local searches on the evaluation queue to identify the regions with the highest influence, including: Initialize the parameters of the alternating selection algorithm, set the maximum number of iterations, and randomly select an initial region from the evaluation queue as the starting point of the current search; Search the evaluation queue for candidate regions that can be combined with the current region to enhance the region's influence, and form new region combinations; Verify whether the new region combination meets the preset conditions; if not, select the next best candidate in turn for verification until the optimal region combination that meets the preset conditions is found. Repeat the region combination search and verification process to expand the region combination until the influence of the region combination reaches a local optimum; A random perturbation is introduced, and a random region is received based on the current optimal region combination. After the characteristics of the random region are changed, a region combination search process is carried out to search for new potential optimal region combinations. Compare the influence of the new and old optimal region combinations, retain the optimal region combination with the highest influence, and if the maximum number of iterations is reached, output the optimal region combination with the highest global influence as the region with the highest influence.

[0012] Furthermore, the units of the wide-area knowledge-sharing system include: The industry knowledge base construction module is used to acquire cross-industry product defect samples, build a cross-industry industrial vision database, and transfer the capabilities of the vision transformer model to a lightweight network through knowledge distillation technology to build a lightweight detection model. The cross-domain transfer learning module is used to perform fusion analysis on product visual defect information, equipment operation information and cross-industry industrial vision databases using domain adaptive transfer technology, and to match and recommend the fusion analysis results through feature topology maps to generate optimized detection solutions. The industry knowledge base construction module and the cross-domain transfer learning module are connected.

[0013] Furthermore, the multimodal cooperative optimization unit includes: The visual sensing data fusion module is used to fuse and correlate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme to locate the root cause of process abnormalities that lead to defects. The dynamic process control module is used to dynamically adjust process parameters in a digital twin environment based on the root causes of process anomalies, and to verify the effects using a virtual trial and error engine, ultimately generating an executable optimization strategy. The visual sensing data fusion module and the dynamic process control module are connected.

[0014] Furthermore, based on the root causes of process anomalies, process parameters are dynamically adjusted in the digital twin environment, and the effects are verified using a virtual trial-and-error engine. Ultimately, executable optimization strategies are generated, including: Initialize the parameters of the control algorithm and set the maximum number of iterations; Initialize the digital twin environment, define the process parameter space, and simultaneously initialize the optimization strategy exploration path, starting from the current abnormal state and the target state; Candidate process parameters are randomly generated, and a dynamic step size strategy is used to adjust the process parameters to improve search efficiency and convergence speed. New process parameters are determined by adaptively calculating multidimensional optimization gradients, and the process parameter optimization path is reconstructed accordingly to achieve better convergence. The feasibility of new process parameters is verified in a virtual trial-and-error engine. If the simulation fails, it is iterated again; if it succeeds, the optimization effect is evaluated. Merge the validated effective process parameters, and if the maximum number of iterations is reached, output an executable optimization strategy.

[0015] Furthermore, the formula for adjusting process parameters using a dynamic step size strategy is as follows: ; In the formula, W new This indicates the adjusted process parameters; W current Indicates the current process parameters; S 0 represents the initial step size; λ Indicates the attenuation coefficient; ω Indicates historical guidance weight; Δ W hist Δ represents the deviation of historically optimal process parameters. W grad Indicates the direction of the performance gradient; t Indicates the current iteration number; T max Indicates the maximum number of iterations; e It represents the base of the natural logarithm.

[0016] The beneficial effects of this invention are as follows: 1. This invention achieves rapid migration and adaptation of lightweight testing models by constructing a cross-industry knowledge-sharing mechanism, significantly reducing the costs of repetitive model development and deployment; it applies digital twin technology to achieve precise control and optimization of process parameters through virtual-physical linkage; and it adopts an edge-cloud collaborative architecture, ensuring real-time response to production site needs while enabling continuous system evolution through data closure. This effectively solves the problems of high cost, poor adaptability, and lagging optimization faced by intelligent upgrades in traditional industrial production, providing a complete and reliable technical support system for the digital transformation of the manufacturing industry.

[0017] 2. This invention achieves deep feature extraction of product images by combining a multi-scale attention network with a transformer encoder; it adaptively adjusts the detection granularity using dynamic receptive field technology to effectively identify minute defects in different materials; it accurately locates potential defect areas through the collaborative work of search and evaluation algorithms; and it improves detection efficiency by employing a multi-level optimization strategy, thereby enhancing the accuracy and adaptability of defect detection. This solves the problem of insufficient ability of traditional methods to identify complex defects and provides reliable technical support for industrial quality control.

[0018] 3. By organically combining a digital twin environment with a virtual trial-and-error engine, intelligent optimization and control of process parameters are achieved. A dynamic step-size adjustment strategy is adopted, introducing both historical parameter deviations and performance gradient directions for guidance. This ensures both search efficiency and convergence accuracy, effectively solving the problems of reliance on manual experience, long debugging cycles, and difficulty in obtaining optimal solutions in traditional process parameter optimization, thus enhancing the adaptive optimization capability of the production process. Attached Figure Description

[0019] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0020] Figure 1 This is a principle block diagram of an industrial Internet of Things intelligent sensing and collaborative optimization platform according to an embodiment of the present invention; Figure 2 This is an application flowchart of the multi-scale attention network module in the industrial Internet of Things intelligent sensing and collaborative optimization platform according to an embodiment of the present invention.

[0021] In the picture: 1. Industrial vision intelligent perception unit; 2. Wide-area knowledge sharing system unit; 3. Multimodal collaborative optimization unit; 4. Edge-cloud collaborative architecture unit. Detailed Implementation

[0022] The technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0023] In the description of this invention, unless otherwise stated, "a plurality of" means two or more. Furthermore, the terms "first," "second," "third," etc., are used for descriptive purposes only and should not be construed as indicating or implying relative importance.

[0024] According to an embodiment of the present invention, an industrial Internet of Things (IoT) intelligent sensing and collaborative optimization platform is provided.

[0025] The present invention will now be further described in conjunction with the accompanying drawings and specific embodiments, such as... Figures 1-2 As shown, the industrial IoT intelligent sensing and collaborative optimization platform according to an embodiment of the present invention includes: The industrial vision intelligent sensing unit 1 is used to acquire product image data from the industrial production line, and to identify the product image data using multi-scale attention network and zero-shot detection technology to obtain product visual defect information and equipment operation information. Unit 2 of the wide-area knowledge sharing system is used to build a cross-industry industrial vision database. It uses domain adaptive transfer and knowledge distillation technology to fuse and analyze product visual defect information, equipment operation information and cross-industry industrial vision database in order to build a lightweight detection model and generate an optimized detection solution. The multimodal collaborative optimization unit 3 is used to correlate and locate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme, and dynamically adjust process parameters in the digital twin environment according to the correlation and location results to generate optimization strategies. Edge-cloud collaborative architecture unit 4 is used to deploy optimization strategies and lightweight detection models to the edge, utilize dynamic model pruning and multi-level caching mechanisms for real-time control, and feed back edge execution data to the cloud to form a closed-loop optimization.

[0026] In this optional embodiment, the industrial vision intelligent sensing unit 1 includes: The multi-scale attention network module is used to identify the acquired product image data using a multi-scale attention network, and to locate product defects and obtain initial visual defect information through a transformer convolutional neural network architecture and dynamic receptive field technology. The zero-shot defect detection module is used to identify unknown defects in product image data using zero-shot detection technology. Based on cross-modal alignment technology, it generates virtual samples through text descriptions for inference, obtains classification results of new defect types, and adds the classification results to the visual defect information to obtain the final visual defect information. The adversarial sample defense module is used to associate visual defect information with the device identifier and timestamp at the time of image acquisition using adversarial defense technology to obtain product visual defect information and equipment operation information; Among them, the industrial vision intelligent perception unit 1 is connected to the wide-area knowledge sharing system unit 2 and the multimodal collaborative optimization unit 3, the multimodal collaborative optimization unit 3 is connected to the edge cloud collaborative architecture unit 4, and the multi-scale attention network module is connected to the zero-shot defect detection module and the adversarial sample defense module.

[0027] In this optional embodiment, a multi-scale attention network is used to identify the acquired product image data, and product defects are located using a transformer convolutional neural network architecture and dynamic receptive field technology to obtain initial visual defect information, including: The acquired product image data is standardized, denoised, and normalized in size to obtain the processed product image data. The processed product image data is input into the converter convolutional neural network architecture, and the convolutional neural backbone network is used to extract product image features at different resolutions in parallel to generate multi-scale feature sequences. The multi-scale feature sequence is input into the transformer encoder, the feature weight distribution is dynamically calculated through the self-attention mechanism, and the feature weight distribution is searched using a search algorithm to locate the key areas of potential defects in the product image. Based on the key regions of potential defects, dynamic receptive field technology is used to adaptively adjust the perception range of the convolution kernel according to the product material characteristics, thereby obtaining optimized product image features with multi-granularity defect features. By using a detection head network to perform bounding box regression and classification on optimized product image features, the coordinates, category, and confidence level of product defects are obtained. The coordinates, categories, and confidence information of all defects are merged and analyzed to generate structured initial visual defect information containing defect type, location, and confidence level.

[0028] Specifically, high-definition wheel hub images captured by production line cameras undergo standardization, noise reduction, and size normalization preprocessing to eliminate uneven lighting, noise, and unify input specifications. Next, the processed images are input into a transformer-convolutional neural network, whose convolutional backbone extracts multi-scale feature sequences from high-resolution details to low-resolution semantics in parallel. This sequence is then fed into the transformer encoder, which dynamically calculates global feature weights through a self-attention mechanism and uses a search algorithm to pinpoint key regions of potential defects such as micropores and scratches. Based on the located key regions, the system employs dynamic receptive field technology, automatically narrowing the receptive field of the convolutional kernel to focus on micro-scratches for the smooth surface characteristics of aluminum alloys, while expanding the receptive field for rough casting areas to capture diffuse porosity clusters, thereby generating an optimized feature map with multi-granularity defect features. The detection head network performs bounding box regression and classification on these optimized features, accurately outputting the coordinates, category (e.g., scratches, pores, dents), and confidence level of each defect. The system integrates and analyzes all defect information to generate a structured initial visual defect report, clearly listing the defect type, pixel-level location, and detection confidence level.

[0029] In this optional embodiment, a multi-scale feature sequence is input into the transformer encoder, the feature weight distribution is dynamically calculated through a self-attention mechanism, and a search algorithm is used to perform a region search on the feature weight distribution to locate potential defect key regions in the product image, including: Initialize the parameters of the search algorithm and set the maximum number of iterations; The multi-scale feature sequence is input into the Transformer encoder, an initial response value is assigned to each feature region, and the global feature weight probability distribution is dynamically calculated through a self-attention mechanism. Based on the calculated feature weight distribution, the influence of all feature regions is evaluated using an evaluation algorithm, and the response sequence of key regions to be processed is generated in descending order of influence. Select the feature region with the highest current weight from the key region response sequence, and sample a set of semantically adjacent feature blocks from the neighborhood of the feature region with the highest weight based on feature similarity; The weighted response of the current region is optimized based on the sampled feature blocks, and the weighted feature center of the sampled feature blocks is calculated. The feature block that best matches the center is identified as the defect focal region and marked. Repeat the neighborhood feature sampling and weight response optimization process. If the maximum number of iterations is reached, output the key areas of potential defects in the located product image.

[0030] Specifically, the multi-scale feature sequences extracted from the convolutional backbone network are input into the Transformer encoder. The encoder uses a self-attention mechanism to perform global correlation calculations on the features of the entire image, dynamically generating a feature weight probability distribution that highlights potential anomaly regions. Based on this weight distribution, the system uses an evaluation algorithm to assess the influence of all feature regions and generates a key region response sequence arranged in descending order of influence. The algorithm selects the feature region with the highest weight at the top of the sequence as the starting point, and samples a set of semantically similar feature blocks from its surrounding neighborhood by calculating feature similarity. These feature blocks are used to optimize the weight response value of the current region, and by calculating their weighted feature centers, the feature block that best matches the center is identified as the defect focal region and marked. The system continuously iterates through the neighborhood sampling and weight optimization process, gradually and accurately locating other key regions until the maximum number of iterations is reached, and finally outputs the precise coordinates of all potential defect key regions.

[0031] Specifically, the search algorithm is a label propagation algorithm, a semi-supervised learning algorithm based on a graph model. It constructs a graph structure through the similarity between samples and iteratively propagates label information so that similar samples receive the same label. In this invention, the algorithm treats feature regions as graph nodes and uses weights generated by a self-attention mechanism as initial influence. By sampling semantically neighboring feature blocks from high-weight regions and iteratively optimizing the region response values, the localization and labeling of the defect focus region are achieved. This method associates discrete feature blocks into a graph structure and dynamically determines key defect regions through a propagation mechanism.

[0032] In this optional embodiment, based on the calculated feature weight distribution, an evaluation algorithm is used to assess the influence of all feature regions, and a key region response sequence to be processed is generated in descending order of influence, including: Initialize the parameters of the evaluation algorithm and set the maximum number of iterations; An evaluation queue is initialized in the feature weight distribution using a spatial uniform sampling method to ensure full coverage of the feature space; Calculate the initial influence of all feature regions in the evaluation queue, sort them, and identify the feature region with the highest current global influence. In the coarse search phase, an alternating selection algorithm is used to perform a global and local search on the evaluation queue to discover the region with the highest influence. During the detailed search phase, the evaluation queue assesses the highest-impact areas discovered, using finer analysis to improve the accuracy of regional impact assessments. The process of repeating influence identification, coarse search, and fine search, if the maximum number of iterations is reached, outputs a sequence of key region responses to be processed in descending order of influence.

[0033] Specifically, a spatial uniform sampling method is employed to construct an initial evaluation queue within the feature weight distribution generated by the Transformer encoder, ensuring comprehensive coverage of the entire wafer surface feature space. The system calculates and sorts the initial influence of all feature regions in the queue, quickly identifying the suspected defect region with the highest current global influence. In the coarse search phase, the algorithm alternates between global exploration and local mining, efficiently locking down potential high-influence regions. In the detailed search phase, the system performs refined evaluation of candidate regions, continuously improving the accuracy of influence calculation through multiple rounds of feature analysis. The influence identification and two-stage search process are executed cyclically until the iteration termination condition is met, ultimately generating a key region response sequence sorted in descending order of influence, providing precise guidance for subsequent defect localization.

[0034] Specifically, the evaluation algorithm is the Kingfisher Optimization Algorithm, a metaheuristic algorithm inspired by the kingfisher's hunting behavior. It effectively balances global exploration and local exploitation by simulating the kingfisher's strategy of circling high in the air and diving into the water to hunt. In this invention, the algorithm treats feature regions as prey. After initializing the evaluation queue, a coarse search phase simulates global exploration, broadly discovering high-influence regions; a detailed search phase simulates local exploitation, refining candidate regions to improve accuracy. An alternating strategy dynamically optimizes the generation of response sequences for key regions.

[0035] In this optional embodiment, during the coarse search phase, an alternating selection algorithm is used to perform a global and local search on the evaluation queue to discover the regions with the highest influence, including: Initialize the parameters of the alternating selection algorithm, set the maximum number of iterations, and randomly select an initial region from the evaluation queue as the starting point of the current search; Search the evaluation queue for candidate regions that can be combined with the current region to enhance the region's influence, and form new region combinations; Verify whether the new region combination meets the preset conditions; if not, select the next best candidate in turn for verification until the optimal region combination that meets the preset conditions is found. Repeat the region combination search and verification process to expand the region combination until the influence of the region combination reaches a local optimum; A random perturbation is introduced, and a random region is received based on the current optimal region combination. After the characteristics of the random region are changed, a region combination search process is carried out to search for new potential optimal region combinations. Compare the influence of the new and old optimal region combinations, retain the optimal region combination with the highest influence, and if the maximum number of iterations is reached, output the optimal region combination with the highest global influence as the region with the highest influence.

[0036] Specifically, an initial region is randomly selected from the evaluation queue as the starting point for the search. The algorithm continuously searches the queue for candidate regions that can effectively complement the current region, generating new region combinations through combination verification. Each new combination must be verified to meet preset correlation and significance conditions. If the verification fails, the algorithm automatically switches to the next best candidate for further attempts until the current optimal region combination is selected. Through multiple iterations of search and verification, the influence of the region combination is gradually increased to a local optimum. To prevent getting trapped in a local optimum, the algorithm actively introduces random perturbation: a randomly modified region is introduced into the existing optimal combination, and the search process for region combinations is restarted based on this region to explore new potential optimization paths. The system compares the influence of the new and old optimal combinations and retains the global optimum as the region with the highest influence.

[0037] Specifically, the alternating selection algorithm is a greedy algorithm, which adopts the optimal solution in the current state at each step, hoping to achieve a global optimum through the accumulation of local optima. In this invention, the algorithm starts from a random initial region and continuously searches the evaluation queue for candidate regions that maximize the influence of the region combination. By gradually constructing and verifying locally optimal combinations, it quickly approaches the region with the highest influence. Although it may get stuck in local optima, a random perturbation strategy is introduced to escape this trap and continue searching for a better solution.

[0038] In this optional embodiment, the wide-area knowledge sharing system unit 2 includes: The industry knowledge base construction module is used to acquire cross-industry product defect samples, build a cross-industry industrial vision database, and transfer the capabilities of the vision transformer model to a lightweight network through knowledge distillation technology to build a lightweight detection model. The cross-domain transfer learning module is used to perform fusion analysis on product visual defect information, equipment operation information and cross-industry industrial vision databases using domain-adversarial transfer technology, and to match and recommend optimized detection solutions by using feature topology maps to match the fusion analysis results. The industry knowledge base construction module and the cross-domain transfer learning module are connected.

[0039] In this optional embodiment, the multimodal cooperative optimization unit 3 includes: The visual sensing data fusion module is used to fuse and correlate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme to locate the root cause of process abnormalities that lead to defects. The dynamic process control module is used to dynamically adjust process parameters in a digital twin environment based on the root causes of process anomalies, and to verify the effects using a virtual trial and error engine, ultimately generating an executable optimization strategy. The visual sensing data fusion module and the dynamic process control module are connected.

[0040] In this optional embodiment, based on the root cause of process anomalies, process parameters are dynamically adjusted in a digital twin environment, and the effects are verified using a virtual trial-and-error engine. The final executable optimization strategy includes: Initialize the parameters of the control algorithm and set the maximum number of iterations; Initialize the digital twin environment, define the process parameter space, and simultaneously initialize the optimization strategy exploration path, starting from the current abnormal state and the target state; Candidate process parameters are randomly generated, and a dynamic step size strategy is used to adjust the process parameters to improve search efficiency and convergence speed. New process parameters are determined by adaptively calculating multidimensional optimization gradients, and the process parameter optimization path is reconstructed accordingly to achieve better convergence. The feasibility of new process parameters is verified in a virtual trial-and-error engine. If the simulation fails, it is iterated again; if it succeeds, the optimization effect is evaluated. Merge the validated effective process parameters, and if the maximum number of iterations is reached, output an executable optimization strategy.

[0041] Specifically, a process parameter space is constructed within a digital twin environment. Using the current abnormal process state causing shrinkage marks and the target optimal state as endpoints, a search space for optimization paths is established. The algorithm rapidly approximates the optimization direction by randomly generating candidate parameter combinations and employing a dynamic step-size strategy to adjust key parameters such as injection speed and holding pressure. Based on adaptively calculated multidimensional optimization gradients, the system intelligently determines new parameter combinations and reconstructs the search path to improve convergence efficiency. Each new parameter combination undergoes injection molding process simulation in a virtual trial-and-error engine to verify its feasibility and evaluate the shrinkage mark improvement effect. After multiple rounds of iterative verification, the system finally merges all effective parameter adjustment schemes and outputs an optimization strategy containing specific parameter settings and execution order.

[0042] Specifically, the control algorithm is RRT. The algorithm is a stochastic sampling algorithm for path planning. It finds an optimal path from the starting point to the target point by progressively constructing a space-filling tree and continuously optimizing the path. In this invention, the algorithm treats process parameter adjustment as a path planning problem. Starting from the current abnormal state and ending at the target state, it randomly samples candidate process parameters in the parameter space of a digital twin environment to construct an optimized path. Through dynamic step size and path reconstruction strategies, it continuously verifies and optimizes parameter combinations, ultimately generating a feasible and efficient process optimization strategy.

[0043] In this optional embodiment, the formula for adjusting process parameters using a dynamic step size strategy is as follows: ; In the formula, W new This indicates the adjusted process parameters; W current Indicates the current process parameters; S 0 represents the initial step size; λ Indicates the attenuation coefficient; ω Indicates historical guidance weight; Δ W hist Δ represents the deviation of historically optimal process parameters. W grad Indicates the direction of the performance gradient; t Indicates the current iteration number; T max Indicates the maximum number of iterations; e It represents the base of the natural logarithm.

[0044] To facilitate understanding of the above technical solutions of the present invention, the following provides a detailed description of the general-purpose emergency panoramic command and intelligent decision-making system of the present invention in practical applications.

[0045] I. Industrial Vision Intelligent Sensing.

[0046] Application Scenarios: At the end of the LCD panel production line, visual defect inspection of finished products is required. Defect types are diverse, including common bright spots, dark spots, scratches, and mura (bright spots). Simultaneously, it is necessary to extract equipment operation information from the images, such as conveyor belt speed and camera shooting frequency.

[0047] 1) Image Acquisition: A high-resolution line scan camera scans the LCD panel at a constant speed, generating RGB images with a resolution of up to 8000x6000 pixels. The imaging time for each panel is 2.5 seconds.

[0048] 2) Multi-scale Attention Network Recognition: The image is input into a network with three parallel branches, processing feature maps of different scales (e.g., 112×112, 224×224, 448×448). An attention mechanism is used to direct the network to focus on potentially problematic areas. For example, when dealing with defects like Mura, which have low contrast and blurred boundaries, the attention weight map shows the network primarily focusing on the backlight uniformity of the panel, rather than the metal frame at the panel edge. The network outputs the defect classification probability for each pixel. A probability threshold > 0.95 is set to confirm a defect.

[0049] 3) Application of Zero-Shot Detection Technology: A novel defect, resembling fine lightning bolts, suddenly appears on the production line. Because this defect type is not in the training dataset, traditional models would miss it. Zero-shot detection technology matches image features with the semantic description of the defect (e.g., linear, bright, irregular) in a shared space, allowing it to be identified as an anomaly even without samples of the defect. The system labels this novel defect as unknown defect type A and records its feature vector.

[0050] The system detected two defects in a panel, as shown in Table 1: Defect 1: Type = Dark spot, Location coordinates = (1250, 880), Confidence level = 0.98.

[0051] Defect 2: Type = Unknown defect type A, Location coordinates = (3200, 2100), Confidence level = 0.91.

[0052] Equipment information: Conveyor belt speed during shooting = 0.5m / s, camera exposure time = 1ms.

[0053] Table 1 Visual Inspection Report for a Single Panel II. Wide-area knowledge sharing system.

[0054] Application Scenario: The causes of Mura defects in LCD panels are complex and related to multiple process parameters. Relying solely on in-house data is insufficient for quickly locating the root cause and optimizing the model. The system needs to draw upon knowledge from cross-industry databases (such as semiconductor silicon wafer inspection and automotive paint inspection).

[0055] 1) Database Query: The system queries cross-industry industrial vision databases for defect models related to low contrast, regionality, and uneven brightness. It was found that model features used in the semiconductor industry to detect uneven silicon wafer thickness, and models used in the automotive industry to detect orange peel texture in paint, show high similarity to the Mura defect.

[0056] 2) Domain Adaptive Transfer: The feature extractor of the silicon wafer thickness non-uniformity detection model (source domain) is transferred to the LCD panel (target domain). Domain adaptation technology minimizes the difference in feature distribution between the source domain (silicon wafer image) and the target domain (LCD image) (calculated using the Maximum Mean Difference, MMD). After calculation, the MMD value decreased from the initial 1.5 to 0.3, indicating that the features of the two domains have been aligned.

[0057] 3) Knowledge Distillation: The knowledge from a complex model (teacher model, 98.5% accuracy, 50M parameters) integrating knowledge of silicon wafers and automotive paint is distilled into a smaller model (student model) specifically designed for LCD panels. The distillation process uses not only real labels but also soft labels output by the teacher model, allowing the student model to learn subtle differences between various defects. The resulting lightweight detection model has only 5M parameters and maintains an accuracy of 97.8%, making it ideal for deployment on edge devices.

[0058] 4) The generated optimized detection scheme: Key takeaways: To address the Mura defect, it is recommended to adjust the image preprocessing workflow and add a homomorphic filtering step to enhance low-contrast features.

[0059] Model version: Lightweight Mura detection model v2.1 (5M parameters).

[0060] III. Multimodal collaborative optimization.

[0061] 1) Graph Neural Network Association Localization: A heterogeneous graph network was constructed. Nodes included defect types (dark spots, bright spots, mura, unknown defect A), equipment parameters (oven temperature, conveyor belt speed, static eliminator voltage), and process steps (cleaning, coating, baking). Data from the past month was input into the graph network for training. The GNN learned the complex relationships between nodes. The association localization results, including the GNN output edge weights, showed that the association weight between unknown defect A and the temperature of zone B of the oven was 0.76, and the association weight with the voltage of static eliminator #3 was 0.85. The association weights for other parameters were all <0.2.

[0062] 2) Dynamic Control of Digital Twin Environment: In the digital twin system, the temperature of zone B of the baking oven is lowered from 185°C to 182°C, and the voltage of electrostatic eliminator #3 is increased from 8.5KV to 9.2KV. System simulation predicts that this parameter adjustment will reduce the occurrence rate of unknown defect A from 5% to 0.8%, while having no significant impact on the occurrence rate of other defects (such as bright spots) (predicted change <0.1%).

[0063] 3) Optimization strategy: Adjust the temperature setting of zone B of the baking oven to 182°C ± 0.5°C. Set the voltage of static eliminator #3 to 9.2KV ± 0.1KV. The incidence of unknown defect A decreased by 4.2 percentage points.

[0064] IV. Edge-Cloud Collaborative Architecture.

[0065] Scenario: Quickly deploy lightweight models and optimization strategies generated in the cloud to the edge of the production line for real-time control and continuous data collection.

[0066] 1) Strategy and Model Deployment: The cloud will distribute the optimized strategy (new temperature and voltage settings) and the lightweight Mura detection model v2.1 (5M) to the edge server of the production line.

[0067] 2) Dynamic Model Pruning and Multi-Level Caching: Dynamic pruning involves the edge server performing further lossless pruning on the 5M model based on real-time load. When CPU load exceeds 80%, 10% of redundant convolutional kernels in the network are automatically removed to generate a 4.5M ultra-fast model, ensuring detection latency <100ms. Multi-level caching includes an L1 cache (memory): caching image data and high-frequency defect features of the 10 panels currently being detected. The L2 cache (edge ​​SSD) includes caching the lightweight Mura detection model v2.1 and its pruned version, as well as the detection results of the most recent 1000 panels.

[0068] 3) Real-time control and feedback: Upon receiving new process parameters, the edge controller immediately adjusts the baking oven and electrostatic eliminator. The new lightweight model performs real-time monitoring of subsequent panels.

[0069] 4) Closed-loop optimization: The edge side packages and feeds back the execution data (as shown in Table 2) and new defect images to the cloud. The cloud uses this new data, especially the occurrence of unknown defect A after parameter adjustment, to update the wide-area knowledge base and GNN model, preparing for the next round of optimization iteration.

[0070] Table 2 Edge-side execution data The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. An industrial IoT intelligent sensing and collaborative optimization platform, characterized in that, include: The industrial vision intelligent sensing unit is used to acquire product image data from industrial production lines. It uses multi-scale attention networks and zero-shot detection technology to identify product image data and obtain product visual defect information and equipment operation information. The wide-area knowledge sharing system unit is used to build a cross-industry industrial vision database. It uses domain adaptive transfer and knowledge distillation technology to fuse and analyze product visual defect information, equipment operation information and cross-industry industrial vision databases to build a lightweight detection model and generate optimized detection solutions. The multimodal collaborative optimization unit is used to correlate and locate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme, and dynamically adjust process parameters in a digital twin environment based on the correlation and location results to generate optimization strategies. The edge-cloud collaborative architecture unit is used to deploy optimization strategies and lightweight detection models to the edge, utilize dynamic model pruning and multi-level caching mechanisms for real-time control, and feed back edge execution data to the cloud to form a closed-loop optimization.

2. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 1, characterized in that, The industrial vision intelligent sensing unit includes: The multi-scale attention network module is used to identify the acquired product image data using a multi-scale attention network, and to locate product defects and obtain initial visual defect information through a transformer convolutional neural network architecture and dynamic receptive field technology. The zero-shot defect detection module is used to identify unknown defects in product image data using zero-shot detection technology. Based on cross-modal alignment technology, it generates virtual samples through text descriptions for inference, obtains classification results of new defect types, and adds the classification results to the visual defect information to obtain the final visual defect information. The adversarial sample defense module is used to associate visual defect information with the device identifier and timestamp at the time of image acquisition using adversarial defense technology to obtain product visual defect information and equipment operation information; The industrial vision intelligent perception unit is connected to the wide-area knowledge sharing system unit and the multimodal collaborative optimization unit, the multimodal collaborative optimization unit is connected to the edge-cloud collaborative architecture unit, and the multi-scale attention network module is connected to the zero-sample defect detection module and the adversarial sample defense module.

3. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 2, characterized in that, The process of using a multi-scale attention network to identify the acquired product image data, and then using a transformer convolutional neural network architecture and dynamic receptive field technology to locate product defects and obtain initial visual defect information includes: The acquired product image data is standardized, denoised, and normalized in size to obtain the processed product image data. The processed product image data is input into the converter convolutional neural network architecture, and the convolutional neural backbone network is used to extract product image features at different resolutions in parallel to generate multi-scale feature sequences. The multi-scale feature sequence is input into the transformer encoder, the feature weight distribution is dynamically calculated through the self-attention mechanism, and the feature weight distribution is searched using a search algorithm to locate the key areas of potential defects in the product image. Based on the key regions of potential defects, dynamic receptive field technology is used to adaptively adjust the perception range of the convolution kernel according to the product material characteristics, thereby obtaining optimized product image features with multi-granularity defect features. By using a detection head network to perform bounding box regression and classification on optimized product image features, the coordinates, category, and confidence level of product defects are obtained. The coordinates, categories, and confidence information of all defects are merged and analyzed to generate structured initial visual defect information containing defect type, location, and confidence level.

4. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 3, characterized in that, The process of inputting multi-scale feature sequences into the transformer encoder, dynamically calculating the feature weight distribution through a self-attention mechanism, and using a search algorithm to perform region search on the feature weight distribution to locate potential defect key regions in the product image includes: Initialize the parameters of the search algorithm and set the maximum number of iterations; The multi-scale feature sequence is input into the transformer encoder, an initial response value is assigned to each feature region, and the global feature weight probability distribution is dynamically calculated through a self-attention mechanism. Based on the calculated feature weight distribution, the influence of all feature regions is evaluated using an evaluation algorithm, and the response sequence of key regions to be processed is generated in descending order of influence. Select the feature region with the highest current weight from the key region response sequence, and sample a set of semantically adjacent feature blocks from the neighborhood of the feature region with the highest weight based on feature similarity; The weighted response of the current region is optimized based on the sampled feature blocks, and the weighted feature center of the sampled feature blocks is calculated. The feature block that best matches the center is identified as the defect focal region and marked. Repeat the neighborhood feature sampling and weight response optimization process. If the maximum number of iterations is reached, output the key areas of potential defects in the located product image.

5. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 4, characterized in that, The process of evaluating the influence of all feature regions based on the calculated feature weight distribution, and generating a key region response sequence to be processed in descending order of influence, includes: Initialize the parameters of the evaluation algorithm and set the maximum number of iterations; An evaluation queue is initialized in the feature weight distribution using a spatial uniform sampling method to ensure full coverage of the feature space; Calculate the initial influence of all feature regions in the evaluation queue, sort them, and identify the feature region with the highest current global influence. In the coarse search phase, an alternating selection algorithm is used to perform a global and local search on the evaluation queue to discover the region with the highest influence. During the detailed search phase, the evaluation queue assesses the highest-impact areas discovered, using finer analysis to improve the accuracy of regional impact assessments. The process of repeating influence identification, coarse search, and fine search, if the maximum number of iterations is reached, outputs a sequence of key region responses to be processed in descending order of influence.

6. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 5, characterized in that, In the coarse search phase, an alternating selection algorithm is used to perform global and local searches on the evaluation queue to discover the regions with the highest influence, including: Initialize the parameters of the alternating selection algorithm, set the maximum number of iterations, and randomly select an initial region from the evaluation queue as the starting point of the current search; Search the evaluation queue for candidate regions that can be combined with the current region to enhance the region's influence, and form new region combinations; Verify whether the new region combination meets the preset conditions; if not, select the next best candidate in turn for verification until the optimal region combination that meets the preset conditions is found. Repeat the region combination search and verification process to expand the region combination until the influence of the region combination reaches a local optimum; A random perturbation is introduced, and a random region is received based on the current optimal region combination. After the characteristics of the random region are changed, a region combination search process is carried out to search for new potential optimal region combinations. Compare the influence of the new and old optimal region combinations, retain the optimal region combination with the highest influence, and if the maximum number of iterations is reached, output the optimal region combination with the highest global influence as the region with the highest influence.

7. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 1, characterized in that, The wide-area knowledge sharing system units include: The industry knowledge base construction module is used to acquire cross-industry product defect samples, build a cross-industry industrial vision database, and transfer the capabilities of the vision transformer model to a lightweight network through knowledge distillation technology to build a lightweight detection model. The cross-domain transfer learning module is used to perform fusion analysis on product visual defect information, equipment operation information and cross-industry industrial vision databases using domain adaptive transfer technology, and to match and recommend the fusion analysis results through feature topology maps to generate optimized detection solutions. The industry knowledge base construction module and the cross-domain transfer learning module are connected.

8. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 1, characterized in that, The multimodal collaborative optimization unit includes: The visual sensing data fusion module is used to fuse and correlate product visual defect information and equipment operation information through graph neural networks based on a lightweight detection model and optimized detection scheme to locate the root cause of process abnormalities that lead to defects. The dynamic process control module is used to dynamically adjust process parameters in a digital twin environment based on the root causes of process anomalies, and to verify the effects using a virtual trial and error engine, ultimately generating an executable optimization strategy. The visual sensing data fusion module and the dynamic process control module are connected.

9. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 8, characterized in that, The process of dynamically adjusting process parameters in a digital twin environment based on the root causes of process anomalies, verifying the effects using a virtual trial-and-error engine, and ultimately generating an executable optimization strategy includes: Initialize the parameters of the control algorithm and set the maximum number of iterations; Initialize the digital twin environment, define the process parameter space, and simultaneously initialize the optimization strategy exploration path, starting from the current abnormal state and the target state; Candidate process parameters are randomly generated, and a dynamic step size strategy is used to adjust the process parameters to improve search efficiency and convergence speed. New process parameters are determined by adaptively calculating multidimensional optimization gradients, and the process parameter optimization path is reconstructed accordingly to achieve better convergence. The feasibility of new process parameters is verified in a virtual trial-and-error engine. If the simulation fails, it is iterated again; if it succeeds, the optimization effect is evaluated. Merge the validated effective process parameters, and if the maximum number of iterations is reached, output an executable optimization strategy.

10. The industrial IoT intelligent sensing and collaborative optimization platform according to claim 9, characterized in that, The formula for adjusting process parameters using the dynamic step size strategy is as follows: ； In the formula, W new This indicates the adjusted process parameters; W current Indicates the current process parameters; S 0 represents the initial step size; λ Indicates the attenuation coefficient; ω Indicates historical guidance weight; Δ W hist This indicates the deviation from the historical best process parameters; Δ W grad Indicates the direction of the performance gradient; t Indicates the current iteration number; T max Indicates the maximum number of iterations; e It represents the base of the natural logarithm.

Images