Real-time monitoring method and related equipment for intelligent manufacturing digital twin system

Abstract

The present invention relates to a real-time monitoring method and related equipment of an intelligent manufacturing digital twin system, comprising the following steps: constructing a three-dimensional digital twin model based on the real-time image sequence; analyzing whether there is any abnormal phenomenon at the production site based on the three-dimensional digital twin model, and if so, identifying abnormal areas in the three-dimensional digital twin model through a preset abnormal recognition algorithm to obtain abnormal state data; performing abnormal deduction on the production site based on the abnormal state data to obtain an abnormal evolution trend; performing risk assessment on the production site based on the abnormal evolution trend to obtain a risk assessment result, and selecting a corresponding risk control strategy based on the risk assessment result, thereby solving the technical problem that traditional methods are difficult to provide sufficient details to fully understand the causes behind the abnormal phenomena, thereby affecting subsequent risk assessment and the selection of control strategies.

Description

Technical Field

[0001] The present invention relates to the technical field of digital twin systems, and in particular to a real-time monitoring method and related equipment for an intelligent manufacturing digital twin system. Background Art

[0002] With the development of intelligent manufacturing technology, the manufacturing industry is undergoing a critical stage of transformation from traditional production mode to intelligent and digital mode. In this process, digital twin technology, as a bridge connecting the physical world and the virtual world, provides unprecedented possibilities for production management of manufacturing enterprises. By accurately simulating real-life production equipment and processes in a virtual environment, enterprises can achieve real-time monitoring, predictive maintenance and optimized decision-making of production. However, in practical applications, building a digital twin system that can accurately reflect the complex and changing production environment and respond to various emergencies in real time still faces many challenges. On the one hand, how to efficiently collect and process massive data from the production site, especially image data, to ensure the accuracy and timeliness of the three-dimensional model is a technical problem that needs to be solved urgently; on the other hand, when abnormal situations occur at the production site, the recognition and response speed of the existing system often cannot meet the rapidly changing production needs.

[0003] Existing monitoring methods mainly rely on traditional sensor networks and preset rules to detect problems on the production line, but this method has limitations, especially in the face of unexpected failures or complex scenarios, its flexibility and adaptability are poor. In addition, traditional methods are difficult to provide enough details to fully understand the reasons behind the abnormal phenomena, which affects the subsequent risk assessment and the selection of control strategies. In order to overcome these shortcomings, it is necessary to develop a new real-time monitoring method that can not only realize the intuitive presentation of the production process based on a high-precision three-dimensional digital twin model, but also automatically analyze and predict potential problems through intelligent algorithms, and take preventive measures in advance. Therefore, the research aims to combine advanced image processing technology and machine learning algorithms to create a more intelligent and efficient monitoring system to improve production efficiency and product quality while reducing operating costs and risks.

[0004] In summary, this study focuses on proposing a real-time monitoring method for an intelligent manufacturing digital twin system, and proposes a comprehensive solution to the problems existing in the above background. By collecting images of the production site and building a three-dimensional digital twin model, it can not only reflect the state of the production environment more realistically, but also use the anomaly recognition algorithm to quickly locate the problem and obtain abnormal state data in a timely manner. Furthermore, by using the dynamic time warping algorithm to deduce the abnormal evolution trend, it is possible to effectively predict possible risks and formulate scientific and reasonable risk control strategies based on this. The application of this method will help improve the intelligence level of manufacturing enterprises, enhance their ability to cope with complex production environments, and provide strong technical support for the realization of intelligent manufacturing. Summary of the invention

[0005] The main purpose of the present invention is to provide a real-time monitoring method and related equipment for an intelligent manufacturing digital twin system, which solves the technical problem that traditional methods are difficult to provide sufficient details to fully understand the causes behind abnormal phenomena, thereby affecting subsequent risk assessment and control strategy selection.

[0006] To achieve the above object, the present invention provides a real-time monitoring method for an intelligent manufacturing digital twin system, comprising the following steps:

[0007] Capturing images of the production site to obtain a real-time image sequence, and constructing a three-dimensional digital twin model based on the real-time image sequence;

[0008] Analyze whether there are any abnormal phenomena in the production site based on the three-dimensional digital twin model. If so, identify abnormal areas in the three-dimensional digital twin model using a preset abnormality recognition algorithm to obtain abnormal state data;

[0009] By using a preset dynamic time warping algorithm, the production site is subjected to abnormal deduction based on the abnormal state data to obtain an abnormal evolution trend;

[0010] Based on the abnormal evolution trend, a risk assessment is performed on the production site to obtain a risk assessment result, and a corresponding risk control strategy is selected based on the risk assessment result.

[0011] Furthermore, the image acquisition of the production site to obtain a real-time image sequence and the construction of a three-dimensional digital twin model based on the real-time image sequence includes:

[0012] Capturing multi-angle images of the production site by a multi-angle stereo vision algorithm to obtain a multi-angle real-time image sequence, and performing feature matching on the multi-angle real-time image sequence to obtain a matching feature point set;

[0013] Performing triangulation on the matching feature point set to obtain dense point cloud data of the production site, and performing surface reconstruction based on the dense point cloud data to obtain a preliminary three-dimensional model;

[0014] Mapping the multi-view images in the preliminary three-dimensional model by a texture mapping algorithm to obtain a texture enhanced three-dimensional model;

[0015] Performing spatial alignment on the texture enhanced three-dimensional model to obtain a globally consistent three-dimensional model, and performing illumination correction on the globally consistent three-dimensional model to obtain a three-dimensional model with uniform illumination;

[0016] The three-dimensional model with uniform illumination is semantically segmented by a deep learning algorithm to obtain and update a three-dimensional digital twin model with semantic information in real time.

[0017] Furthermore, by using a preset abnormality recognition algorithm, abnormal areas are identified in the three-dimensional digital twin model to obtain abnormal state data, including:

[0018] Inputting the three-dimensional digital twin model into a preset anomaly recognition algorithm; wherein the anomaly recognition algorithm includes a focusing module, a DySample module, an InceptionNext module and a YOLOv5s module, and the focusing module includes a spatial pyramid pooling structure and an ELAN attention mechanism;

[0019] Inputting the three-dimensional digital twin model into the spatial pyramid pooling structure in the focusing module to perform multi-scale feature extraction to obtain a multi-scale feature map;

[0020] Inputting the multi-scale feature map into the ELAN attention mechanism for feature focusing to obtain an enhanced focused feature map;

[0021] Inputting the enhanced focus feature map into the DySample module for lightweight dynamic upsampling to obtain an upsampled feature map;

[0022] Input the upsampled feature map into the InceptionNext module for multi-scale feature fusion to obtain a fused feature map;

[0023] The fused feature map is input into the YOLOv5s module to identify and analyze abnormal regions to obtain abnormal state data; wherein the abnormal state data includes the abnormal region location, abnormal degree and abnormal type.

[0024] Furthermore, the abnormal phenomena at the production site are deduced based on the abnormal state data by using a preset dynamic time warping algorithm to obtain an abnormal evolution trend, including:

[0025] The abnormal phenomena at the production site are analyzed for spatiotemporal evolution using a preset dynamic time warping algorithm to obtain a state transition sequence; wherein the state transition sequence includes an abnormal evolution path, a state jump point, and a steady-state interval;

[0026] Performing causal correlation analysis on the state transition sequence to obtain an abnormal propagation chain, and performing topological structure analysis on the abnormal propagation chain to obtain propagation network characteristics; wherein the propagation network characteristics include key node distribution, propagation path strength and network connectivity;

[0027] Performing dynamic evolution analysis on the propagation network characteristics through a preset fractal theory to obtain a fractal feature set, and calculating the abnormal diffusion rate based on the fractal feature set;

[0028] Based on the Koopman operator technology, the propagation network characteristics are dynamically modeled based on the abnormal diffusion rate to obtain an evolutionary dynamics equation, and the abnormal phenomenon at the production site is predicted based on the evolutionary dynamics equation to obtain predicted evolutionary situation parameters, wherein the predicted evolutionary situation parameters include abnormal expansion direction, evolutionary speed and impact radius;

[0029] The predicted evolution situation parameters are subjected to dimensionality reduction processing to obtain main evolution characteristics, and through fuzzy comprehensive evaluation, abnormal trends of abnormal phenomena at the production site are deduced based on the main evolution characteristics to obtain abnormal evolution trends.

[0030] Furthermore, based on the main evolution characteristics, abnormal trends of abnormal phenomena at the production site are deduced to obtain abnormal evolution trends, including:

[0031] Classify and analyze the main evolution characteristics to obtain classification results; wherein the classification results include equipment failure, process deviation, material defect, and environmental abnormality;

[0032] Performing association rule mining on abnormal phenomena based on the classification results to obtain an association rule set;

[0033] Fuzzy logic reasoning is performed on the association rule set to obtain an abnormal trend prediction result, and the abnormal evolution trend is obtained based on the abnormal trend prediction result.

[0034] Further, the risk assessment of the production site is performed based on the abnormal evolution trend to obtain a risk assessment result, and a corresponding risk control strategy is selected based on the risk assessment result, including:

[0035] Quantitatively evaluate the abnormal evolution trend of the production site to obtain abnormal risk quantification results, and predict the risk of the production site based on the abnormal risk quantification results to obtain risk warning data; wherein the risk warning data includes abnormal risk level, abnormal risk duration and risk loss assessment;

[0036] Based on the risk warning data, a multi-dimensional risk assessment is performed on the production site to obtain a risk assessment result; wherein the risk assessment result includes the possibility of occurrence in a designated area of ​​the production site, the degree of impact of abnormal risks on production, and the potential diffusion path of abnormal risks;

[0037] Based on the risk assessment results, a benefit-cost analysis is performed on the production site to obtain a risk control solution set; wherein the risk control solution set includes a production line adjustment strategy, a resource allocation strategy, and a human resource deployment strategy;

[0038] Based on grey correlation analysis, a risk control solution set is evaluated to obtain a solution evaluation result, and based on an ant colony algorithm, the solution evaluation result is optimized and selected to obtain a risk control strategy.

[0039] Furthermore, the benefit-cost analysis of the production site is performed based on the risk assessment result to obtain a risk control solution set, including:

[0040] Based on the variational inference method, a probability distribution analysis is performed on the risk assessment result to obtain a risk probability density function, and the risk probability density function is divided into risk levels based on kernel density estimation to obtain a risk grading result;

[0041] The risk classification results are quantitatively evaluated through multi-criteria decision making to obtain a risk cost matrix, and the risk cost matrix is ​​subjected to benefit analysis to obtain a benefit evaluation result; wherein the benefit evaluation result includes equipment loss cost, production delay loss and maintenance expenditure;

[0042] Based on the benefit evaluation results, the resource allocation of the production site is optimized to obtain a resource allocation plan, and the resource allocation plan is optimized through the preset constraint planning rules to obtain an optimized scheduling strategy; wherein the optimized scheduling strategy includes a personnel deployment plan, an equipment maintenance plan and a material allocation strategy;

[0043] Generate alternatives for the optimization scheduling strategy through a combinatorial optimization algorithm to obtain an alternative solution set, and perform multi-objective trade-offs on the alternative solution set based on Pareto frontier technology to obtain a feasible solution set; wherein the feasible solution set includes risk control effect, resource utilization efficiency and economic cost benefit;

[0044] Based on game theory, the feasible solution set is compared and analyzed to obtain a solution evaluation result, and the solution evaluation result is sorted based on the grey correlation method to obtain a solution priority sequence; wherein the solution priority sequence includes solution feasibility, solution economy and solution urgency;

[0045] Based on a dynamic programming algorithm, risk control is performed on the production site based on the scheme priority sequence to obtain a risk control scheme set.

[0046] The present invention also provides a real-time monitoring device for an intelligent manufacturing digital twin system, comprising:

[0047] An acquisition module is used to acquire images of the production site, obtain a real-time image sequence, and construct a three-dimensional digital twin model based on the real-time image sequence;

[0048] An identification module is used to analyze whether there are abnormal phenomena in the production site based on the three-dimensional digital twin model. If there are abnormal phenomena, an abnormal area is identified in the three-dimensional digital twin model through a preset abnormality identification algorithm to obtain abnormal state data;

[0049] A deduction module, used to perform abnormal deduction on the production site based on the abnormal state data through a preset dynamic time warping algorithm to obtain an abnormal evolution trend;

[0050] An assessment module is used to perform risk assessment on the production site based on the abnormal evolution trend, obtain a risk assessment result, and select a corresponding risk control strategy based on the risk assessment result.

[0051] The present invention also provides a computer device, comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of any one of the above methods when executing the computer program.

[0052] The present invention also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned methods are implemented.

[0053] The real-time monitoring method of the intelligent manufacturing digital twin system provided by the present invention comprises the following steps: performing image acquisition on the production site to obtain a real-time image sequence, and constructing a three-dimensional digital twin model based on the real-time image sequence; analyzing whether there is an abnormal phenomenon on the production site based on the three-dimensional digital twin model, and if there is, performing abnormal area identification in the three-dimensional digital twin model through a preset abnormal identification algorithm to obtain abnormal state data; performing abnormal deduction on the production site based on the abnormal state data through a preset dynamic time warping algorithm to obtain an abnormal evolution trend; performing risk assessment on the production site based on the abnormal evolution trend to obtain a risk assessment result, and selecting a corresponding risk control strategy based on the risk assessment result, through the above technical means, solves the technical problem that the traditional method is difficult to provide enough details to fully understand the reasons behind the abnormal phenomenon, thereby affecting the subsequent risk assessment and the selection of control strategies, and realizes the abnormal phenomenon analysis and identification algorithm based on the three-dimensional digital twin model, so that the system can detect abnormal conditions on the production line at an early stage. By accurately locating the abnormal area and obtaining abnormal state data, this method not only improves the accuracy of abnormal detection, but also shortens the response time, thereby reducing the beneficial effect of downtime caused by failure or erroneous operation. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 It is a schematic diagram of the steps of a real-time monitoring method of an intelligent manufacturing digital twin system in one embodiment of the present invention;

[0055] Figure 2 It is a structural block diagram of a real-time monitoring device of an intelligent manufacturing digital twin system in one embodiment of the present invention;

[0056] Figure 3 It is a schematic block diagram of the structure of a computer device according to an embodiment of the present invention.

[0057] The realization of the purpose, functional features and advantages of the present invention will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0058] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention.

[0059] like Figure 1 As shown, Figure 1 It is a schematic diagram of the steps of a real-time monitoring method of an intelligent manufacturing digital twin system in one embodiment of the present invention;

[0060] In one embodiment of the present invention, a real-time monitoring method for an intelligent manufacturing digital twin system is provided, comprising the following steps:

[0061] Step S1, collect images of the production site to obtain a real-time image sequence, and build a three-dimensional digital twin model based on the real-time image sequence.

[0062] Specifically, in the step of "capturing images of the production site, obtaining a real-time image sequence, and building a three-dimensional digital twin model based on the real-time image sequence" mentioned above, it is first necessary to deploy an efficient image acquisition system. The system consists of a series of high-resolution cameras that are strategically placed throughout the production line to ensure that detailed views of key areas of the production line can be captured. Each frame of the image captured by the camera is a data snapshot at a point in time. As the production line operates, these snapshots are connected to form a continuous image sequence, the so-called "real-time image sequence". For example, in an automobile manufacturing workshop, cameras can be installed at different locations on the body assembly line to continuously capture the vehicle assembly process from multiple angles. Next, in order to build a three-dimensional digital twin model, these real-time image sequences must be deeply processed. Through computer vision algorithms and deep learning technology, we can extract feature information from the image sequence, such as the shape, size, position of objects, and their relative relationship. Subsequently, using this feature information combined with the pre-established three-dimensional model library of the production line, the system will gradually reconstruct a virtual environment that is highly consistent with the production line in the physical world. In this process, the algorithm not only considers the position of static objects, but also tracks dynamically changing objects, such as moving robot arms or parts being assembled, to ensure that the 3D model accurately reflects the real-time status of the production site. For example, in the above-mentioned automobile manufacturing scenario, when a car body moves along the production line and passes through different workstations, the system updates its position and status in the 3D digital twin model based on the real-time image sequence. If a workstation is performing a welding operation, the actions of the welding robot will also be displayed accordingly in the 3D model, including how it picks up the welding gun, executes the welding path, and the reset process after completing the task. Such real-time mapping enables managers to understand the current status of the production line by viewing the 3D digital twin model even if they are not on site, and then make timely and effective decisions. In addition, this real-time monitoring method lays a solid foundation for a series of subsequent intelligent management measures such as anomaly detection and risk assessment, because it provides the most realistic data support.

[0063] Step S2: analyzing whether there are any abnormal phenomena in the production site based on the three-dimensional digital twin model. If so, abnormal areas are identified in the three-dimensional digital twin model through a preset abnormality identification algorithm to obtain abnormal status data.

[0064] Specifically, after the image acquisition of the production site is realized and the three-dimensional digital twin model is constructed, the next key step is to analyze whether there are abnormal phenomena in the production site based on the three-dimensional digital twin model. This process involves comparing and analyzing the real-time acquired data with the preset standard or historical data. For example, in an automobile manufacturing workshop, once the three-dimensional digital twin model is established, the system begins to continuously receive the latest information from the production line and map it into the virtual environment. At this time, through the built-in intelligent algorithm, the system can automatically detect and evaluate whether these new data conform to the normal operation mode. If any deviation from the expected behavior or state is found during the analysis process, that is, there is an abnormal phenomenon, the system will immediately start the preset abnormality recognition algorithm. This algorithm is designed to accurately locate the area where the abnormality occurs in the three-dimensional digital twin model, and can describe the specific characteristics of the abnormality in detail, so as to obtain abnormal state data. In the application scenario of automobile manufacturing, suppose that the moving path of the welding robot suddenly deviates and no longer follows the established welding trajectory. Such changes will be captured by the three-dimensional digital twin model. Subsequently, the abnormality recognition algorithm will conduct an in-depth analysis of this situation, confirm the exact location, degree and possible impact range of the deviation, and record the relevant parameters as abnormal state data. Furthermore, in order to ensure the accuracy of anomaly identification, algorithms usually combine a variety of technical means, such as classifiers and deep neural networks in machine learning, to enhance the ability to understand complex situations. These advanced computing methods can help the system distinguish between normal small fluctuations and real abnormal events, reducing the false alarm rate. For example, when a welding robot occasionally shakes slightly due to external interference, the system needs to be intelligent enough to judge that this is just a temporary non-critical problem, not a serious fault. Only when an abnormality is indeed detected will the subsequent processing mechanism be triggered, such as issuing an alarm to notify maintenance personnel to check the equipment, or adjusting relevant settings on the production line to avoid potential risks. In this way, not only the speed and accuracy of anomaly detection are improved, but also reliable data support is provided for subsequent risk assessment and control strategy selection. In this way, the operating efficiency and safety of the entire intelligent manufacturing system have been significantly improved, and a solid technical foundation has been laid for the intelligent transformation of enterprises.

[0065] Step S3, using a preset dynamic time warping algorithm, perform abnormal deduction on the production site based on the abnormal state data to obtain the abnormal evolution trend.

[0066] Specifically, after the abnormal phenomenon is identified and the abnormal state data is obtained, the next key step is to perform abnormal deduction on the production site based on the abnormal state data through a preset dynamic time warping algorithm to obtain the abnormal evolution trend. This process aims to use previously collected data and models to predict the development path and possible impact of abnormal conditions in the future. Specifically, the dynamic time warping algorithm is a technology that can process nonlinear time series data, which allows the system to compare and analyze abnormal state data at different time points, even if there are time inconsistencies or delays between these data. For example, in an automobile manufacturing workshop, when the moving path of a welding robot deviates, the system will not only record the state data at this abnormal moment, but also continue to monitor subsequent operations. By applying the dynamic time warping algorithm, the system can compare the current abnormal state with similar historical events, while taking into account the time-varying characteristics unique to the production line, such as periodic operation rhythm, equipment response speed and other factors. This enables the system to simulate the trend of how abnormal conditions evolve over time in a three-dimensional digital twin model. For example, if the welding robot continues to operate along the deviated path, it may lead to a decline in welding quality, which in turn affects the strength of the entire body structure. This prediction is not limited to a single event, but can also be extended to the combined impact of multiple related events, such as a series of chain reactions caused by the failure of a key component. In order to achieve more accurate anomaly deduction, dynamic time warping algorithms are usually combined with other advanced data analysis techniques, such as pattern recognition, statistical methods, and machine learning models. These technologies work together to help the system better understand the complex behavior patterns under abnormal conditions and make reasonable estimates of future development trends. In the above-mentioned automotive manufacturing application scenario, assuming that the system detects the deviation of the welding robot path, through the dynamic time warping algorithm, it can predict that if this deviation is not corrected in time, it may cause more serious quality problems in the next few minutes, and may even cause the production line to stop. In addition, the algorithm can also evaluate the impact of abnormal conditions on upstream and downstream processes, such as whether it will affect the progress of the next assembly link, or which parameters need to be adjusted to mitigate potential risks. In summary, the process of performing anomaly deduction on the production site through the preset dynamic time warping algorithm not only provides management with a clear view of the abnormal evolution trend, but also enables them to take necessary preventive measures in advance to reduce losses and optimize production processes. The application of this method greatly enhances the predictability and responsiveness of the intelligent manufacturing system, ensures the safe and stable operation of the production process, and also provides strong support for enterprises to cope with the challenges of complex production environments.

[0067] Step S4: performing risk assessment on the production site based on the abnormal evolution trend, obtaining a risk assessment result, and selecting a corresponding risk control strategy based on the risk assessment result.

[0068] Specifically, after obtaining the abnormal evolution trend through the preset dynamic time warping algorithm, the next key step is to conduct a risk assessment on the production site based on the abnormal evolution trend, obtain the risk assessment result, and select the corresponding risk control strategy based on the risk assessment result. This process aims to quantify the severity and possibility of potential risks by using the predicted abnormal development path, so as to provide a scientific basis for taking appropriate countermeasures. Specifically, in the automobile manufacturing workshop, when the path deviation of the welding robot is detected and the possible quality problems or production line shutdown are predicted, the system will conduct a detailed risk assessment based on these abnormal evolution trends. First, the system will analyze the impact of the abnormal condition on the current process. For example, the decline in welding quality may directly lead to insufficient strength of the body structure, which not only affects the product quality, but also may bring safety hazards. At the same time, the system will also consider the chain reaction of the abnormal situation to the entire production process, such as whether it will affect the progress of the subsequent assembly link, or whether the parameters of other equipment need to be adjusted to mitigate potential risks. By comprehensively evaluating various factors, the system can obtain a comprehensive risk assessment result, clearly indicating which aspects are threatened and the extent of the threat. Once the risk assessment is completed, the system will select the corresponding risk control strategy based on the results. For the example in the above automobile manufacturing scenario, if the risk assessment results show that the path deviation of the welding robot may cause major quality problems, the system may recommend stopping the relevant operations immediately and sending technicians to inspect and calibrate the robot to prevent the problem from getting worse. In addition, the system can also propose preventive maintenance plans, such as regular inspection and maintenance of key equipment to ensure that they are always in optimal working condition. For those situations where the risk exists but is low, the system can recommend some milder measures, such as adjusting the production speed, increasing the frequency of quality inspection, etc., to balance the relationship between production efficiency and risk management. In order to ensure the accuracy and reliability of risk assessment, the system usually integrates a variety of data analysis tools and technologies, including but not limited to statistical models, machine learning algorithms, and the knowledge base of expert systems. These technologies work together to help the system more accurately determine the actual risks brought about by abnormal evolution trends, and formulate the most reasonable control strategy accordingly. For example, suppose that during a risk assessment process, the system finds that although the welding robot path has a slight deviation, it will not have a serious impact on product quality in the short term. At this time, it may recommend continuing monitoring without taking radical intervention measures, but if the deviation continues to deteriorate, it should be adjusted in time. This flexible and intelligent risk assessment and control mechanism not only improves the scientific level of production management, but also enhances the ability of enterprises to cope with complex and changing production environments, providing strong technical support for the realization of intelligent manufacturing.

[0069] In a specific embodiment, the process of collecting images of the production site to obtain a real-time image sequence and constructing a three-dimensional digital twin model based on the real-time image sequence includes:

[0070] Capturing multi-angle images of the production site by a multi-angle stereo vision algorithm to obtain a multi-angle real-time image sequence, and performing feature matching on the multi-angle real-time image sequence to obtain a matching feature point set;

[0071] Performing triangulation on the matching feature point set to obtain dense point cloud data of the production site, and performing surface reconstruction based on the dense point cloud data to obtain a preliminary three-dimensional model;

[0072] Mapping the multi-view images in the preliminary three-dimensional model by a texture mapping algorithm to obtain a texture enhanced three-dimensional model;

[0073] Performing spatial alignment on the texture enhanced three-dimensional model to obtain a globally consistent three-dimensional model, and performing illumination correction on the globally consistent three-dimensional model to obtain a three-dimensional model with uniform illumination;

[0074] The three-dimensional model with uniform illumination is semantically segmented by a deep learning algorithm to obtain and update a three-dimensional digital twin model with semantic information in real time.

[0075] Specifically, in the intelligent manufacturing system, it is a complex and multi-step process to collect images of the production site and build a three-dimensional digital twin model, which involves the application of multiple advanced technologies and algorithms. Specifically, it is necessary to first collect multi-angle images of the production site through a multi-view stereo vision algorithm to obtain a real-time image sequence of multiple perspectives, and perform feature matching on these real-time image sequences of multiple perspectives to obtain a matching feature point set. In this process, high-resolution cameras deployed at key positions of the production line will capture real-time images of the production line from different angles to form a series of image sequences. For example, in an automobile manufacturing workshop, cameras are installed at different positions of the body assembly line to continuously shoot the vehicle assembly process from multiple angles. These cameras can not only provide rich visual information, but also ensure that the acquired data is more comprehensive and accurate through multi-view stereo vision algorithms. Subsequently, in order to integrate these scattered image information into a coherent whole, the system will triangulate the matching feature point set to obtain dense point cloud data of the production site. This step is to find common feature points by analyzing the geometric relationship between each perspective, and then use the position information of these feature points to calculate the actual coordinates in space, thereby generating dense point cloud data. In the automotive manufacturing scenario, as the car body moves along the production line and passes through different workstations, the system updates its position and status in the 3D digital twin model based on the real-time image sequence. Once the dense point cloud data is obtained, the next step is to reconstruct the surface based on this data to obtain a preliminary 3D model. This process aims to convert discrete point cloud data into a continuous surface structure, making the 3D model closer to the object form in the real world. In order to further enhance the realism and detail expression of the 3D model, the system maps the multi-view images in the preliminary 3D model through a texture mapping algorithm to obtain a texture-enhanced 3D model. Texture mapping is a computer graphics technology that can project the information of a 2D image onto a 3D model, so that the surface of the model presents realistic material and color effects. Continuing with the example of automotive manufacturing, when the welding robot performs a task, its motion trajectory and operation details will be accurately reflected in the 3D digital twin model, including the color, glossiness and other subtle details of the weld. Such texture mapping not only improves the visual quality of the model, but also provides more detailed data support for subsequent monitoring and analysis. However, it is not enough to just have a texture-enhanced 3D model. In order to ensure the global consistency of the model under different viewing angles, the system also needs to spatially align the texture-enhanced 3D model to obtain a globally consistent 3D model. Spatial alignment is to eliminate model misalignment caused by viewing angle differences or device errors, ensuring that all parts can be seamlessly connected. In addition, in order to improve the visual effect of the model, the system will perform lighting correction on the globally consistent 3D model to obtain a 3D model with uniform lighting.Light correction solves the brightness and shadow problems caused by uneven distribution or occlusion of light sources, making the model look more natural and smooth. For example, in an automobile manufacturing workshop, even if the lighting conditions change, the three-dimensional digital twin model after light correction can still clearly and accurately display the status of the production line without distortion due to light. Finally, in order to give the three-dimensional model more practical value, the system will perform semantic segmentation on the three-dimensional model with uniform lighting through a deep learning algorithm to obtain a three-dimensional digital twin model with semantic information. Semantic segmentation is an image processing technology that can classify each pixel in an image according to the content and mark the functional attributes of different objects or areas. In automobile manufacturing applications, this means that the three-dimensional model can not only display the physical shape, but also distinguish which parts are welding areas, spraying areas, or assembly areas. This three-dimensional digital twin model with semantic information greatly enriches the system's cognitive ability, enabling it to understand the meaning of each element in the production environment, thereby achieving more intelligent management and decision support. At the same time, in order to maintain the consistency of the model with the actual production environment, the system will continuously update the three-dimensional digital twin model in real time to ensure that it is always up to date and reflects the latest changes in the production site. In summary, through a series of complex processing steps such as multi-view image acquisition, feature matching, triangulation, surface reconstruction, texture mapping, spatial alignment, illumination correction, and semantic segmentation of the production site, the three-dimensional digital twin model finally constructed not only has a high degree of visualization characteristics, but also contains rich semantic information, providing strong technical support for intelligent manufacturing. The application of this model not only improves the transparency and efficiency of production, but also enhances the ability of enterprises to cope with complex production environments, laying a solid foundation for the realization of intelligent manufacturing.

[0076] In a specific embodiment, abnormal area identification is performed in the three-dimensional digital twin model through a preset abnormality identification algorithm to obtain abnormal state data, including:

[0077] Inputting the three-dimensional digital twin model into a preset anomaly recognition algorithm; wherein the anomaly recognition algorithm includes a focusing module, a DySample module, an InceptionNext module and a YOLOv5s module, and the focusing module includes a spatial pyramid pooling structure and an ELAN attention mechanism;

[0078] Inputting the three-dimensional digital twin model into the spatial pyramid pooling structure in the focusing module to perform multi-scale feature extraction to obtain a multi-scale feature map;

[0079] Inputting the multi-scale feature map into the ELAN attention mechanism for feature focusing to obtain an enhanced focused feature map;

[0080] Inputting the enhanced focus feature map into the DySample module for lightweight dynamic upsampling to obtain an upsampled feature map;

[0081] Input the upsampled feature map into the InceptionNext module for multi-scale feature fusion to obtain a fused feature map;

[0082] The fused feature map is input into the YOLOv5s module to identify and analyze abnormal regions to obtain abnormal state data; wherein the abnormal state data includes the abnormal region location, abnormal degree and abnormal type.

[0083] Specifically, in the intelligent manufacturing system, in order to accurately identify and analyze abnormal conditions at the production site, the claims describe an abnormal area identification method based on a three-dimensional digital twin model. This process involves the application of multiple advanced algorithm modules, including a focusing module, a DySample module, an InceptionNext module, and a YOLOv5s module, wherein the focusing module is further subdivided into a spatial pyramid pooling structure (Spatial PyramidPooling, SPP) and an ELAN attention mechanism. The entire recognition process not only demonstrates complex technical details, but also reflects how to ensure the accuracy and efficiency of anomaly detection through multi-level processing. First, the three-dimensional digital twin model is input into a preset anomaly recognition algorithm. This step is the basis of the entire anomaly recognition process, and it provides a data source for subsequent feature extraction and analysis. Taking an automobile manufacturing workshop as an example, when a welding robot on a production line has a path deviation or an operational error, these abnormal behaviors will be reflected in real time in the three-dimensional digital twin model. At this point, the model is fed into the anomaly recognition algorithm as input, opening up an in-depth exploration of abnormal phenomena. Next, the three-dimensional digital twin model is input into the spatial pyramid pooling structure in the focusing module for multi-scale feature extraction to obtain a multi-scale feature map. The spatial pyramid pooling structure is a method that can capture information at different scales. It generates a series of feature maps with different resolutions by layered sampling of the input image. This multi-scale feature extraction is crucial for dealing with complex production environments because it can capture various changes from global to local. For example, in the automotive manufacturing scenario, the spatial pyramid pooling structure can help identify the slight movement deviation of the welding robot, and even those changes that are not easily perceived can be accurately recorded. Subsequently, the multi-scale feature map is input into the ELAN attention mechanism for feature focusing to obtain an enhanced focus feature map. The role of the ELAN attention mechanism is to highlight the key features that best represent the abnormal state while suppressing irrelevant information. In this way, the system can focus more on the specific location and features of the abnormality. Continuing with the example of automotive manufacturing, if the welding robot's welding gun has unexpected movements, the ELAN attention mechanism will automatically enhance the feature expression of these abnormal movements, making them easier to identify in subsequent analysis. The enhanced focus feature map is then input into the DySample module for lightweight dynamic upsampling to obtain an upsampled feature map. The DySample module is designed to solve the problem of information loss that may occur in the traditional upsampling process by dynamically adjusting the sampling ratio and method to ensure that each pixel receives enough attention. In automotive manufacturing applications, this means that even the slightest abnormal changes will not be ignored, and all important features can be fully preserved, thereby improving the accuracy of abnormality recognition.Next, the upsampled feature map is input into the InceptionNext module for multi-scale feature fusion to obtain a fused feature map. The InceptionNext module uses an improved Inception network structure, which can effectively integrate feature information from different scales without increasing too much computational burden. This step is critical for comprehensively understanding the overall characteristics of the abnormal situation, because it allows the system to examine the problem from multiple angles and make a more comprehensive judgment. For example, when analyzing the abnormal behavior of a welding robot, the InceptionNext module can provide a more complete perspective by fusing features of different scales to help determine the root cause of the abnormality. Finally, the fused feature map is input into the YOLOv5s module for abnormal area identification and analysis to obtain abnormal state data. YOLOv5s is an optimized target detection algorithm that can quickly locate and classify target objects in an image in a short time. In this process, YOLOv5s can not only point out the location of the abnormal area, but also evaluate its degree of abnormality and determine the type of abnormality. In the example of an automobile manufacturing workshop, once an abnormality occurs in a welding robot, the YOLOv5s module can quickly identify specific abnormal parts, such as weld deviation, reduced welding quality, etc., and evaluate their severity, thereby providing a basis for timely corrective measures. In summary, by inputting the three-dimensional digital twin model into the preset abnormality recognition algorithm, the spatial pyramid pooling structure and ELAN attention mechanism of the focusing module are used to extract and focus multi-scale features, and then the DySample module is used for lightweight dynamic upsampling, combined with the InceptionNext module for multi-scale feature fusion, and finally the YOLOv5s module completes the abnormal area identification and analysis. The whole process realizes efficient and accurate detection of abnormal situations in the production site. This method not only improves the speed and accuracy of abnormality recognition, but also provides reliable data support for subsequent risk assessment and control strategy selection, thereby enhancing the intelligence level of the intelligent manufacturing system and its ability to cope with complex production environments.

[0084] In a specific embodiment, the abnormal phenomenon of the production site is deduced based on the abnormal state data by using a preset dynamic time warping algorithm to obtain an abnormal evolution trend, including:

[0085] The abnormal phenomena at the production site are analyzed for spatiotemporal evolution using a preset dynamic time warping algorithm to obtain a state transition sequence; wherein the state transition sequence includes an abnormal evolution path, a state jump point, and a steady-state interval;

[0086] Performing causal correlation analysis on the state transition sequence to obtain an abnormal propagation chain, and performing topological structure analysis on the abnormal propagation chain to obtain propagation network characteristics; wherein the propagation network characteristics include key node distribution, propagation path strength and network connectivity;

[0087] Performing dynamic evolution analysis on the propagation network characteristics through a preset fractal theory to obtain a fractal feature set, and calculating the abnormal diffusion rate based on the fractal feature set;

[0088] Based on the Koopman operator technology, the propagation network characteristics are dynamically modeled based on the abnormal diffusion rate to obtain an evolutionary dynamics equation, and the abnormal phenomenon at the production site is predicted based on the evolutionary dynamics equation to obtain predicted evolutionary situation parameters, wherein the predicted evolutionary situation parameters include abnormal expansion direction, evolutionary speed and impact radius;

[0089] The predicted evolution situation parameters are subjected to dimensionality reduction processing to obtain main evolution characteristics, and through fuzzy comprehensive evaluation, abnormal trends of abnormal phenomena at the production site are deduced based on the main evolution characteristics to obtain abnormal evolution trends.

[0090] Specifically, in the intelligent manufacturing system, the process of obtaining the state transition sequence by performing spatiotemporal evolution analysis on the abnormal phenomena at the production site through the preset dynamic time warping algorithm is a key step to ensure the safe and stable operation of the production process. This process not only relies on advanced data analysis technology, but also combines causal association analysis, topological structure analysis, fractal theory, Koopman operator technology and fuzzy comprehensive evaluation and other methods, aiming to extract valuable prediction information from complex data and provide a scientific basis for the risk management of enterprises. First, by performing spatiotemporal evolution analysis on the abnormal phenomena at the production site, the system uses the dynamic time warping (DTW) algorithm to process the abnormal state data and obtain the state transition sequence. This state transition sequence includes abnormal evolution path, state jump point and steady-state interval, which records in detail the change process of abnormal phenomena in time and space dimensions. For example, in an automobile manufacturing workshop, if a welding robot has a path deviation or an operation error, these abnormal behaviors will leave traces in the three-dimensional digital twin model. Through the DTW algorithm, the system can capture the time trajectory of these abnormal behaviors, identify how the abnormality is transferred from one state to another, and the conditions and frequency of these transfers. This step not only reveals the occurrence and development rules of abnormal phenomena, but also provides a basis for subsequent in-depth analysis. Next, the state transition sequence is subjected to causal association analysis to obtain the abnormal propagation chain, and the abnormal propagation chain is subjected to topological structure analysis to obtain the propagation network characteristics. Causal association analysis aims to find the causal relationship between different events to help understand the occurrence mechanism and development path of abnormal phenomena. Topological structure analysis focuses on building a network model that describes the abnormal propagation mode, which includes characteristics such as key node distribution, propagation path strength, and network connectivity. For example, in the automobile manufacturing scenario, if it is found that the failure of a welding robot causes problems in multiple other workstations, then through causal association and topological structure analysis, a clear abnormal propagation chain can be constructed to show how the failure spreads on the production line and point out which nodes are the most critical influencing factors. This analysis not only reveals the root cause of the abnormal phenomenon, but also provides a basis for formulating effective response strategies. Then, the propagation network characteristics are dynamically evolved through the preset fractal theory to obtain a fractal feature set, and the abnormal diffusion rate is calculated based on the fractal feature set. Fractal theory provides a way to quantify the internal structure and dynamic changes of complex systems. It can help us understand the propagation patterns of abnormal phenomena in the network and their changes over time. By calculating the fractal feature set, the system can evaluate the diffusion rate of abnormal phenomena, that is, how fast the abnormal situation spreads in the production site. In the example of automobile manufacturing, if a problem occurs in a certain equipment, through fractal theory analysis, it can be predicted how long it will take for the problem to affect the entire production line, thereby buying valuable time for taking preventive measures.This method not only improves the accuracy of prediction, but also enhances the response speed of the system. Subsequently, based on the Koopman operator technology, the propagation network characteristics are dynamically modeled based on the abnormal diffusion rate to obtain the evolutionary dynamics equation, and the abnormal phenomena at the production site are predicted based on the evolutionary dynamics equation to obtain the predicted evolutionary state parameters. Koopman operator technology is a powerful mathematical tool that allows us to convert the dynamic characteristics of nonlinear systems into linear forms, which facilitates more accurate predictions. By establishing the evolutionary dynamics equation, the system can simulate the evolution process of abnormal phenomena in the future and predict important parameters such as their expansion direction, evolution speed and impact radius. For example, in an automobile manufacturing workshop, if it is found that the abnormal phenomenon is expanding along a specific path, then using the Koopman operator technology, it is possible to predict in advance how the abnormality will affect different parts of the production line, and formulate corresponding response strategies accordingly. This method not only improves the accuracy of prediction, but also enhances the predictability and initiative of the system. Finally, the predicted evolutionary state parameters are subjected to dimensionality reduction processing to obtain the main evolutionary features, and through fuzzy comprehensive evaluation, the abnormal phenomena at the production site are deduced based on the main evolutionary features to obtain the abnormal evolution trend. Dimensionality reduction is to remove redundant information and highlight the most important evolution features, while fuzzy comprehensive evaluation combines expert knowledge and quantitative analysis to provide a more flexible and comprehensive evaluation method. In the automotive manufacturing application scenario, this means that the system can not only predict the development direction and speed of abnormal phenomena, but also evaluate the degree of its impact on production and quality, thereby providing decision support for management. Through fuzzy comprehensive evaluation, the system can comprehensively consider multiple factors, such as risk level, impact range and urgency, and finally come up with a clear abnormal evolution trend to help enterprises take effective control measures in time. In summary, through a series of complex and sophisticated processing steps such as spatiotemporal evolution analysis, causal correlation analysis, dynamic evolution analysis of fractal theory, dynamic modeling of Koopman operator and fuzzy comprehensive evaluation of abnormal phenomena at the production site, this method achieves a deep understanding of abnormal phenomena at the production site and accurate prediction of future evolution trends. This method not only improves the speed and accuracy of abnormal detection and response, but also provides strong technical support for the risk management and intelligent transformation of enterprises, ensuring the safe and stable operation of the production process. Through this scientific and rigorous analysis process, enterprises can identify and solve potential problems at an early stage, ensure the safe and stable operation of the production process, and also provide strong support for optimizing production management and improving product quality.

[0091] In a specific embodiment, based on the main evolution characteristics, abnormal trends of abnormal phenomena at the production site are deduced to obtain abnormal evolution trends, including:

[0092] Classify and analyze the main evolution characteristics to obtain classification results; wherein the classification results include equipment failure, process deviation, material defect, and environmental abnormality;

[0093] Performing association rule mining on abnormal phenomena based on the classification results to obtain an association rule set;

[0094] Fuzzy logic reasoning is performed on the association rule set to obtain an abnormal trend prediction result, and the abnormal evolution trend is obtained based on the abnormal trend prediction result.

[0095] Specifically, in the intelligent manufacturing system, the abnormal trend deduction of the abnormal phenomena at the production site based on the main evolution characteristics to obtain the abnormal evolution trend is an important link to ensure the safe and efficient operation of the production process. This process not only relies on advanced data analysis technology, but also combines a variety of methods such as classification analysis, association rule mining and fuzzy logic reasoning, aiming to extract valuable prediction information from complex data and provide a scientific basis for the risk management of enterprises. First, the main evolution characteristics are classified and analyzed to obtain classification results. This step is the basis of the entire abnormal trend deduction. It identifies different types of abnormal situations by carefully classifying the main evolution characteristics obtained by the previous processing. Specifically, the classification results include several major categories such as equipment failure, process deviation, material defect and environmental anomaly. For example, in an automobile manufacturing workshop, when the path of the welding robot is offset, this may be classified as equipment failure; if the welding quality is inconsistent, it may be the result of process deviation; and pores or cracks in the weld point to material defects; as for changes in lighting conditions or temperature fluctuations, they belong to environmental anomalies. Through this classification analysis, the nature of abnormal phenomena can be more clearly understood, laying a solid foundation for further in-depth analysis. Next, association rule mining is performed on the abnormal phenomena based on the classification results to obtain an association rule set. Association rule mining is a data mining technique that is used to discover potential connections between different events in a data set. In this process, the system will look for commonalities between abnormal phenomena and their relationships with other factors based on the classification results. For example, in an automobile manufacturing scenario, if a welding robot is found to frequently have path deviations (equipment failures) accompanied by a decrease in welding quality (process deviations), then association rule mining can help determine whether there is a causal relationship between the two. In addition, this step can also reveal other implicit patterns, such as the association between a specific batch of materials and welding quality problems, or the impact of changes in environmental conditions on equipment performance. By building a comprehensive association rule set, the system can better understand the background and causes of abnormal phenomena, thereby providing strong support for subsequent predictions. Then, fuzzy logic reasoning is performed on the association rule set to obtain an abnormal trend prediction result, and based on this result, the abnormal evolution trend is obtained. Fuzzy logic reasoning is a mathematical tool for dealing with uncertainty and ambiguity, which allows the system to make reasonable judgments in the absence of precise data. At this stage, the system will use the previously obtained association rule set, combined with the principles of fuzzy logic, to evaluate the development trend of various abnormal phenomena in the future. For example, in an automotive manufacturing application, if an association rule set indicates that a welding robot’s path deviation (equipment failure) is often accompanied by a decrease in weld quality (process deviation), fuzzy logic reasoning can help predict whether this combination will continue to deteriorate or lead to new problems such as line downtime or reduced product quality.In addition, fuzzy logic reasoning can also consider more variables, such as the operator's experience level, the execution of maintenance plans, etc., to provide a more comprehensive and accurate trend forecast. To achieve this, the system will take into account multiple factors, including but not limited to historical data, current status, and external influences. In this way, not only can the specific evolution direction of abnormal phenomena be predicted, but also the scope and severity of their impact can be evaluated. For example, in an automobile manufacturing workshop, if it is predicted that the path deviation of a welding robot may cause a wider range of quality problems, the system can take preventive measures in advance, such as adjusting equipment parameters, strengthening quality inspections, or arranging emergency repairs to avoid potential risks. Ultimately, these prediction results will be integrated into an overall abnormal evolution trend, providing enterprise management with a clear view of what may happen in the future, helping them make more informed decisions. In summary, by classifying and analyzing the main evolution features, mining association rules for abnormal phenomena based on the classification results, and then performing fuzzy logic reasoning on the resulting association rule set, the abnormal trend deduction of abnormal phenomena on the production site is finally achieved. This method not only improves the speed and accuracy of abnormal detection and response, but also enhances the ability of enterprises to cope with complex and changing production environments, laying a solid technical foundation for the realization of intelligent manufacturing. Through this scientific and rigorous analysis process, enterprises can identify and solve potential problems at an early stage, ensure the safe and stable operation of the production process, and also provide strong support for optimizing production management and improving product quality.

[0096] In a specific embodiment, the risk assessment of the production site is performed based on the abnormal evolution trend to obtain a risk assessment result, and a corresponding risk control strategy is selected based on the risk assessment result, including:

[0097] Quantitatively evaluate the abnormal evolution trend of the production site to obtain abnormal risk quantification results, and predict the risk of the production site based on the abnormal risk quantification results to obtain risk warning data; wherein the risk warning data includes abnormal risk level, abnormal risk duration and risk loss assessment;

[0098] Based on the risk warning data, a multi-dimensional risk assessment is performed on the production site to obtain a risk assessment result; wherein the risk assessment result includes the possibility of occurrence in a designated area of ​​the production site, the degree of impact of abnormal risks on production, and the potential diffusion path of abnormal risks;

[0099] Based on the risk assessment results, a benefit-cost analysis is performed on the production site to obtain a risk control solution set; wherein the risk control solution set includes a production line adjustment strategy, a resource allocation strategy, and a human resource deployment strategy;

[0100] Based on grey correlation analysis, a risk control solution set is evaluated to obtain a solution evaluation result, and based on an ant colony algorithm, the solution evaluation result is optimized and selected to obtain a risk control strategy.

[0101] Specifically, in the intelligent manufacturing system, the process of conducting risk assessment on the production site based on the abnormal evolution trend, obtaining risk assessment results, and selecting corresponding risk control strategies based on the risk assessment results is an important link to ensure the safe and efficient operation of the production process. This process not only relies on advanced data analysis technology, but also combines quantitative assessment, multi-dimensional risk assessment, benefit-cost analysis, and optimization selection, aiming to extract valuable prediction information from complex data and provide a scientific basis for the risk management of enterprises. First, the abnormal evolution trend of the production site is quantitatively evaluated to obtain the abnormal risk quantification result, and the production site is risk predicted based on the abnormal risk quantification result to obtain risk warning data. This step is the basis of the entire risk assessment. It digitizes and quantifies the abnormal evolution trend obtained by the previous processing to determine the degree of risk that abnormal phenomena may bring. For example, in an automobile manufacturing workshop, if it is found through abnormal trend deduction that the path deviation of the welding robot may cause welding quality problems, then the system will calculate specific abnormal risk quantification results according to this evolution trend, such as the probability of failure, the scope of impact, etc. Subsequently, based on these quantitative results, the system will conduct risk prediction and generate risk warning data including abnormal risk level, abnormal risk duration and risk loss assessment. This step helps management to understand the specific situation of potential risks in a timely manner and provides an important reference for subsequent decision-making. Next, a multi-dimensional risk assessment is conducted on the production site based on the risk warning data to obtain risk assessment results. Multi-dimensional risk assessment is a comprehensive evaluation method that takes into account multiple aspects such as the possibility of risk occurrence, the degree of impact on production, and potential diffusion paths. In this process, the system will analyze in detail the possibility of abnormality in each designated area, evaluate the impact of abnormal risk on the production process, and identify the path where abnormal risk may spread. For example, in the automotive manufacturing scenario, if a welding robot has a problem, the system will not only evaluate the risk of the problem itself, but also consider whether it will affect the overall operation of other workstations or production lines. In addition, the system will analyze how abnormal risks spread along the production chain and whether there is a possibility of a chain reaction. Through such a multi-dimensional assessment, the system can fully grasp the risk situation and provide support for the formulation of effective response measures. Then, based on the risk assessment results, a benefit-cost analysis is conducted on the production site to obtain a set of risk control solutions. Benefit-cost analysis is an economic tool that helps companies weigh the pros and cons before taking any action, ensuring that the selected strategy can effectively control risks without causing unnecessary waste of resources. At this stage, the system will propose a series of risk control plans, including production line adjustment strategies, resource allocation strategies, and human resource deployment strategies.For example, in automobile manufacturing applications, if a high-risk problem occurs at a welding station, the system may suggest adjusting the order of the production line, reallocating production equipment, or adding temporary manpower support to solve the problem. The selection of these solutions should not only consider the short-term risk mitigation effect, but also the long-term cost-effectiveness ratio to ensure that the company maximizes profits while minimizing losses. Finally, the risk control solution set is evaluated based on gray correlation analysis to obtain the solution evaluation results, and the solution evaluation results are optimized and selected based on the ant colony algorithm to obtain the risk control strategy. Gray correlation analysis is a mathematical method for dealing with uncertainty and fuzziness, which can help the system make reasonable judgments in the absence of complete information. In this process, the system will comprehensively evaluate all proposed control solutions to find the option that best meets the current needs. The ant colony algorithm is an optimization algorithm that simulates the foraging behavior of ants in nature. It can find the optimal solution among many alternative solutions. For example, in an automobile manufacturing workshop, when multiple risk control solutions are placed in front of you, gray correlation analysis can preliminarily screen out several more suitable solutions, and the ant colony algorithm further optimizes these solutions and finally determines an optimal risk control strategy. This method not only improves the speed and accuracy of decision-making, but also enhances the ability of enterprises to cope with complex and changing production environments. In summary, by quantitatively evaluating the abnormal evolution trend of the production site, predicting risks based on the quantitative results, and then conducting multi-dimensional risk assessment of the production site, and then conducting benefit-cost analysis to obtain a set of risk control solutions, and finally optimizing the risk control strategy through gray correlation analysis and ant colony algorithm, the whole process achieves a deep understanding of abnormal phenomena at the production site and accurate prediction of future evolution trends. This method not only improves the speed and accuracy of anomaly detection and response, but also provides strong technical support for the enterprise's risk management and intelligent transformation, ensuring the safe and stable operation of the production process. Through this scientific and rigorous analysis process, enterprises can identify and solve potential problems at an early stage, ensure the safe and stable operation of the production process, and also provide strong support for optimizing production management and improving product quality.

[0102] In a specific embodiment, the benefit-cost analysis of the production site is performed based on the risk assessment result to obtain a risk control solution set, including:

[0103] Based on the variational inference method, a probability distribution analysis is performed on the risk assessment result to obtain a risk probability density function, and the risk probability density function is divided into risk levels based on kernel density estimation to obtain a risk grading result;

[0104] The risk classification results are quantitatively evaluated through multi-criteria decision making to obtain a risk cost matrix, and the risk cost matrix is ​​subjected to benefit analysis to obtain a benefit evaluation result; wherein the benefit evaluation result includes equipment loss cost, production delay loss and maintenance expenditure;

[0105] Based on the benefit evaluation results, the resource allocation of the production site is optimized to obtain a resource allocation plan, and the resource allocation plan is optimized through the preset constraint planning rules to obtain an optimized scheduling strategy; wherein the optimized scheduling strategy includes a personnel deployment plan, an equipment maintenance plan and a material allocation strategy;

[0106] Generate alternatives for the optimization scheduling strategy through a combinatorial optimization algorithm to obtain an alternative solution set, and perform multi-objective trade-offs on the alternative solution set based on Pareto frontier technology to obtain a feasible solution set; wherein the feasible solution set includes risk control effect, resource utilization efficiency and economic cost benefit;

[0107] Based on game theory, the feasible solution set is compared and analyzed to obtain a solution evaluation result, and the solution evaluation result is sorted based on the grey correlation method to obtain a solution priority sequence; wherein the solution priority sequence includes solution feasibility, solution economy and solution urgency;

[0108] Based on a dynamic programming algorithm, risk control is performed on the production site based on the scheme priority sequence to obtain a risk control scheme set.

[0109] Specifically, in the intelligent manufacturing system, the process of performing benefit-cost analysis on the production site based on the risk assessment results to obtain a set of risk control solutions is a complex and multi-level decision-making process. This process not only relies on advanced mathematical and statistical methods, but also combines multi-criteria decision-making, resource allocation optimization, combinatorial optimization algorithms, and game theory, aiming to extract the most effective risk control strategy from complex multidimensional data. The following will explain in detail how to implement this part described in the claims. First, the risk assessment results are analyzed for probability distribution based on the variational reasoning method to obtain a risk probability density function, and the risk probability density function is divided into risk levels based on kernel density estimation to obtain a risk grading result. Variational reasoning is a method for approximating complex probability models, and here it is used to model the uncertainty in risk assessment results. By performing probability distribution analysis on the data of the risk assessment results, a probability density function reflecting different risk levels can be obtained. For example, in an automobile manufacturing workshop, if a welding robot has a risk of path deviation, the probability distribution of the equipment failure can be calculated by the variational reasoning method. Next, the probability density function is smoothed using Kernel Density Estimation (KDE) to more accurately identify different risk levels. The result of this step is a clear risk grading result, which provides a solid foundation for subsequent cost-benefit analysis. Subsequently, the risk grading results are quantitatively evaluated through multi-criteria decision-making to obtain a risk cost matrix, and the risk cost matrix is ​​analyzed for benefits to obtain benefit evaluation results. Multi-criteria decision-making is a method that can consider multiple evaluation criteria at the same time. It is used here to comprehensively evaluate the cost impact of different types of risks. Specifically, based on the risk grading results, the system constructs a risk cost matrix that includes key indicators such as equipment loss cost, production delay loss, and maintenance expenditure. These cost indicators reflect the economic losses that may be caused by different risk levels. By performing benefit analysis on the risk cost matrix, the system can quantitatively evaluate the actual impact of each risk on production and generate detailed benefit evaluation results. For example, in an automotive manufacturing scenario, if the failure of a welding robot causes the production line to stop, the system will calculate the direct and indirect costs incurred, such as repair costs, production loss during downtime, etc., thereby providing management with clear cost-benefit information. Then, based on the benefit evaluation results, the resource allocation of the production site is optimized to obtain a resource allocation plan, and the resource allocation plan is scheduled and optimized through the preset constraint planning rules to obtain an optimized scheduling strategy. The goal of this stage is to maximize resource utilization efficiency and minimize costs while meeting production and safety requirements.For example, in automotive manufacturing applications, if a production line is found to have a high-risk area, the system will recommend adjusting the staffing plan, adding equipment maintenance plans, or optimizing material allocation strategies. These measures can not only reduce the possibility of risk occurrence, but also improve production efficiency. In addition, by introducing preset constraint planning rules, such as human resource constraints, equipment availability, and inventory levels, the system can further optimize these resource allocation plans to ensure that they are both feasible and effective. Finally, the system generates a set of optimal scheduling strategies, including specific staffing plans, equipment maintenance plans, and material allocation strategies to deal with potential risk challenges. Then, the combinatorial optimization algorithm is used to generate alternative solutions for the optimal scheduling strategy to obtain an alternative solution set, and the alternative solution set is weighed based on the Pareto frontier technology to obtain a feasible solution set. The combinatorial optimization algorithm allows the system to find the optimal or near-optimal choice among many possible solutions. In this process, the system generates a series of alternative solutions, each of which represents a different risk control method. For example, in an automotive manufacturing workshop, there may be multiple ways to reduce the risk of welding robot failure, such as strengthening daily inspections, replacing old parts, or upgrading the control system. In order to select the best solution, the system uses Pareto frontier technology to make multi-objective trade-offs, that is, to find the optimal solution without sacrificing any goal. The result of this step is a set of feasible solutions that include risk control effects, resource utilization efficiency, and economic cost-benefit, which provides enterprises with a variety of choices. Finally, the feasible solution set is compared and analyzed based on game theory to obtain the solution evaluation results, and the solution evaluation results are ranked based on the gray correlation method to obtain the solution priority sequence. Game theory provides a method to simulate the decision-making process in a competitive and cooperative environment, which can help the system understand the interaction between different solutions. Through game theory analysis, the system can evaluate the performance of each solution in the face of other solutions and determine its advantages and disadvantages. For example, in the automotive manufacturing scenario, some solutions may be very effective in the short term, but not as economical as other solutions in the long run. In order to better compare these solutions, the system uses the gray correlation method to rank the solutions, which is a mathematical tool that can handle uncertainty and ambiguity. Finally, the system generates a solution priority sequence, which includes important indicators such as solution feasibility, solution economy, and solution urgency, providing a clear decision-making basis for enterprise management. In summary, through probability distribution analysis based on variational reasoning methods, quantitative evaluation of multi-criteria decision-making, resource allocation optimization, generation of alternative solutions by combinatorial optimization algorithms, multi-objective trade-offs based on Pareto cutting-edge technology, and comparative analysis of solutions using game theory and grey correlation methods, the entire process achieved a deep understanding of abnormal phenomena in the production site and an accurate prediction of future evolution trends.This method not only improves the speed and accuracy of anomaly detection and response, but also provides strong technical support for the company's risk management and intelligent transformation, ensuring the safe and stable operation of the production process. Through this scientific and rigorous analysis process, companies can identify and solve potential problems at an early stage, ensure the safe and stable operation of the production process, and provide strong support for optimizing production management and improving product quality. Finally, based on the dynamic programming algorithm, the production site is risk controlled according to the priority sequence of the solution, and a comprehensive set of risk control solutions is obtained, which provides a solid guarantee for companies to cope with complex and changing production environments.

[0110] The above describes the real-time monitoring method of the intelligent manufacturing digital twin system in the embodiment of the present invention. The following describes the real-time monitoring device of the intelligent manufacturing digital twin system in the embodiment of the present invention. Figure 2 In one embodiment of the present invention, a real-time monitoring device of a smart manufacturing digital twin system includes:

[0111] The acquisition module 21 is used to acquire images of the production site, obtain a real-time image sequence, and construct a three-dimensional digital twin model based on the real-time image sequence;

[0112] An identification module 22 is used to analyze whether there are abnormal phenomena in the production site based on the three-dimensional digital twin model. If there are abnormal phenomena, an abnormal area is identified in the three-dimensional digital twin model through a preset abnormality identification algorithm to obtain abnormal state data;

[0113] A deduction module 23 is used to perform abnormal deduction on the production site based on the abnormal state data by using a preset dynamic time warping algorithm to obtain an abnormal evolution trend;

[0114] The evaluation module 24 is used to perform risk evaluation on the production site based on the abnormal evolution trend, obtain a risk evaluation result, and select a corresponding risk control strategy based on the risk evaluation result.

[0115] In this embodiment, for the specific implementation of each unit in the above system embodiment, please refer to the above method embodiment, which will not be repeated here.

[0116] Reference Figure 3 The present invention also provides a computer device in an embodiment, wherein the internal structure of the computer device can be as follows: Figure 3As shown. The computer device includes a processor, a memory, a display screen, an input device, a network interface and a database connected through a system bus. Among them, the processor designed by the computer is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and the computer program in the non-volatile storage medium. The database of the computer device is used to store the corresponding data in this embodiment. The network interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, the above method is implemented.

[0117] Those skilled in the art will understand that Figure 3 The structure shown in the figure is merely a block diagram of a portion of the structure related to the solution of the present invention, and does not constitute a limitation on the computer device to which the solution of the present invention is applied.

[0118] An embodiment of the present invention further provides a computer-readable storage medium on which a computer program is stored, and when the computer program is executed by a processor, the above method is implemented. It can be understood that the computer-readable storage medium in this embodiment can be a volatile readable storage medium or a non-volatile readable storage medium.

[0119] Those skilled in the art can understand that all or part of the processes in the above-mentioned embodiment methods can be completed by instructing the relevant hardware through a computer program, and the computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above-mentioned methods. Among them, any reference to memory, storage, database or other media provided by the present invention and used in the embodiments may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM) or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. As an illustration and not limitation, RAM is available in a variety of forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double-speed data rate SDRAM (SSRSDRAM), enhanced SDRAM (ESDRAM), synchronous link (Synchlink) DRAM (SLDRAM), memory bus (Rambus) direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and memory bus dynamic RAM.

[0120] It should be noted that, in this article, the terms "include", "comprises" or any other variations thereof are intended to cover non-exclusive inclusion, so that a process, device, article or method including a series of elements includes not only those elements, but also includes other elements not explicitly listed, or also includes elements inherent to such process, device, article or method. In the absence of further restrictions, an element defined by the sentence "includes a ..." does not exclude the presence of other identical elements in the process, device, article or method including the element.

[0121] The above description is only a preferred embodiment of the present invention, and does not limit the patent scope of the present invention. Any equivalent structure or equivalent process transformation made by using the contents of the present invention specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present invention.

Claims

1. A real-time monitoring method for an intelligent manufacturing digital twin system, characterized in that: The following steps are involved: Capturing images of the production site to obtain a real-time image sequence, and constructing a three-dimensional digital twin model based on the real-time image sequence; Analyze whether there are any abnormal phenomena in the production site based on the three-dimensional digital twin model. If so, identify abnormal areas in the three-dimensional digital twin model using a preset abnormality recognition algorithm to obtain abnormal state data; By using a preset dynamic time warping algorithm, the production site is subjected to abnormal deduction based on the abnormal state data to obtain an abnormal evolution trend; Performing a risk assessment on the production site based on the abnormal evolution trend to obtain a risk assessment result, and selecting a corresponding risk control strategy based on the risk assessment result; The abnormal phenomenon of the production site is deduced based on the abnormal state data by using a preset dynamic time warping algorithm to obtain an abnormal evolution trend, including: The abnormal phenomena at the production site are analyzed for spatiotemporal evolution using a preset dynamic time warping algorithm to obtain a state transition sequence; wherein the state transition sequence includes an abnormal evolution path, a state jump point, and a steady-state interval; Performing causal correlation analysis on the state transition sequence to obtain an abnormal propagation chain, and performing topological structure analysis on the abnormal propagation chain to obtain propagation network characteristics; wherein the propagation network characteristics include key node distribution, propagation path strength and network connectivity; Performing dynamic evolution analysis on the propagation network characteristics through a preset fractal theory to obtain a fractal feature set, and calculating the abnormal diffusion rate based on the fractal feature set; Based on the Koopman operator technology, the propagation network characteristics are dynamically modeled based on the abnormal diffusion rate to obtain an evolutionary dynamics equation, and the abnormal phenomenon at the production site is predicted based on the evolutionary dynamics equation to obtain predicted evolutionary situation parameters, wherein the predicted evolutionary situation parameters include abnormal expansion direction, evolutionary speed and impact radius; The predicted evolution situation parameters are subjected to dimensionality reduction processing to obtain main evolution characteristics, and through fuzzy comprehensive evaluation, abnormal trends of abnormal phenomena at the production site are deduced based on the main evolution characteristics to obtain abnormal evolution trends.

2. The real-time monitoring method of the intelligent manufacturing digital twin system according to claim 1 is characterized in that: The method of collecting images of the production site to obtain a real-time image sequence and constructing a three-dimensional digital twin model based on the real-time image sequence includes: Capturing multi-angle images of the production site by a multi-angle stereo vision algorithm to obtain a multi-angle real-time image sequence, and performing feature matching on the multi-angle real-time image sequence to obtain a matching feature point set; Performing triangulation on the matching feature point set to obtain dense point cloud data of the production site, and performing surface reconstruction based on the dense point cloud data to obtain a preliminary three-dimensional model; Mapping the multi-view images in the preliminary three-dimensional model by a texture mapping algorithm to obtain a texture enhanced three-dimensional model; Performing spatial alignment on the texture enhanced three-dimensional model to obtain a globally consistent three-dimensional model, and performing illumination correction on the globally consistent three-dimensional model to obtain a three-dimensional model with uniform illumination; The three-dimensional model with uniform illumination is semantically segmented by a deep learning algorithm to obtain and update a three-dimensional digital twin model with semantic information in real time.

3. The real-time monitoring method of the intelligent manufacturing digital twin system according to claim 1 is characterized in that: By using a preset abnormality recognition algorithm, abnormal areas are identified in the three-dimensional digital twin model to obtain abnormal state data, including: Inputting the three-dimensional digital twin model into a preset anomaly recognition algorithm; wherein the anomaly recognition algorithm includes a focusing module, a DySample module, an InceptionNext module and a YOLOv5s module, and the focusing module includes a spatial pyramid pooling structure and an ELAN attention mechanism; Inputting the three-dimensional digital twin model into the spatial pyramid pooling structure in the focusing module to perform multi-scale feature extraction to obtain a multi-scale feature map; Inputting the multi-scale feature map into the ELAN attention mechanism for feature focusing to obtain an enhanced focused feature map; Inputting the enhanced focus feature map into the DySample module for lightweight dynamic upsampling to obtain an upsampled feature map; Input the upsampled feature map into the InceptionNext module for multi-scale feature fusion to obtain a fused feature map; The fused feature map is input into the YOLOv5s module to identify and analyze abnormal regions to obtain abnormal state data; wherein the abnormal state data includes the abnormal region location, abnormal degree and abnormal type.

4. The real-time monitoring method of the intelligent manufacturing digital twin system according to claim 1 is characterized in that: Based on the main evolution characteristics, abnormal trends of abnormal phenomena at the production site are deduced to obtain abnormal evolution trends, including: Classify and analyze the main evolution characteristics to obtain classification results; wherein the classification results include equipment failure, process deviation, material defect, and environmental abnormality; Based on the classification results, association rule mining is performed on the abnormal phenomena to obtain an association rule set; Fuzzy logic reasoning is performed on the association rule set to obtain an abnormal trend prediction result, and the abnormal evolution trend is obtained based on the abnormal trend prediction result.

5. The real-time monitoring method of the intelligent manufacturing digital twin system according to claim 1 is characterized in that: The risk assessment of the production site based on the abnormal evolution trend is performed to obtain a risk assessment result, and a corresponding risk control strategy is selected based on the risk assessment result, including: Quantitatively evaluate the abnormal evolution trend of the production site to obtain abnormal risk quantification results, and predict the risk of the production site based on the abnormal risk quantification results to obtain risk warning data; wherein the risk warning data includes abnormal risk level, abnormal risk duration and risk loss assessment; Based on the risk warning data, a multi-dimensional risk assessment is performed on the production site to obtain a risk assessment result; wherein the risk assessment result includes the possibility of occurrence in a designated area of ​​the production site, the degree of impact of abnormal risks on production, and the potential diffusion path of abnormal risks; Based on the risk assessment results, a benefit-cost analysis is performed on the production site to obtain a risk control solution set; wherein the risk control solution set includes a production line adjustment strategy, a resource allocation strategy, and a human resource deployment strategy; Based on grey correlation analysis, a risk control solution set is evaluated to obtain a solution evaluation result, and based on an ant colony algorithm, the solution evaluation result is optimized and selected to obtain a risk control strategy.

6. The real-time monitoring method of the intelligent manufacturing digital twin system according to claim 5 is characterized in that: The benefit-cost analysis of the production site is performed based on the risk assessment result to obtain a risk control solution set, including: Based on the variational inference method, a probability distribution analysis is performed on the risk assessment result to obtain a risk probability density function, and the risk probability density function is divided into risk levels based on kernel density estimation to obtain a risk grading result; The risk classification results are quantitatively evaluated through multi-criteria decision making to obtain a risk cost matrix, and the risk cost matrix is ​​subjected to benefit analysis to obtain a benefit evaluation result; wherein the benefit evaluation result includes equipment loss cost, production delay loss and maintenance expenditure; Based on the benefit evaluation results, the resource allocation of the production site is optimized to obtain a resource allocation plan, and the resource allocation plan is optimized through the preset constraint planning rules to obtain an optimized scheduling strategy; wherein the optimized scheduling strategy includes a personnel deployment plan, an equipment maintenance plan and a material allocation strategy; Generate alternatives for the optimization scheduling strategy through a combinatorial optimization algorithm to obtain an alternative solution set, and perform multi-objective trade-offs on the alternative solution set based on Pareto frontier technology to obtain a feasible solution set; wherein the feasible solution set includes risk control effect, resource utilization efficiency and economic cost benefit; Based on game theory, the feasible solution set is compared and analyzed to obtain a solution evaluation result, and the solution evaluation result is sorted based on the grey correlation method to obtain a solution priority sequence; wherein the solution priority sequence includes solution feasibility, solution economy and solution urgency; Based on a dynamic programming algorithm, risk control is performed on the production site based on the scheme priority sequence to obtain a risk control scheme set.

7. A real-time monitoring device for an intelligent manufacturing digital twin system, characterized in that: include: An acquisition module is used to acquire images of the production site, obtain a real-time image sequence, and construct a three-dimensional digital twin model based on the real-time image sequence; An identification module is used to analyze whether there are abnormal phenomena in the production site based on the three-dimensional digital twin model. If there are abnormal phenomena, an abnormal area is identified in the three-dimensional digital twin model through a preset abnormality identification algorithm to obtain abnormal state data; A deduction module, used to perform abnormal deduction on the production site based on the abnormal state data through a preset dynamic time warping algorithm to obtain an abnormal evolution trend; An assessment module, used to perform risk assessment on the production site based on the abnormal evolution trend, obtain a risk assessment result, and select a corresponding risk control strategy based on the risk assessment result; The abnormal phenomenon of the production site is deduced based on the abnormal state data by using a preset dynamic time warping algorithm to obtain an abnormal evolution trend, including: The abnormal phenomena at the production site are analyzed for spatiotemporal evolution using a preset dynamic time warping algorithm to obtain a state transition sequence; wherein the state transition sequence includes an abnormal evolution path, a state jump point, and a steady-state interval; Performing causal correlation analysis on the state transition sequence to obtain an abnormal propagation chain, and performing topological structure analysis on the abnormal propagation chain to obtain propagation network characteristics; wherein the propagation network characteristics include key node distribution, propagation path strength and network connectivity; Performing dynamic evolution analysis on the propagation network characteristics through a preset fractal theory to obtain a fractal feature set, and calculating the abnormal diffusion rate based on the fractal feature set; Based on the Koopman operator technology, the propagation network characteristics are dynamically modeled based on the abnormal diffusion rate to obtain an evolutionary dynamics equation, and the abnormal phenomenon at the production site is predicted based on the evolutionary dynamics equation to obtain predicted evolutionary situation parameters, wherein the predicted evolutionary situation parameters include abnormal expansion direction, evolutionary speed and impact radius; The predicted evolution situation parameters are subjected to dimensionality reduction processing to obtain main evolution characteristics, and through fuzzy comprehensive evaluation, abnormal trends of abnormal phenomena at the production site are deduced based on the main evolution characteristics to obtain abnormal evolution trends.

8. A computer device comprising a memory and a processor, wherein a computer program is stored in the memory, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 6 are implemented.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 6 are implemented.

Images