A smart factory management method and system based on digital twin technology

Abstract

The present invention relates to the technical field of smart factories, and discloses a smart factory management method and system based on digital twin technology, the system comprising a physical equipment and physical model update matching module, a physical equipment and personnel operation parameterized risk prediction module; through the physical equipment and physical model update matching module, state feature data of physical equipment entering and leaving the factory, state feature data of operators entering and leaving the factory are collected and a data set is established, the matching physical equipment three-dimensional model data is called to match the physical equipment layout in the current factory, the matching human body three-dimensional model data is called to match the operator's work station in the current factory, and the Internet of Things is used to make the factory physical equipment and the physical equipment three-dimensional model connect and respond, thereby realizing automatic monitoring and updating of the factory physical equipment and personnel entry and exit data so that the physical equipment and the three-dimensional physical model are automatically matched and updated.

Description

Technical Field

[0001] The present invention relates to the technical field of smart factories, and specifically to a smart factory management method and system based on digital twin technology. Background Art

[0002] Smart factories are based on relevant data of the entire life cycle of products. In a computer virtual environment, they simulate, evaluate and optimize the entire production process, and further expand to a new production organization mode for the entire product life cycle. In a virtual environment, the production and manufacturing process is compressed and advanced, and can be evaluated and tested. Digital twin technology makes full use of data such as physical models, sensor updates, and operation history, integrates multi-disciplinary, multi-physical quantity, multi-scale, and multi-probability simulation processes, and completes mapping in virtual space, thereby reflecting the entire life cycle of the corresponding physical equipment. Smart factories combine digital twin technology to shorten the time from product design to production, and improve product reliability and success rate. Smart factories use digital twin technology to simulate, simulate, and monitor online physical equipment in virtual space in all aspects of product design, manufacturing, assembly, and logistics. However, in the existing digital twin technology, physical models of physical equipment in virtual space need to be established manually, and physical models of physical equipment in the factory and in virtual space cannot be automatically updated and matched synchronously, nor can risk prediction be performed based on physical equipment operating parameters and personnel physiological parameters.

[0003] The Chinese patent publication number is CN115310936A, which discloses a digital twin-based smart logistics factory visualization and data service technology system. It uses digital twin technology, three-dimensional visualization technology, and data service technology systems to open up data channels, and can realize the visualization and data service of the logistics factory operation process in the factory area. It can realize real-time viewing of the factory environment, logistics process status, and abnormal conditions, monitor the operating equipment status data, analyze and infer the operating parameters, realize decision-making functions, and help to avoid faults in time. It has high efficiency, real-time, and fidelity. High efficiency is reflected in the establishment of static three-dimensional models and AGV path optimization; real-time is reflected in data collection and message subscription push; fidelity is reflected in the dynamic rendering of three-dimensional models, using the static three-dimensional model component module to obtain logistics equipment components and warehouse scene photos for CAD drawing and cooperate with Maya modeling unit and 3D engine unit to build the physical model of the entity equipment. However, CAD drawing using logistics equipment components and warehouse scene photos still requires human intervention in modeling, and real-time synchronous matching and updating of entity equipment and physical models cannot be achieved; the fault alarm module collects the logistics information of the entity equipment for fault alarm, and cannot predict equipment failure risks and operator event risks based on the entity equipment operating parameters and personnel physiological parameters. Summary of the invention

[0004] 1. Technical issues to be resolved

[0005] In order to solve the problem that in the above-mentioned existing digital twin technology, physical models of physical equipment are established in virtual space, manual modeling is required, and the physical equipment in the factory and the physical models in the virtual space cannot be automatically updated and matched synchronously, and risk prediction cannot be performed based on the operating parameters of the physical equipment and the physiological parameters of the personnel. The above-mentioned factory physical equipment and personnel entry and exit data are automatically monitored and updated, so that the physical equipment and the physical model are automatically matched, and risk prediction is performed for the operating parameters of the physical equipment and the physiological parameters of the operators.

[0006] (II) Technical solution

[0007] The present invention is implemented through the following technical solution: a smart factory management method based on digital twin technology, the method comprising the following steps:

[0008] S1. Collecting status characteristic data of physical equipment entering and leaving the factory, and obtaining status characteristic data of operators entering and leaving the factory;

[0009] S2, obtaining the current layout image of the physical equipment inside the factory; obtaining the current working position image of the operator inside the factory;

[0010] S3, comparing the current factory internal physical equipment layout image with the original factory internal physical equipment layout image to obtain current difference image feature data; comparing the current factory internal operator work station image with the original factory internal operator work station image to obtain current operator difference image feature data;

[0011] S4, identifying the physical equipment in the physical equipment layout inside the factory through the current difference image feature data; identifying the operator in the operator's work station inside the factory through the current operator difference image feature data;

[0012] S5, using the physical equipment and operators identified by S4 to respectively match the physical equipment three-dimensional model corresponding to the physical equipment state feature data in S1 and the human body three-dimensional model corresponding to the operator state feature data, respectively match them to the digital virtual space factory layout, and after the physical equipment in the physical equipment layout inside the factory is matched with the physical equipment three-dimensional model, a response is made through the Internet of Things connection;

[0013] S6. Obtaining the operating status parameters of the physical equipment and the physiological parameters of the operators in the current factory layout online in real time through sensors; and matching the operating status parameters of the physical equipment and the physiological parameters of the operators to the digital virtual space factory layout;

[0014] S7, based on the current factory physical equipment operation status parameters and the updated current operator operation physiological parameters, calculate and predict the probability of failure of the factory internal physical equipment operation and the probability of emergencies of the operator operation;

[0015] S8. When at least one of the failure probability and the emergency probability reaches a set threshold, an early warning prompt is issued.

[0016] Preferably, the state characteristic data of the physical equipment entering and exiting the factory includes a three-dimensional model of the physical equipment.

[0017] The state characteristic data of the physical equipment entering and leaving the factory also includes one or more of model, name, purpose, use environment conditions, and external dimensions.

[0018] The state characteristic data of the operator entering and leaving the factory includes a three-dimensional model of the human body.

[0019] The status characteristic data of the operator entering and leaving the factory also includes one or more of name, education background, job title, unit, department, age, and gender.

[0020] The collecting of state characteristic data of physical equipment entering and leaving the factory, and the obtaining of state characteristic data of operators entering and leaving the factory include the following steps:

[0021] S11. Establishing a state feature data set A = [a1, ..., a i , …, a n ], i = 1, 2, 3, ... n, i represents the status characteristic data number of different types of entity equipment, a i Represents the status characteristic data of physical equipment inside different types of factories;

[0022] Establish the state characteristic data set B = [b1, ..., b j , …, b m ], j = 1, 2, 3, ... m, j represents the operator status characteristic data number of different individuals, b j Represents the status characteristic data of different individual operators.

[0023] Preferably, the steps of obtaining the current factory internal physical equipment layout image and extracting the original factory internal physical equipment layout image and identifying the difference image features using an artificial neural network are as follows:

[0024] S21, obtain and set the current factory internal physical equipment layout image as C 当前 , the original factory internal physical equipment layout image is C 原始 ;

[0025] S22, using the DFP algorithm in the quasi-Newton algorithm to calculate C 当前 and C 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain C 1 当前 and C 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δ k and k Represents a vector sequence;

[0026] Using multiple groups of C 1 当前 and C 1 原始 The data are used to train the artificial neural network to generate test data C 2 当前 and C 2 原始 , until the test data C generated by the artificial neural network 2 当前 and C 2 原始 Satisfy image recognition accuracy;

[0027] S23, using the artificial neural network trained in S22 to identify and extract C 1 当前 With C 1 原始 The current difference image feature data C 差异 ;

[0028] Similarly, the steps of obtaining the distinguishing image feature data of the current operator according to steps S21, S22, and S23 are as follows:

[0029] S24, first obtain and set the current factory internal operator work station image as D 当前 The original factory internal operator workstation image is D 原始 ;

[0030] S25, using the DFP algorithm in the quasi-Newton algorithm to 当前 and D 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain D 1 当前 and D 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δk and k Represents a vector sequence;

[0031] Using multiple groups of D 1 当前 and D 1 原始 The data are used to train the artificial neural network to generate test data D 2 当前 and D 2 原始 , until the test data D generated by the artificial neural network 2 当前 and D 2 原始 Satisfy image recognition accuracy;

[0032] S26, using the artificial neural network trained in S25 to identify and extract D 1 当前 With D 1 原始 The current operator distinguishing image feature data D 区别 .

[0033] Preferably, the current difference image feature data identifies the physical equipment in the physical equipment layout inside the factory; the current operator difference image feature data identifies the operator in the operator work station inside the factory,

[0034] The physical equipment and operators identified by S4 are matched to obtain the corresponding three-dimensional model of the physical equipment in the physical equipment state feature data in S1 and the corresponding three-dimensional model of the human body in the operator state feature data, and they are matched to the digital virtual space factory layout respectively. The operation steps are as follows:

[0035] S51, using an artificial neural network based on the current difference image feature data C in S23 差异 Traverse the state feature data set A of the physical equipment entering and leaving the factory to identify the physical equipment data a of the specific changes in the physical equipment layout inside the factory i ;

[0036] S52, artificial neural network calls the internal entity device data a of set A. i The 3D model of the physical equipment is created and matched to the digital virtual space factory layout;

[0037] S53, using an artificial neural network to distinguish image feature data D according to the current operator in S26 区别 Traverse the state feature data set B of operators entering and leaving the factory to identify the operator data b of the specific changes in the operator's work station inside the factory j ;

[0038] S54, artificial neural network calls set B internal operator data b j The corresponding three-dimensional model of the human body is created and matched to the digital virtual space factory layout.

[0039] Preferably, the operating status parameters of the factory physical equipment include one or more of power, speed, voltage, current, pressure, temperature, power, flow, heat, composition, humidity, and liquid level; the operating physiological parameters of the operator include one or more of body temperature, respiratory rate, systolic blood pressure, diastolic blood pressure, and heart rate;

[0040] The steps of obtaining the operating status parameters of the physical equipment and the physiological parameters of the operator in the current factory layout in real time online by combining the sensor with the physical equipment 3D model and the human body 3D model matched in the digital virtual space factory layout are as follows:

[0041] S61. Establish the factory physical equipment operation status parameter matrix q=1,2,3,…t,z=1,2,3,…r,q represents the number of different types of operation status parameters of physical equipment, z represents the number of samples of different types of operation status parameters of physical equipment, e qz Indicates different types of operating status parameters of physical equipment with different numbers of samples;

[0042] Establishing the operator's physiological parameter matrix w=1,2,3,…s,x=1,2,3,…v,w represents the number of different types of physiological parameters of the operator's operation, x represents the number of samples of different types of physiological parameters of the operator's operation, f wx Indicates different types of physiological parameters of operators operating with different numbers of samples.

[0043] Preferably, the operation steps of calculating the physical equipment failure probability and the operator event probability are as follows:

[0044] S71. Based on the factory physical equipment operation status parameter matrix E in S61, solve the probability of abnormal operation status parameters in different types of operation status parameters of physical equipment respectively. where e q Indicates different types of operating status parameters of physical equipment;

[0045] like It indicates the probability of abnormal operation status parameter of entity device q type Probability of exceeding the safety threshold of the physical device q type operating status parameter

[0046] like It indicates the probability of abnormal operation status parameter of entity device q type The probability of not exceeding the safety threshold of the physical device q type operating status parameter

[0047] S72, according to the operator's physiological parameter matrix F in S61, respectively solve the probability of abnormal physiological parameters in different types of physiological parameters of the operator's operation where f w Indicates different types of physiological parameters of operators’ operations;

[0048] like It indicates the probability of abnormal physiological parameters of operator operation type w Probability of exceeding the safety threshold of the operator's physiological parameters of type w

[0049] like It indicates the probability of abnormal physiological parameters of operator operation type w The probability of not exceeding the safety threshold of the operator's type w physiological parameters

[0050] S73, counting the probability and value of different types of abnormal operation status parameters of physical equipment exceeding the safety threshold and

[0051] The probability and value of operators’ different types of abnormal physiological parameters exceeding the safety threshold are calculated. and

[0052] S74, when Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device. The physical device is likely to fail if it continues to operate. Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device not exceeding the threshold. The physical device has a small probability of failure when it continues to operate;

[0053] when It indicates the probability of abnormal physiological parameters of operators exceeding the safety threshold and exceeding the operator's safe operation probability threshold. If the operator continues to work, an emergency is likely to occur. It indicates the probability of all abnormal physiological parameters of the operator exceeding the safety threshold and the probability threshold of safe operation of the operator. If the operator continues to work, there is a small probability that an emergency will occur.

[0054] Preferably, the operation steps of the early warning prompt S8 in case of a running failure of a physical device or an emergency of an operator are as follows:

[0055] S81, according to the judgment result of S74, when If the physical equipment continues to operate, it is likely to fail, and the warning prompts the management personnel to carry out maintenance on the corresponding physical equipment. The physical equipment will continue to operate with a small probability of failure, without the need for early warning to alert management personnel;

[0056] when If the operator continues to work, there is a high probability that an emergency will occur. The early warning prompts the management to conduct a physical examination of the operator. There is a small chance that an emergency may occur while the operator continues to work, and there is no need to warn the management staff.

[0057] A smart factory management system, the system includes a physical equipment and physical model update matching module, a physical equipment and personnel operation parameterized risk prediction module;

[0058] The physical equipment and physical model update matching module includes a factory equipment and personnel entry and exit data acquisition module, a factory internal physical equipment layout recognition module, a virtual space physical model matching module, and a physical equipment and physical model communication response module;

[0059] The factory equipment and personnel entry and exit data collection module uses information entry equipment to collect status characteristic data of physical equipment entering and leaving the factory, as well as to obtain status characteristic data of operators entering and leaving the factory;

[0060] The module for identifying the layout of physical equipment inside the factory uses a camera lens and an artificial neural network algorithm to collect images of the layout of physical equipment inside the factory and images of the workstations of operators inside the factory, and compares them with the original images of the layout of physical equipment inside the factory and the original images of the workstations of operators inside the factory to identify the corresponding difference image feature numbers and the distinguishing image feature data of the current operator; at the same time, the difference image feature numbers and the distinguishing image feature data of the current operator are used to extract the physical equipment in the layout of physical equipment inside the factory and the operators in the workstations of operators inside the factory;

[0061] The virtual space physical model matching module uses physical equipment and operators to retrieve the state feature data of physical equipment entering and leaving the factory, the state feature data of operators entering and leaving the factory, and the corresponding physical equipment 3D model data and human body 3D model data, and matches the physical equipment 3D model data to the physical equipment layout in the current factory in the digitally constructed virtual space factory, and matches the human body 3D model data to the operator's work station in the current factory;

[0062] The physical device and physical model communication response module uses the Internet of Things network to enable the physical device connection response after the three-dimensional model of the physical device in the virtual space and the physical device in the real factory layout change;

[0063] The physical equipment and personnel operation parameterized risk prediction module includes a factory physical equipment operation status parameter acquisition module, an operator operation physiological parameter acquisition module, a physical equipment failure probability and operator emergency probability operation module, and an equipment failure and operator emergency risk warning module;

[0064] A module for obtaining the operating status parameters of the plant's physical equipment, using sensors to obtain the operating status parameters of the plant's physical equipment online in real time;

[0065] The operator's physiological parameters acquisition module uses smart wearable devices and sensors to measure the operator's physiological parameters online;

[0066] The physical equipment failure probability and operator emergency probability operation module uses the factory physical equipment operation status parameters and the operator's physiological parameters combined with the probability algorithm to calculate and predict the probability of failure of the factory's internal physical equipment operation and the probability of emergency events in the operator's operation;

[0067] The equipment failure and operator emergency risk warning module uses information push or alarms to warn of the high probability of operational failures in physical equipment and the high probability of emergencies in operators' continued operations.

[0068] Preferably, the physical equipment and physical model update matching module and the physical equipment and personnel operation parameterized risk prediction module are connected through Internet communication, and the physical equipment and physical model communication response module is respectively connected with the factory physical equipment operation status parameter acquisition module and the operator operation physiological parameter acquisition module through Internet communication.

[0069] (III) Beneficial effects

[0070] The present invention provides a smart factory management method and system based on digital twin technology, which has the following beneficial effects:

[0071] 1. Automatically monitor and update the factory physical equipment and personnel entry and exit data to automatically match and update the physical equipment and three-dimensional physical model; predict the probability of failure of physical equipment in the factory and the probability of emergencies in the operator's work, and at the same time warn that the physical equipment is likely to fail if it continues to run and the operator is likely to have an emergency if he continues to work, thereby realizing risk prediction of physical equipment failures and emergencies in the operator's work in the smart factory.

[0072] 2. Through the factory equipment and personnel entry and exit data acquisition module and the factory internal physical equipment layout identification module, the state feature data of physical equipment entering and exiting the factory, the state feature data of operators entering and exiting the factory are collected and a data set is established. The current difference image feature data is extracted by using an artificial neural network for identifying the physical equipment in the factory layout, and the current operator difference image feature data is identified for identifying the operator in the factory work station. The corresponding physical equipment three-dimensional model data in the state feature data of the physical equipment entering and exiting the factory is obtained based on the physical equipment, and the human body three-dimensional model data is obtained based on the current operator difference image feature data; the virtual space physical model matching module and the physical equipment and physical model communication response module respectively match the physical equipment three-dimensional model data to the current factory internal physical equipment layout in the digitally constructed virtual space factory, and match the human body three-dimensional model data to the current factory internal operator work station. At the same time, the Internet of Things is used to connect and respond to the factory physical equipment and physical equipment three-dimensional models, thereby realizing the automatic monitoring and updating of the physical equipment and personnel entry and exit data in the smart factory, so that the physical equipment and physical models are automatically matched and updated, which improves the convenience of digital twin technology in the application of smart factories.

[0073] 3. The factory physical equipment operation status parameter acquisition module and the operator operation physiological parameter acquisition module are used to obtain the factory physical equipment operation status parameters and the operator operation physiological parameters online in real time and establish a data matrix; the physical equipment failure probability and operator emergency probability operation module and the equipment failure and operator emergency risk warning module combine the factory physical equipment operation status parameters and the operator operation physiological parameters with the probability algorithm to calculate and predict the probability of failure of the factory internal physical equipment operation and the probability of emergency of the operator operation. At the same time, the warning prompts that the physical equipment is likely to fail if it continues to operate and the operator is likely to have an emergency if it continues to operate, thereby predicting the risks of physical equipment operation failures and operator operation emergencies, thereby improving the reliability of equipment operation and the safety of personnel operations in smart factories. BRIEF DESCRIPTION OF THE DRAWINGS

[0074] Figure 1 This is a flow chart of the smart factory management method based on digital twin technology of the present invention;

[0075] Figure 2 This is a module structure diagram of the smart factory management system of the present invention. DETAILED DESCRIPTION

[0076] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0077] See also Figure 1 , a smart factory management method based on digital twin technology, the method comprising the following steps:

[0078] S1. Collecting status characteristic data of physical equipment entering and leaving the factory, and obtaining status characteristic data of operators entering and leaving the factory;

[0079] S2, obtaining the current layout image of the physical equipment inside the factory; obtaining the current working position image of the operator inside the factory;

[0080] S3, comparing the current factory internal physical equipment layout image with the original factory internal physical equipment layout image to obtain current difference image feature data; comparing the current factory internal operator work station image with the original factory internal operator work station image to obtain current operator difference image feature data;

[0081] S4, identifying the physical equipment in the physical equipment layout inside the factory through the current difference image feature data; identifying the operator in the operator's work station inside the factory through the current operator difference image feature data;

[0082] S5, using the physical equipment and operators identified by S4 to respectively match the physical equipment three-dimensional model corresponding to the physical equipment state feature data in S1 and the human body three-dimensional model corresponding to the operator state feature data, respectively match them to the digital virtual space factory layout, and after the physical equipment in the physical equipment layout inside the factory is matched with the physical equipment three-dimensional model, a response is made through the Internet of Things connection;

[0083] The original factory internal physical equipment layout image is set to the layout image when there is no physical equipment in the factory, and the original factory internal operator work station image is set to the work station image when there is no operator in the factory.

[0084] If the current factory internal physical equipment layout image contains additional image feature data of physical equipment compared to the original factory internal physical equipment layout image, indicating that physical equipment has entered the factory, the current difference image feature data is mapped to the digital virtual space factory layout; if the current factory internal operator work station image contains additional image feature data of operators compared to the original factory internal operator work station image, indicating that operators have entered the factory, the current operator difference image feature data is mapped to the digital virtual space factory layout.

[0085] S6. Obtaining the operating status parameters of the physical equipment and the physiological parameters of the operators in the current factory layout online in real time through sensors; and matching the operating status parameters of the physical equipment and the physiological parameters of the operators to the digital virtual space factory layout;

[0086] S7, based on the current factory physical equipment operation status parameters and the updated current operator operation physiological parameters, calculate and predict the probability of failure of the factory internal physical equipment operation and the probability of emergencies of the operator operation;

[0087] S8. When at least one of the failure probability and the emergency probability reaches a set threshold, an early warning prompt is issued.

[0088] For further information, see Figure 1 The state characteristic data of physical equipment entering and leaving the factory includes a three-dimensional model of the physical equipment, and also includes one or more of the model, name, purpose, use environment conditions, and external dimensions; the state characteristic data of operators entering and leaving the factory includes a three-dimensional model of the human body, and also includes one or more of the name, education background, professional title, unit, department, age, and gender;

[0089] Collecting the status characteristic data of physical equipment entering and leaving the factory, and obtaining the status characteristic data of operators entering and leaving the factory includes the following steps:

[0090] S11. Establishing a state feature data set A = [a1, ..., a i , …, a n ], i = 1, 2, 3, ... n, i represents the status characteristic data number of different types of entity equipment, a i Represents the status characteristic data of physical equipment inside different types of factories;

[0091] Establish the state characteristic data set B = [b1, ..., b j , ..., b m ], j = 1, 2, 3, ... m, j represents the operator status characteristic data number of different individuals, b j Represents the status characteristic data of different individual operators.

[0092] For further information, see Figure 1 , the steps of obtaining the current factory internal physical equipment layout image and extracting the original factory internal physical equipment layout image and using artificial neural network to identify the difference image features are as follows:

[0093] S21, obtain and set the current factory internal physical equipment layout image as C 当前 , the original factory internal physical equipment layout image is C 原始 ;

[0094] S22, using the DFP algorithm in the quasi-Newton algorithm to calculate C 当前 and C 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain C 1 当前 and C 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δ k and k Represents a vector sequence;

[0095] Using multiple groups of C 1 当前 and C 1 原始 The data are used to train the artificial neural network to generate test data C 2 当前 and C 2 原始 , until the test data C generated by the artificial neural network 2 当前 and C 2 原始 Satisfy image recognition accuracy;

[0096] S23, using the artificial neural network trained in S22 to identify and extract C 1 当前 With C 1 原始 The current difference image feature data C 差异 ;

[0097] Similarly, the steps of obtaining the distinguishing image feature data of the current operator according to steps S21, S22, and S23 are as follows:

[0098] S24, first obtain and set the current factory internal operator work station image as D 当前 The original factory internal operator workstation image is D 原始 ;

[0099] S25, using the DFP algorithm in the quasi-Newton algorithm to 当前 and D 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain D 1 当前 and D 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δ k and k Represents a vector sequence;

[0100] Using multiple groups of D 1 当前 and D 1 原始 The data are used to train the artificial neural network to generate test data D 2 当前 and D 2 原始 , until the test data D generated by the artificial neural network 2 当前 and D 2 原始 Satisfy image recognition accuracy;

[0101] S26, using the artificial neural network trained in S25 to identify and extract D 1 当前 With D 1 原始 The current operator distinguishing image feature data D 区别 .

[0102] For further information, see Figure 1 , identifying the physical equipment in the physical equipment layout inside the factory through the current difference image feature data; identifying the operator in the operator's work station inside the factory through the current operator difference image feature data;

[0103] The physical equipment and operators identified by S4 are matched to obtain the corresponding physical equipment 3D model in the physical equipment state feature data in S1 and the corresponding human body 3D model in the operator state feature data, and the operation steps of matching them to the digital virtual space factory layout are as follows:

[0104] S51, using an artificial neural network based on the current difference image feature data C in S23 差异 Traverse the state feature data set A of the physical equipment entering and leaving the factory to identify the physical equipment data a of the specific changes in the physical equipment layout inside the factory i;

[0105] S52, artificial neural network calls the internal entity device data a of set A. i The 3D model of the physical equipment is created and matched to the digital virtual space factory layout;

[0106] S53, using an artificial neural network to distinguish image feature data D according to the current operator in S26 区别 Traverse the state feature data set B of operators entering and leaving the factory to identify the operator data b of the specific changes in the operator's work station inside the factory j ;

[0107] S54, artificial neural network calls set B internal operator data b i The corresponding three-dimensional model of the human body is created and matched to the digital virtual space factory layout.

[0108] The operating status parameters of the factory physical equipment include one or more of power, speed, voltage, current, pressure, temperature, power, flow, heat, composition, humidity, and liquid level; the operating physiological parameters of the operator include one or more of body temperature, respiratory rate, systolic blood pressure, diastolic blood pressure, and heart rate;

[0109] Based on the physical equipment 3D model and human body 3D model matched to the digital virtual space factory layout, combined with sensors, the operation steps of obtaining the physical equipment operation status parameters and operator physiological parameters in the current factory layout in real time online are as follows:

[0110] S61. Establish the factory physical equipment operation status parameter matrix q=1,2,3,…t,z=1,2,3,…r,q represents the number of different types of operation status parameters of physical equipment, z represents the number of samples of different types of operation status parameters of physical equipment, e qz Indicates different types of operating status parameters of physical equipment with different numbers of samples;

[0111] Establishing the operator's physiological parameter matrix w=1,2,3,…s,x=1,2,3,…v,w represents the number of different types of physiological parameters of the operator's operation, x represents the number of samples of different types of physiological parameters of the operator's operation, f wx Indicates different types of physiological parameters of operators operating with different numbers of samples.

[0112] For further information, see Figure 1 ,The steps to calculate the probability of physical equipment failure and operator event probability are as follows:

[0113] S71. Based on the factory physical equipment operation status parameter matrix E in S61, solve the probability of abnormal operation status parameters in different types of operation status parameters of physical equipment respectively. where e q Indicates different types of operating status parameters of physical equipment;

[0114] like It indicates the probability of abnormal operation status parameter of entity device q type Probability of exceeding the safety threshold of the physical device q type operating status parameter

[0115] like It indicates the probability of abnormal operation status parameter of entity device q type The probability of not exceeding the safety threshold of the physical device q type operating status parameter

[0116] S72, according to the operator's physiological parameter matrix F in S61, respectively solve the probability of abnormal physiological parameters in different types of physiological parameters of the operator's operation where f w Indicates different types of physiological parameters of operators’ operations;

[0117] like It indicates the probability of abnormal physiological parameters of operator operation type w Probability of exceeding the safety threshold of the operator's physiological parameters of type w

[0118] like It indicates the probability of abnormal physiological parameters of operator operation type w The probability of not exceeding the safety threshold of the operator's type w physiological parameters

[0119] S73, counting the probability and value of different types of abnormal operation status parameters of physical equipment exceeding the safety threshold and

[0120] The probability and value of operators’ different types of abnormal physiological parameters exceeding the safety threshold are calculated. and

[0121] S74, when Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device. The physical device is likely to fail if it continues to operate. Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device not exceeding the threshold. The physical device has a small probability of failure when it continues to operate;

[0122] when It indicates the probability of abnormal physiological parameters of operators exceeding the safety threshold and exceeding the operator's safe operation probability threshold. If the operator continues to work, an emergency is likely to occur. It indicates the probability of all abnormal physiological parameters of the operator exceeding the safety threshold and the probability threshold of safe operation of the operator. If the operator continues to work, there is a small probability that an emergency will occur.

[0123] For further information, see Figure 1 , the operation steps for early warning S8 when a physical device fails or an operator encounters an emergency are as follows:

[0124] S81, according to the judgment result of S74, when If the physical equipment continues to operate, it is likely to fail, and the warning prompts the management personnel to carry out maintenance on the corresponding physical equipment. The physical equipment will continue to operate with a small probability of failure, without the need for early warning to alert management personnel;

[0125] when If the operator continues to work, there is a high probability that an emergency will occur. The early warning prompts the management to conduct a physical examination of the operator. There is a small chance that an emergency may occur while the operator continues to work, and there is no need to warn the management staff.

[0126] For further information, see Figure 1-Figure 2 , a smart factory management system, the system includes a physical equipment and physical model update matching module, a physical equipment and personnel operation parameterized risk prediction module;

[0127] Automatic monitoring and updating of factory physical equipment and personnel entry and exit data enables automatic matching and updating of physical equipment and three-dimensional physical models; predicts the probability of failure of physical equipment in the factory and the probability of emergencies in operators' operations, and simultaneously warns of the high probability of operational failure of physical equipment and the high probability of emergencies in operators' operations if the equipment continues to operate, thereby achieving risk prediction of physical equipment operation failure and emergencies in operators' operations in smart factories.

[0128] The physical equipment and physical model update matching module includes a factory equipment and personnel entry and exit data acquisition module, a factory internal physical equipment layout recognition module, a virtual space physical model matching module, and a physical equipment and physical model communication response module;

[0129] The factory equipment and personnel entry and exit data collection module uses information entry equipment to collect status characteristic data of physical equipment entering and leaving the factory, as well as to obtain status characteristic data of operators entering and leaving the factory;

[0130] The module for identifying the layout of physical equipment inside the factory uses a camera lens and an artificial neural network algorithm to collect images of the layout of physical equipment inside the factory and images of the workstations of operators inside the factory, and compares them with the original images of the layout of physical equipment inside the factory and the original images of the workstations of operators inside the factory to identify the corresponding difference image feature numbers and the distinguishing image feature data of the current operator; at the same time, the difference image feature numbers and the distinguishing image feature data of the current operator are used to extract the physical equipment in the layout of physical equipment inside the factory and the operators in the workstations of operators inside the factory;

[0131] The virtual space physical model matching module uses physical equipment and operators to retrieve the state feature data of physical equipment entering and leaving the factory, the state feature data of operators entering and leaving the factory, and the corresponding physical equipment 3D model data and human body 3D model data, and matches the physical equipment 3D model data to the physical equipment layout in the current factory in the digitally constructed virtual space factory, and matches the human body 3D model data to the operator's work station in the current factory;

[0132] The physical device and physical model communication response module uses the Internet of Things network to enable the physical device connection response after the three-dimensional model of the physical device in the virtual space and the physical device in the real factory layout change;

[0133] Through the factory equipment and personnel entry and exit data acquisition module and the factory internal physical equipment layout recognition module, the state feature data of physical equipment entering and exiting the factory, the state feature data of operators entering and exiting the factory are collected and a data set is established. The current difference image feature data is extracted by using an artificial neural network for identifying the physical equipment in the factory layout, and the current operator difference image feature data is identified for identifying the operator in the factory work station. The corresponding physical equipment three-dimensional model data in the state feature data of the physical equipment entering and exiting the factory is obtained based on the physical equipment, and the human body three-dimensional model data is obtained based on the current operator difference image feature data; the virtual space physical model matching module and the physical equipment and physical model communication response module respectively match the physical equipment three-dimensional model data to the current factory internal physical equipment layout in the digitally constructed virtual space factory, and match the human body three-dimensional model data to the current factory internal operator work station. At the same time, the Internet of Things is used to connect and respond to the factory physical equipment and physical equipment three-dimensional models, thereby realizing the automatic monitoring and updating of the physical equipment and personnel entry and exit data in the smart factory, so that the physical equipment and physical models are automatically matched and updated, which improves the convenience of digital twin technology in the application of smart factories.

[0134] The physical equipment and personnel operation parameterized risk prediction module includes a factory physical equipment operation status parameter acquisition module, an operator operation physiological parameter acquisition module, a physical equipment failure probability and operator emergency probability operation module, and an equipment failure and operator emergency risk warning module;

[0135] A module for obtaining the operating status parameters of the plant's physical equipment, using sensors to obtain the operating status parameters of the plant's physical equipment online in real time;

[0136] The operator's physiological parameters acquisition module uses smart wearable devices and sensors to measure the operator's physiological parameters online;

[0137] The physical equipment failure probability and operator emergency probability operation module uses the factory physical equipment operation status parameters and the operator's physiological parameters combined with the probability algorithm to calculate and predict the probability of failure of the factory's internal physical equipment operation and the probability of emergency events in the operator's operation;

[0138] The equipment failure and operator emergency risk warning module uses information push or alarms to warn of the high probability of operational failures in physical equipment and the high probability of emergencies in operators' continued operations.

[0139] The physical equipment and physical model update matching module and the physical equipment and personnel operation parameterized risk prediction module are connected through Internet communication, and the physical equipment and physical model communication response module is connected with the factory physical equipment operation status parameter acquisition module and the operator work physiological parameter acquisition module respectively through Internet communication.

[0140] The factory physical equipment operation status parameter acquisition module and the operator operation physiological parameter acquisition module are used to obtain the factory physical equipment operation status parameters and the operator operation physiological parameters online in real time and establish a data matrix; the physical equipment failure probability and operator emergency probability operation module and the equipment failure and operator emergency risk warning module combine the factory physical equipment operation status parameters and the operator operation physiological parameters with the probability algorithm to calculate and predict the probability of failure of the factory internal physical equipment operation and the probability of emergency events in the operator's operation. At the same time, the warning prompts that the physical equipment is likely to fail if it continues to operate and the operator is likely to have an emergency if it continues to operate, thereby predicting the risks of physical equipment operation failures and operator operation emergencies, thereby improving the reliability of equipment operation and the safety of personnel operations in smart factories.

Claims

1. A smart factory management method based on digital twin technology, characterized in that: The method comprises the following steps: S1. Collecting status characteristic data of physical equipment entering and leaving the factory, and obtaining status characteristic data of operators entering and leaving the factory; S2, obtaining the current layout image of the physical equipment inside the factory; obtaining the current working position image of the operator inside the factory; S3, comparing the current factory internal physical equipment layout image with the original factory internal physical equipment layout image to obtain current difference image feature data; comparing the current factory internal operator work station image with the original factory internal operator work station image to obtain current operator difference image feature data; S4, identifying the physical equipment in the physical equipment layout inside the factory through the current difference image feature data; identifying the operator in the operator's work station inside the factory through the current operator difference image feature data; S5, using the physical equipment and operators identified in S4 to respectively match the physical equipment three-dimensional model corresponding to the physical equipment state feature data in S1 and the human body three-dimensional model corresponding to the operator state feature data, respectively matching the physical equipment three-dimensional model and the human body three-dimensional model to the digital virtual space factory layout, and the physical equipment in the physical equipment layout inside the factory is matched with the physical equipment three-dimensional model and then connected and responded through the Internet of Things; S6. Obtaining the operating status parameters of the physical equipment and the physiological parameters of the operators in the current factory layout online in real time through sensors; and matching the operating status parameters of the physical equipment and the physiological parameters of the operators to the digital virtual space factory layout; S7, based on the current factory physical equipment operation status parameters and the updated current operator operation physiological parameters, calculate and predict the probability of failure of the factory internal physical equipment operation and the probability of emergencies of the operator operation; S8. When at least one of the failure probability and the emergency probability reaches a set threshold, an early warning is issued; The state characteristic data of the physical equipment entering and leaving the factory also includes one or more of the model, name, purpose, use environment conditions, and external dimensions; the state characteristic data of the operator entering and leaving the factory also includes one or more of the name, education background, professional title, unit, department, age, and gender; The collecting of state characteristic data of physical equipment entering and leaving the factory, and the obtaining of state characteristic data of operators entering and leaving the factory include the following steps: S11. Establishing a state feature data set A = [a1, ..., a i , …, a n ], i = 1, 2, 3, ... n, i represents the status characteristic data number of different types of entity equipment, a i Represents the status characteristic data of physical equipment inside different types of factories; Establish the state characteristic data set B = [b1, ..., b j , …, b m ], j = 1, 2, 3, ... m, j represents the operator status characteristic data number of different individuals, b j Represents the status characteristic data of different individual operators; The steps of obtaining the current factory internal physical equipment layout image and extracting the original factory internal physical equipment layout image and using an artificial neural network to identify the difference image features are as follows: S21, obtain and set the current factory internal physical equipment layout image as C 当前 , the original factory internal physical equipment layout image is C 原始 ; S22, using the DFP algorithm in the quasi-Newton algorithm to calculate C 当前 and C 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain C 1 当前 and C 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δ k and k Represents a vector sequence; Using multiple groups of C 1 当前 and C 1 原始 The data are used to train the artificial neural network to generate test data C 2 当前 and C 2 原始 , until the test data C generated by the artificial neural network 2 当前 and C 2 原始 Satisfy image recognition accuracy; S23, using the artificial neural network trained in S22 to identify and extract C 1 当前 With C 1 原始 The current difference image feature data C 差异 ; Similarly, the steps of obtaining the distinguishing image feature data of the current operator according to steps S21, S22, and S23 are as follows: S24, first obtain and set the current factory internal operator work station image as D 当前 The original factory internal operator workstation image is D 原始 ; S25, using the DFP algorithm in the quasi-Newton algorithm to 当前 and D 原始 Perform de-meaning, normalization, and whitening preprocessing to obtain D 1 当前 and D 1 原始 , the formula of the DFP algorithm in the quasi-Newton algorithm is as follows: Among them G k+1 represents the k+1th matrix, G k represents the k-th matrix, δ k and k Represents a vector sequence; Using multiple groups of D 1 当前 and D 1 原始 The data are used to train the artificial neural network to generate test data D 2 当前 and D 2 原始 , until the test data D generated by the artificial neural network 2 当前 and D 2 原始 Satisfy image recognition accuracy; S26, using the artificial neural network trained in S25 to identify and extract D 1 当前 With D 1 原始 The current operator distinguishing image feature data D 区别 ; The physical equipment in the physical equipment layout inside the factory is identified by using the current difference image feature data; the operator in the operator's work station inside the factory is identified by using the current operator difference image feature data; The physical equipment and operators identified by S4 are matched to obtain the corresponding physical equipment 3D model in the physical equipment state feature data in S1 and the corresponding human body 3D model in the operator state feature data, and the operation steps of matching them to the digital virtual space factory layout are as follows: S51, using an artificial neural network based on the current difference image feature data C in S23 差异 Traverse the state feature data set A of the physical equipment entering and leaving the factory to identify the physical equipment data a of the specific changes in the physical equipment layout inside the factory i ; S52, artificial neural network calls the internal entity device data a of set A. i The 3D model of the physical equipment is created and matched to the digital virtual space factory layout; S53, using an artificial neural network to distinguish image feature data D according to the current operator in S26 区别 Traverse the state feature data set B of operators entering and leaving the factory to identify the operator data b of the specific changes in the operator's work station inside the factory j ; S54, artificial neural network calls set B internal operator data b j The corresponding 3D human body model is matched to the digital virtual space factory layout; Based on the physical equipment 3D model and human body 3D model matched to the digital virtual space factory layout, combined with sensors, the operation steps of obtaining the physical equipment operation status parameters and operator physiological parameters in the current factory layout in real time online are as follows: S61. Establish the factory physical equipment operation status parameter matrix q=1,2,3,…t,z=1,2,3,…r,q represents the number of different types of operation status parameters of physical equipment, z represents the number of samples of different types of operation status parameters of physical equipment, e qz Indicates different types of operating status parameters of physical equipment with different numbers of samples; Establish the operator's physiological parameter matrix F = [f 1x , …, f wx , …, f sx ], w=1,2,3,…s, x=1,2,3,…v, w represents the number of different types of physiological parameters of the operator's operation, x represents the number of samples of different types of physiological parameters of the operator's operation, f wx Indicates different types of physiological parameters of operators operating with different numbers of samples.

2. According to claim 1, a smart factory management method based on digital twin technology is characterized by: The operation steps of calculating and predicting the probability of failure of physical equipment in the factory and the probability of emergencies in the operation of operators are as follows: S71. Based on the factory physical equipment operation status parameter matrix E in S61, solve the probability of abnormal operation status parameters in different types of operation status parameters of physical equipment respectively. where e q Indicates different types of operating status parameters of physical equipment; like It indicates the probability of abnormal operation status parameter of entity device q type Probability of exceeding the safety threshold of the physical device q type operating status parameter like It indicates the probability of abnormal operation status parameter of entity device q type The probability of not exceeding the safety threshold of the physical device q type operating status parameter S72, based on the operator's physiological parameter matrix F in S61, respectively solve the probability of abnormal physiological parameters in different types of operator's physiological parameters where f w Indicates different types of physiological parameters of operators’ operations; like It indicates the probability of abnormal physiological parameters of operator operation type w Probability of exceeding the safety threshold of the operator's physiological parameters of type w like It indicates the probability of abnormal physiological parameters of operator operation type w The probability of not exceeding the safety threshold of the operator's type w physiological parameters S73, counting the probability and value of different types of abnormal operation status parameters of physical equipment exceeding the safety threshold and The probability and value of operators’ different types of abnormal physiological parameters exceeding the safety threshold are calculated. and S74, when Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device. The physical device is likely to fail if it continues to operate. Indicates the probability of all abnormal operation status parameters of the physical device exceeding the safety threshold and the probability threshold of safe operation of the physical device not exceeding the threshold. The physical device has a small probability of failure when it continues to operate; when It indicates the probability of all abnormal physiological parameters of the operator exceeding the safety threshold and the probability threshold of the operator's safe operation. If the operator continues to work, there is a high probability that an emergency will occur. It indicates the probability of all abnormal physiological parameters of the operator exceeding the safety threshold and the probability threshold of safe operation of the operator. If the operator continues to work, there is a small probability that an emergency will occur.

3. According to claim 2, a smart factory management method based on digital twin technology is characterized in that: The following are the steps to take when a physical device fails or an operator encounters an emergency: S81, according to the judgment result of S74, when If the physical equipment continues to operate, it is likely to fail, and the warning prompts the management personnel to carry out maintenance on the corresponding physical equipment. The physical equipment will continue to operate with a small probability of failure, without the need for early warning to alert management personnel; when If the operator continues to work, there is a high probability that an emergency will occur. The early warning prompts the management to conduct a physical examination of the operator. There is a small chance that an emergency may occur while the operator continues to work, and there is no need to warn the management staff.

4. A smart factory management system, used to implement the smart factory management method based on digital twin technology according to any one of claims 1 to 3, characterized in that: The system includes a physical equipment and physical model update matching module, a physical equipment and personnel operation parameterized risk prediction module; The physical equipment and physical model update matching module includes a factory equipment and personnel entry and exit data acquisition module, a factory internal physical equipment layout recognition module, a virtual space physical model matching module, and a physical equipment and physical model communication response module; The factory equipment and personnel entry and exit data collection module uses information entry equipment to collect status characteristic data of physical equipment entering and leaving the factory, as well as to obtain status characteristic data of operators entering and leaving the factory; The module for identifying the layout of physical equipment inside the factory uses a camera lens in conjunction with an artificial neural network algorithm to collect images of the layout of physical equipment inside the factory and images of the workstations of operators inside the factory, and compares these images with the original images of the layout of physical equipment inside the factory and the original images of the workstations of operators inside the factory to identify the corresponding difference image feature numbers and the distinguishing image feature data of the current operator; and simultaneously uses the difference image feature numbers and the distinguishing image feature data of the current operator to extract the physical equipment in the layout of physical equipment inside the factory and the operators in the workstations of operators inside the factory; The virtual space physical model matching module uses the physical equipment and the operator to retrieve the physical equipment 3D model data and the human body 3D model data corresponding to the state feature data of the physical equipment entering and exiting the factory and the state feature data of the operator entering and exiting the factory, and matches the physical equipment 3D model data to the physical equipment layout in the current factory in the digitally constructed virtual space factory, and matches the human body 3D model data to the operator's work station in the current factory; The physical device and physical model communication response module uses the Internet of Things network to enable the physical device connection response after the three-dimensional model of the physical device in the virtual space and the physical device connection response after the real factory layout changes; The physical equipment and personnel operation parameterized risk prediction module includes a factory physical equipment operation status parameter acquisition module, an operator operation physiological parameter acquisition module, a physical equipment failure probability and operator emergency probability operation module, and an equipment failure and operator emergency risk warning module; A module for obtaining the operating status parameters of the plant's physical equipment, using sensors to obtain the operating status parameters of the plant's physical equipment online in real time; The operator's physiological parameters acquisition module uses smart wearable devices and sensors to measure the operator's physiological parameters online; The physical equipment failure probability and operator emergency probability operation module uses the factory physical equipment operation status parameters and the operator's physiological parameters combined with the probability algorithm to calculate and predict the probability of failure of the factory's internal physical equipment operation and the probability of emergency events in the operator's operation; The equipment failure and operator emergency risk warning module uses information push or alarms to warn of the high probability of operational failures in physical equipment and the high probability of emergencies in operators' continued operations.

5. A smart factory management system according to claim 4, characterized in that: The physical equipment and physical model update matching module and the physical equipment and personnel operation parameterized risk prediction module are connected through Internet communication, and the physical equipment and physical model communication response module is respectively connected with the factory physical equipment operation status parameter acquisition module and the operator operation physiological parameter acquisition module through Internet communication.

Images