Big data processing method and system based on industrial internet platform

Abstract

The present invention relates to the field of industrial Internet. Provided are a big data processing method and system based on an industrial Internet platform. The system comprises: a data acquisition module, which comprises a plurality of device data acquisition units and a plurality of region data acquisition units; a data transmission module, which comprises a plurality of edge servers, a virtual host running on each edge server and being used for receiving state data of an industrial device acquired by the corresponding at least one device data acquisition unit and / or state data of an operator in a target region acquired by the at least one region data acquisition unit, preprocessing the data, generating a device heartbeat data packet and / or a region heartbeat data packet, and uploading same to a data processing cloud platform; and a data processing module, which comprises the data processing cloud platform and is used for analyzing data so as to determine the states of the plurality of industrial devices and the states of the operators in the plurality of target regions. The present invention has the advantages of improving the efficiency and quality of monitoring industrial devices and operators.

Description

A big data processing method and system based on industrial Internet platform Technical Field

[0001] The present invention relates to the field of industrial Internet, and in particular to a big data processing method and system based on an industrial Internet platform. Background Art

[0002] In recent years, a new round of scientific and technological revolution and industrial transformation has rapidly developed. The internet has rapidly expanded from the consumer sector to the production sector, and the industrial economy has deepened its development from digitalization to networking and intelligentization. The innovative development of the internet and the new industrial revolution have formed a historic intersection, giving rise to the Industrial Internet. The Industrial Internet (IIoT) is a new type of infrastructure, application model, and industrial ecosystem that deeply integrates next-generation information and communication technologies with the industrial economy. By comprehensively connecting people, machines, objects, and systems, it has established a new manufacturing and service system covering the entire industrial chain and value chain. It provides a path for the digitalization, networking, and intelligent development of industry and even industries, and is a key cornerstone of the Fourth Industrial Revolution.

[0003] Conventional industrial monitoring systems are centered around a PLC, primarily focused on industrial control functions. Several detection modules are connected to the PLC processing module for signal acquisition and data analysis. In existing PLC-based industrial monitoring systems, upgrading to the Industrial Internet requires deploying gateways at each connection node. Furthermore, corresponding data acquisition boards are required for different data signals. This is the commonly used "PLC + data acquisition board + gateway" industrial monitoring system in existing technology. This system suffers from poor compatibility and signal synchronization. Furthermore, it is difficult to build multimedia detection platforms in this traditional industrial monitoring system. For example, video and audio signals are poorly coordinated, making cross-analysis and processing difficult. This traditional industrial monitoring system is costly, bulky, and complex. Furthermore, it suffers from limited functionality, low data transmission rates, and poor signal delay synchronization, making it difficult to adapt to today's increasingly intelligent industrial landscape.

[0004] Therefore, it is necessary to provide a big data processing method and system based on the industrial Internet platform to improve the efficiency and quality of industrial equipment and operator monitoring.

[0005] Summary of the Invention

[0006] One of the embodiments of this specification provides a big data processing system based on an industrial Internet platform, including: a data acquisition module, including multiple device data acquisition units and multiple regional data acquisition units, wherein the multiple device data acquisition units are used to collect status data of multiple industrial devices, and the multiple regional data acquisition units are used to collect status data of workers in multiple target areas; a data transmission module, including multiple edge servers, each of which runs at least one virtual host, each of which corresponds to at least one device data acquisition unit and / or at least one regional data acquisition unit, and the virtual host is used to receive the status data of industrial devices collected by the corresponding at least one device data acquisition unit and / or the status data of workers in the target area collected by the at least one regional data acquisition unit, perform data preprocessing, generate device heartbeat data packets corresponding to the industrial devices and / or regional heartbeat data packets corresponding to the target areas, and upload the device heartbeat data packets and / or the regional heartbeat data packets to a data processing cloud platform; a data processing module, including the data processing cloud platform, is used to perform data analysis based on the device heartbeat data packets and / or the regional heartbeat data packets to determine the status of the multiple industrial devices and the status of workers in the multiple target areas.

[0007] In some embodiments, the data transmission module also includes multiple server monitoring units, one of the server monitoring units corresponds to one of the edge servers, the server monitoring unit includes a server monitoring component and at least one virtual host monitoring component, one of the virtual host monitoring components corresponds to a virtual host of the edge server corresponding to the server monitoring unit, wherein the server monitoring component is used to obtain the status data of the corresponding edge server, and the virtual host monitoring component is used to obtain the status data of the corresponding virtual host; the data transmission module also includes a virtual host scheduling server, which is used to dynamically adjust the correspondence between the multiple virtual hosts and the multiple device data acquisition units and / or multiple regional data acquisition units based on the status data of the edge server and the status data of the virtual host.

[0008] In some embodiments, the status data of the industrial equipment includes multiple types of status information of the industrial equipment, and the status data of the operators in the target area includes image information of the target area; the virtual host performs data preprocessing including: for each corresponding device data acquisition unit, based on the multiple types of status information of the industrial equipment collected by the device data acquisition unit during the current monitoring cycle, generating an information matrix of the industrial equipment, denoising the information matrix of the industrial equipment through a denoising model to generate a denoised information matrix of the industrial equipment, and completing the denoised information matrix of the industrial equipment through a completion model to generate a completed information matrix of the industrial equipment; for each corresponding area data acquisition unit, filtering the image information of the target area collected by the area data acquisition unit based on a preset condition set through an image screening model.

[0009] In some embodiments, the data preprocessing of the virtual host also includes: determining the failure probability of the industrial equipment based on the completed information matrix of the industrial equipment, and when the failure probability of the industrial equipment is greater than a preset failure probability threshold, taking the industrial equipment as the target industrial equipment; determining the optimal data acquisition frequency of the device data acquisition unit corresponding to the target industrial equipment according to the failure probability of the target industrial equipment; predicting the future load of the virtual host through a load prediction model based on the optimal data acquisition frequency of the device data acquisition unit corresponding to the target industrial equipment, and uploading the predicted future load of the virtual host to the virtual host scheduling server; the virtual host scheduling server dynamically adjusts the correspondence between multiple virtual hosts and the multiple device data acquisition units and / or multiple regional data acquisition units based on the status data of the edge server and the status data of the virtual host, including: taking the virtual host that uploads the future load as the target virtual host; determining the virtual host to be scheduled from the multiple virtual hosts based on the status data of each edge server, the status data of each virtual host and the future load of the target virtual host, and scheduling the virtual host to be scheduled.

[0010] In some embodiments, the virtual host scheduling server determines a virtual host to be scheduled from a plurality of the virtual hosts based on the status data of each of the edge servers, the status data of each of the virtual hosts, and the future load of the virtual host, including: determining a target edge server based on the status data of each of the edge servers, and all virtual hosts running on the target edge server are used as virtual hosts to be scheduled; and judging whether the target virtual host is the virtual host to be scheduled based on the future load of the target virtual host and the status data of the target virtual host.

[0011] In some embodiments, the virtual host scheduling server schedules the virtual hosts to be scheduled, including: for each of the virtual hosts to be scheduled, determining the optimal edge server from the multiple edge servers except the target edge server and the edge server where the virtual host to be scheduled is located, and migrating the virtual host to be scheduled to the optimal edge server; the migrated virtual host is also used to send the optimal data collection frequency to the equipment data collection unit corresponding to the target industrial equipment, and the equipment data collection unit is also used to collect status data of the target industrial equipment according to the optimal data collection frequency.

[0012] In some embodiments, the virtual host uploads the device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform, including: encrypting the device heartbeat data packet and / or the regional heartbeat data packet based on the correspondence between the multiple virtual hosts and the multiple device data acquisition units and the multiple regional data acquisition units and the relevant information of the virtual hosts, and uploading the encrypted device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform.

[0013] In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and / or the area heartbeat data packet to determine the status of the multiple industrial equipment and the status of the operators in the multiple target areas, including: establishing a device relationship map, wherein the device relationship map is used to record the association relationship between the multiple industrial equipment; for each of the industrial equipment, determining at least one associated industrial equipment based on the device relationship map, and determining the status of the industrial equipment based on the device heartbeat data packet of the industrial equipment and the device heartbeat data packet of the at least one associated industrial equipment.

[0014] In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial equipment and the status of the operators in the multiple target areas, including: establishing a regional relationship map, wherein the regional relationship map is used to record the association relationship between the multiple target areas; for each target area, determining at least one associated target area based on the regional relationship map, and determining the status of the operators in the target area based on the regional heartbeat data packet of the target area and the regional heartbeat data packet of the associated at least one target area.

[0015] One of the embodiments of this specification provides a big data processing method based on an industrial Internet platform, including: collecting status data of multiple industrial equipment; collecting status data of workers in multiple target areas; a virtual host receives the status data of at least one corresponding industrial equipment and / or the status data of workers in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial equipment and / or a regional heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and / or the regional heartbeat data packet to a data processing cloud platform; the data processing cloud platform performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial equipment and the status of the workers in the multiple target areas.

[0016] Compared with the existing technology, the big data processing method and system based on the industrial Internet platform provided in this specification has at least the following beneficial effects:

[0017] 1. By setting up multiple device data collection units and multiple regional data collection units, standardized collection of industrial equipment status data and operator status data is achieved. Through multiple edge servers and running virtual hosts on the edge servers, synchronous preprocessing of multiple industrial equipment status data and multiple operator status data is achieved, improving the timeliness of data preprocessing. Based on the data processing cloud platform, comprehensive data analysis of device heartbeat data packets and / or regional heartbeat data packets is performed to more accurately determine the status of multiple industrial equipment and the status of operators in multiple target areas. Automatic data collection, automatic processing and data fusion analysis are achieved, improving the efficiency and quality of industrial equipment and operator monitoring.

[0018] 2. Adjust the optimal data collection frequency of the corresponding equipment data collection unit based on the failure probability of the industrial equipment. This ensures that the status data of the industrial equipment subsequently collected by the equipment data collection unit is more real-time and has a larger data volume. This allows the data processing cloud platform to more accurately and immediately determine whether the industrial equipment has failed.

[0019] 3. Dynamically adjust the correspondence between multiple virtual hosts and the multiple device data acquisition units and / or multiple regional data acquisition units to effectively ensure the effective and real-time performance of data preprocessing;

[0020] 4. Based on the correspondence between multiple virtual hosts and multiple device data acquisition units and multiple regional data acquisition units and the relevant information of the virtual hosts, the device heartbeat data packets and / or regional heartbeat data packets are encrypted, which can effectively prevent the leakage of the device heartbeat data packets and / or regional heartbeat data packets;

[0021] 5. Determine at least one associated industrial device based on the device relationship map, determine the status of the industrial device based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of at least one associated industrial device, and realize the fusion of the device heartbeat data packets of multiple industrial devices, so that the determined status of the industrial equipment is more accurate, and determine at least one associated target area based on the area relationship map, determine the status of the operator in the target area based on the area heartbeat data packet of the target area and the area heartbeat data packet of at least one associated target area, and realize the fusion of the area heartbeat data packets of multiple target areas, so that the determined status of the operator in the target area is more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] This specification will be further described in the form of exemplary embodiments, which will be described in detail with reference to the accompanying drawings. These embodiments are not limiting, and in these embodiments, the same numbers represent the same structures, wherein:

[0023] FIG1 is a module diagram of a big data processing system based on an industrial Internet platform according to some embodiments of this specification;

[0024] FIG2 is a flow chart of a method for processing big data based on an industrial Internet platform according to some embodiments of this specification;

[0025] FIG3 is a schematic diagram of a flow chart of data preprocessing performed by a virtual host according to some embodiments of this specification. DETAILED DESCRIPTION

[0026] To more clearly illustrate the technical solutions of the embodiments of this specification, the following briefly describes the drawings required for describing the embodiments. Obviously, the drawings described below are merely examples or embodiments of this specification. Those skilled in the art can apply this specification to other similar scenarios based on these drawings without inventive effort. Unless otherwise apparent from the context or otherwise noted, the same reference numerals in the figures represent the same structure or operation.

[0027] It should be understood that the terms "system," "device," "unit," and / or "module" used herein are a method for distinguishing different components, elements, parts, portions, or assemblies at different levels. However, if other terms can achieve the same purpose, the terms may be replaced by other expressions.

[0028] As used in this specification and claims, unless the context clearly indicates otherwise, the words "a," "an," "an," and / or "the" do not refer to the singular but also include the plural. Generally speaking, the terms "comprises" and "include" only indicate the inclusion of the steps and elements specifically identified, and these steps and elements do not constitute an exclusive list. A method or apparatus may also include other steps or elements.

[0029] Flowcharts are used throughout this specification to illustrate the operations performed by systems according to embodiments of this specification. It should be understood that preceding or following operations do not necessarily need to be performed in exact order. Instead, the steps may be processed in reverse order or simultaneously. Furthermore, other operations may be added to these processes, or one or more operations may be removed from these processes.

[0030] The industrial Internet platform mainly includes three core layers: edge, platform (industrial PaaS), and application.

[0031] (1) Edge: This includes data collection, storage, and integration of heterogeneous data and pre-processing for cloud platforms. The edge constitutes the data foundation of the Internet platform architecture.

[0032] (2) Platform: Based on Industrial PaaS, the open cloud operating system of the industrial platform realizes functions such as big data processing, data analysis, and microservices. It integrates data management capabilities and application development technology to help customers build customized apps.

[0033] (3) Application: Based on the platform, design personalized industrial SaaS and APP for specific customers and specific scenarios to realize the ultimate value of the industrial Internet platform.

[0034] Figure 1 is a schematic diagram of a module of a big data processing system based on an industrial internet platform according to some embodiments of this specification. As shown in Figure 1 , a big data processing system based on an industrial internet platform may include a data acquisition module, a data transmission module, and a data processing module. Each module is described in detail below.

[0035] The data acquisition module may include multiple equipment data acquisition units and multiple area data acquisition units, wherein the multiple equipment data acquisition units are used to collect status data of multiple industrial equipment, and the multiple area data acquisition units are used to collect status data of operators in multiple target areas.

[0036] Specifically, the equipment data acquisition unit may include various devices for acquiring status data of industrial equipment. For example, these devices may be used to acquire various required information about industrial equipment, such as acoustic, optical, thermal, electrical, mechanical, chemical, biological, and positional information. By way of example only, the equipment data acquisition unit may include at least temperature and humidity sensors, sound acquisition devices, vibration sensors, current sensors, voltage sensors, gas monitoring devices, and locators. In other words, the status data of industrial equipment includes various types of status information about industrial equipment.

[0037] The area data acquisition unit may include an image acquisition device, and the image acquisition device is used to acquire image information of the target area.

[0038] The data transmission module may include multiple edge servers, each edge server running at least one virtual host, each virtual host corresponding to at least one device data acquisition unit and / or at least one regional data acquisition unit, the virtual host is used to receive the status data of industrial equipment collected by the corresponding at least one device data acquisition unit and / or the status data of the operating personnel in the target area collected by at least one regional data acquisition unit, perform data preprocessing, generate device heartbeat data packets corresponding to the industrial equipment and / or regional heartbeat data packets corresponding to the target area, and upload the device heartbeat data packets and / or regional heartbeat data packets to the data processing cloud platform.

[0039] The equipment data collection unit and the area data collection unit can use a wireless network to send the collected status data of the industrial equipment and / or the status data of the operators in the target area to the corresponding virtual host.

[0040] Virtual hosting uses specialized hardware and software technologies to partition a physical server into multiple logical storage units. Each logical unit lacks a physical entity, but each can function on the network like a real physical host, with a separate IP address (or shared IP address), independent domain name, and complete Internet server functionality (supporting WWW, FTP, email, etc.).

[0041] FIG3 is a flow chart of data preprocessing performed by a virtual host according to some embodiments of this specification. In some embodiments, data preprocessing performed by the virtual host includes:

[0042] For each corresponding device data acquisition unit, based on multiple types of status information of the industrial equipment collected by the device data acquisition unit within a current monitoring cycle (also referred to as a "first current monitoring cycle"), an information matrix of the industrial equipment is generated, the information matrix of the industrial equipment is denoised using a denoising model to generate a denoised information matrix of the industrial equipment, and the denoised information matrix of the industrial equipment is completed using a completion model to generate a completed information matrix of the industrial equipment, wherein the denoising model may be an LSTM (Long Short-Term Memory) model, and a row vector of the information matrix of the industrial equipment is composed of multiple types of status information acquired by the device data acquisition unit at a time point within the current monitoring cycle, and one element of the row vector represents one type of status information;

[0043] For each corresponding regional data acquisition unit, the image information of the target area collected by the regional data acquisition unit is screened using an image screening model based on a preset set of conditions. Specifically, the preset set of conditions may include image quality requirements (e.g., exposure, color, clarity, noise, etc.), target object requirements, etc. For example, if the clarity of the collected image information of the target area is low or the collected image information of the target area does not contain a human body, the image information may be screened out as invalid image information.

[0044] As shown in FIG3 , in some embodiments, the virtual host performing data preprocessing further includes:

[0045] Based on the completed information matrix of the industrial equipment, the failure probability of the industrial equipment is determined. When the failure probability of the industrial equipment is greater than a preset failure probability threshold, the industrial equipment is regarded as a target industrial equipment.

[0046] Determine the optimal data collection frequency of the device data collection unit corresponding to the target industrial equipment based on the failure probability of the target industrial equipment;

[0047] Based on the optimal data collection frequency of the device data collection unit corresponding to the target industrial equipment, the future load of the virtual host is predicted through the load prediction model, and the predicted future load of the virtual host is uploaded to the virtual host scheduling server.

[0048] Specifically, the virtual host can determine the failure probability of the industrial equipment based on the completed information matrix of the industrial equipment using a fault prediction model. The virtual host can also determine the optimal data collection frequency of the device data collection unit corresponding to the target industrial equipment using a frequency determination model based on the failure probability of the target industrial equipment, the performance of the device data collection unit corresponding to the target industrial equipment, and the maximum computing power constraint. It is understood that the greater the failure probability of the target industrial equipment, the higher the optimal data collection frequency of the device data collection unit corresponding to the target industrial equipment. The better the performance of the device data collection unit corresponding to the target industrial equipment and / or the higher the maximum computing power constraint, the higher the maximum value of the optimal data collection frequency of the device data collection unit corresponding to the target industrial equipment. The virtual host can use a load prediction model to predict the amount of industrial equipment status data collected by the device data collection unit corresponding to the target industrial equipment at the optimal data collection frequency within a monitoring cycle, thereby determining the future load of the virtual host at the optimal data collection frequency within a monitoring cycle. Among them, the fault prediction model, frequency determination model and load prediction model can be machine learning models such as artificial neural network (ANN) model, recurrent neural network (RNN) model, long short-term memory network (LSTM) model, bidirectional recurrent neural network (BRNN) model, etc.

[0049] In some embodiments, the data transmission module also includes multiple server monitoring units, one server monitoring unit corresponds to one edge server, the server monitoring unit includes a server monitoring component and at least one virtual host monitoring component, one virtual host monitoring component corresponds to a virtual host of the edge server corresponding to the server monitoring unit, wherein the server monitoring component is used to obtain the status data of the corresponding edge server, and the virtual host monitoring component is used to obtain the status data of the corresponding virtual host.

[0050] In some embodiments, the data transmission module also includes a virtual host scheduling server for dynamically adjusting the correspondence between multiple virtual hosts and multiple device data collection units and / or multiple regional data collection units based on the status data of the edge server and the status data of the virtual host.

[0051] Specifically, the server monitoring component may include various devices for acquiring edge server status information, such as temperature and humidity sensors, sound collection devices, vibration sensors, current sensors, and voltage sensors. The server monitoring component may transmit the collected edge server status data to the virtual host scheduling server via a wireless network. The virtual host monitoring component may utilize software programs to acquire virtual host status data, such as CPU utilization, memory utilization, disk utilization, network utilization, and service response time.

[0052] In some embodiments, the virtual host scheduling server may pre-store trained image screening models and fault prediction models, frequency determination models, and load prediction models corresponding to different industrial devices. For example, the virtual host scheduling server may pre-store the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device A, the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device B, and the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device C. Once the correspondence between the virtual hosts and multiple device data acquisition units and multiple regional data acquisition units is determined, the virtual host scheduling server may distribute the corresponding models to the virtual hosts. For example, if virtual host 1 corresponds to the device data acquisition unit corresponding to industrial device A and the regional data acquisition unit corresponding to target region 1, and virtual host 2 corresponds to the device data acquisition unit corresponding to industrial device B and the device data acquisition unit corresponding to industrial device C, the virtual host scheduling server may distribute the image screening model and the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device A to virtual host 1, and distribute the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device B and the fault prediction model, frequency determination model, and load prediction model corresponding to industrial device C to virtual host 2.

[0053] The virtual host scheduling server dynamically adjusts the correspondence between multiple virtual hosts and multiple device data collection units and / or multiple regional data collection units based on the status data of the edge server and the status data of the virtual host, including:

[0054] The virtual host where future loads will be uploaded will be used as the target virtual host;

[0055] Based on the status data of each edge server, the status data of each virtual host and the future load of the target virtual host, a virtual host to be scheduled is determined from multiple virtual hosts, and the virtual host to be scheduled is scheduled.

[0056] In some embodiments, the virtual host scheduling server determines a virtual host to be scheduled from a plurality of virtual hosts based on the status data of each edge server, the status data of each virtual host, and the future load of the virtual host, including:

[0057] Based on the status data of each edge server, a target edge server is determined, and all virtual hosts running on the target edge server are used as virtual hosts to be scheduled;

[0058] Based on the future load of the target virtual host and the status data of the target virtual host, it is determined whether the target virtual host is a virtual host to be scheduled.

[0059] Specifically, for each edge server, the virtual host scheduling server can generate a server status information matrix corresponding to the edge server based on the status data of the edge server of the industrial equipment collected by the corresponding server monitoring component during the current monitoring cycle (also referred to as the "second current monitoring cycle"). The row vector of the server status information matrix is ​​composed of status information of multiple types of edge servers obtained by the equipment data acquisition unit at a time point during the second current monitoring cycle. An element of the row vector represents a type of status information, and a fault prediction model is used to determine whether the edge server is currently faulty based on the server status information matrix. If it is determined that the edge server is not currently faulty, the data prediction model is used to predict the server status information matrix corresponding to the next second current monitoring cycle based on the server status information matrix corresponding to the second current monitoring cycle, and based on the predicted server status information matrix corresponding to the next second current monitoring cycle, it is determined whether the edge server may fail in the future. When it is determined that the edge server is currently faulty or it is predicted that the edge server may fail in the future, the edge server is used as the target edge server, and all virtual hosts running on the target edge server are used as virtual hosts to be scheduled (also referred to as "first type of virtual hosts to be scheduled").

[0060] For each target virtual host, the virtual host scheduling server can predict whether the target virtual host can bear the future load based on the status data of the target virtual host and the remaining computer resources and future load of the edge server where the target virtual host is located. If it is predicted that the target virtual host cannot bear the future load, the target virtual host will be treated as a virtual host to be scheduled (also called a "second type of virtual host to be scheduled").

[0061] In some embodiments, the virtual host scheduling server schedules the virtual host to be scheduled, including:

[0062] For each virtual host to be scheduled, determine an optimal edge server from multiple edge servers excluding the target edge server and the edge server where the virtual host to be scheduled is located (also referred to as candidate edge servers), and migrate the virtual host to be scheduled to the optimal edge server;

[0063] The migrated virtual host is further used to send the optimal data collection frequency to the device data collection unit corresponding to the target industrial device, and the device data collection unit is further used to collect status data of the target industrial device according to the optimal data collection frequency.

[0064] Specifically, for the first type of virtual hosts to be scheduled, the virtual host scheduling server can determine the optimal edge server from the non-target edge servers based on the load information of the first type of virtual hosts to be scheduled, the remaining computer resource information of the non-target edge server, the device data collection unit corresponding to the first type of virtual hosts to be scheduled, and the communication distance between the regional data collection unit and the non-target edge server, add a virtual host to the optimal edge server, and migrate the first type of virtual hosts to be scheduled to the newly added virtual host in the optimal edge server through hot migration, cold migration or storage migration.

[0065] For example, the virtual host scheduling server may determine the priority value of the non-target edge server based on the load information of the first type of virtual hosts to be scheduled, the remaining computer resource information of the non-target edge server, and the communication distance between the device data collection unit and the regional data collection unit corresponding to the first type of virtual hosts to be scheduled and the non-target edge server using the following formula:

[0066] Among them, P (i,j) is the priority value of the i-th non-target edge server as the host of the j-th first-class virtual host to be scheduled, R (residue,i) is the normalized remaining computer resources of the i-th non-target edge server, R (reality,j) is the normalized load of the jth first-class virtual host to be scheduled, L (n,i) is the communication distance between the nth device data acquisition unit corresponding to the jth first-class virtual host to be scheduled and the i-th non-target edge server, L (m,i) is the communication distance between the mth regional data collection unit corresponding to the jth first-class virtual host to be scheduled and the i-th non-target edge server, N is the total number of device data collection units corresponding to the jth first-class virtual host to be scheduled, M is the total number of regional data collection units corresponding to the jth first-class virtual host to be scheduled, a 11 、a 12 All are preset weights, and W1 is a preset parameter.

[0067] The non-target edge server with the largest priority value is selected as the optimal edge server.

[0068] For each second-category virtual host to be scheduled, the virtual host scheduling server may determine a scheduling priority value for each candidate edge server based on the predicted future load of the second-category virtual host to be scheduled, the current load of the candidate edge server, the communication distance between the device data collection unit and the area data collection unit corresponding to the second-category virtual host to be scheduled and the candidate edge server, and the correlation between the virtual host running on the candidate edge server and the second-category virtual host to be scheduled. For example,

[0069] For example, the scheduling priority value of each candidate edge server can be determined based on the predicted future load of the second type of virtual hosts to be scheduled, the remaining computer resource information of the candidate edge server, the communication distance between the device data collection unit and the regional data collection unit corresponding to the second type of virtual hosts to be scheduled and the candidate edge server, and the correlation between the virtual hosts running on the candidate edge server and the second type of virtual hosts to be scheduled according to the following formula:

[0070] Among them, P (q,p) is the scheduling priority value of the pth candidate edge server as the host of the qth second-class virtual host to be scheduled, R (residue,p) is the normalized remaining computer resources of the p-th candidate edge server, R (prediction,q) is the normalized predicted future load of the qth second-class virtual host to be scheduled, L (c,q) is the communication distance between the cth device data acquisition unit corresponding to the qth second-class virtual host to be scheduled and the pth candidate edge server, L (d,q) is the communication distance between the dth regional data collection unit corresponding to the qth second-class virtual host to be scheduled and the pth candidate edge server, C is the total number of device data collection units corresponding to the qth second-class virtual host to be scheduled, D is the total number of regional data collection units corresponding to the qth second-class virtual host to be scheduled, a 21 、a 22 and a 23 are all preset weights, W2 is the preset parameter, C (h,q) is the correlation between the hth virtual host running on the pth candidate edge server and the qth second-class virtual host to be scheduled, C (c,e) is the correlation between the cth device data collection unit corresponding to the qth second-class virtual host to be scheduled and the eth device data collection unit corresponding to the hth virtual host running on the pth candidate edge server, E is the total number of device data collection units corresponding to the hth virtual host running on the pth candidate edge server, C (d,f)is the correlation between the dth regional data collection unit corresponding to the qth second-class virtual host to be scheduled and the fth regional data collection unit corresponding to the hth virtual host running on the pth candidate edge server, F is the total number of regional data collection units corresponding to the hth virtual host running on the pth candidate edge server, the correlation between any two device data collection units can be determined based on the correlation between the industrial devices corresponding to the two device data collection units, and the correlation between any two industrial devices can be determined in any way, for example, based on the correlation coefficient between any two industrial devices. For another example, it can be determined manually. The correlation between any two regional data collection units can be determined based on the correlation between the target areas corresponding to the two regional data collection units, and the correlation between any two target areas can be determined in any way, for example, based on the distance between the two target areas.

[0071] In some embodiments, the virtual host scheduling server can establish a scheduling scheme generation model, wherein the goal of the scheduling scheme generation model is to minimize the sum of the loads of the migrated virtual hosts, and the input of the scheduling scheme generation model includes the predicted future load of each second-class virtual host to be scheduled, the status data of the edge server, and the scheduling priority value of each candidate edge server relative to each second-class virtual host to be scheduled. The output of the scheduling scheme generation model includes the optimal virtual host scheduling scheme, wherein the optimal virtual host scheduling scheme includes at least the optimal edge server corresponding to each second-class virtual host to be scheduled. The scheduling scheme generation model can be a machine learning model such as an artificial neural network (ANN) model, a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, a bidirectional recurrent neural network (BRNN) model, etc.

[0072] In some embodiments, the virtual host uploads the device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform, including:

[0073] Based on the correspondence between multiple virtual hosts and multiple device data acquisition units and multiple regional data acquisition units and the relevant information of the virtual hosts, the device heartbeat data packet and / or the regional heartbeat data packet is encrypted, and the encrypted device heartbeat data packet and / or the regional heartbeat data packet is uploaded to the data processing cloud platform.

[0074] Specifically, for each device data acquisition unit, the virtual host can generate a first identification pair based on the unique device identification of the device data acquisition unit (for example, the device ID of the corresponding industrial device) and the unique host identification of the virtual host (for example, the IP of the virtual host), and use the first identification pair to encrypt the device heartbeat data packet corresponding to the device data acquisition unit for the first time to generate a first encrypted device heartbeat data packet, and then use the private key corresponding to the virtual host to encrypt the first encrypted device heartbeat data packet for the second time to generate an encrypted device heartbeat data packet.

[0075] For each regional data acquisition unit, the virtual host can generate a second identification pair based on the unique regional identification of the regional data acquisition unit (for example, the regional ID of the corresponding target area) and the unique host identification of the virtual host (for example, the IP of the virtual host). Through the second identification pair, the regional heartbeat data packet corresponding to the regional data acquisition unit is encrypted for the first time to generate a regional heartbeat data packet after the first encryption. The regional heartbeat data packet after the first encryption is then encrypted for a second time using the private key corresponding to the virtual host to generate an encrypted regional heartbeat data packet.

[0076] The data processing module may include a data processing cloud platform for performing data analysis based on device heartbeat data packets and / or regional heartbeat data packets to determine the status of multiple industrial devices and the status of operators in multiple target areas.

[0077] In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of multiple industrial devices and the status of operators in multiple target areas, including:

[0078] Establishing a device relationship map, wherein the device relationship map is used to record the relationship between multiple industrial devices;

[0079] For each industrial device, at least one associated industrial device is determined based on the device relationship map, and the status of the industrial device is determined based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of at least one associated industrial device.

[0080] For example, the data processing cloud platform can determine the status of the industrial equipment based on the device heartbeat data packet of the industrial equipment and the device heartbeat data packet of at least one associated industrial device through a first status determination model, wherein the first status determination model can be an artificial neural network (ANN) model, a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, a bidirectional recurrent neural network (BRNN) model, and other machine learning models.

[0081] In some embodiments, the data processing module performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of multiple industrial devices and the status of operators in multiple target areas, including:

[0082] Establishing a regional relationship map, wherein the regional relationship map is used to record the association relationship between multiple target areas;

[0083] For each target area, at least one associated target area is determined based on the area relationship map, and the status of the operator in the target area is determined based on the area heartbeat data packet of the target area and the area heartbeat data packet of at least one associated target area.

[0084] For example, the data processing cloud platform can determine the status of the operating personnel in the target area based on the regional heartbeat data packet of the target area and the regional heartbeat data packet of at least one associated target area through a second state determination model, wherein the second state determination model can be an artificial neural network (ANN) model, a recurrent neural network (RNN) model, a long short-term memory network (LSTM) model, a bidirectional recurrent neural network (BRNN) model, and other machine learning models.

[0085] Figure 2 is a flow chart of a big data processing method based on an industrial Internet platform according to some embodiments of this specification. As shown in Figure 2, a big data processing method based on an industrial Internet platform may include the following steps.

[0086] Step 210, collecting status data of multiple industrial devices;

[0087] Step 220, collecting status data of workers in multiple target areas;

[0088] Step 230: The virtual host receives the status data of at least one corresponding industrial device and / or the status data of an operator in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial device and / or a regional heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform.

[0089] In step 240 , the data processing cloud platform performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the plurality of industrial devices and the status of the operators in the plurality of target areas.

[0090] In some embodiments, a big data processing method based on an industrial Internet platform can be executed by a big data processing system based on an industrial Internet platform. For more descriptions of a big data processing method based on an industrial Internet platform, please refer to the relevant descriptions of a big data processing system based on an industrial Internet platform, which will not be repeated here.

[0091] While the basic concepts have been described above, it will be apparent to those skilled in the art that the detailed disclosure is merely illustrative and does not limit this specification. Although not explicitly stated herein, various modifications, improvements, and revisions to this specification may be made by those skilled in the art. Such modifications, improvements, and revisions are suggested in this specification and remain within the spirit and scope of the exemplary embodiments of this specification.

[0092] This specification also uses specific terms to describe the embodiments of this specification. For example, "one embodiment," "an embodiment," and / or "some embodiments" refer to a feature, structure, or characteristic associated with at least one embodiment of this specification. Therefore, it should be emphasized and noted that references to "one embodiment," "an embodiment," or "an alternative embodiment" two or more times in different locations in this specification do not necessarily refer to the same embodiment. Furthermore, certain features, structures, or characteristics of one or more embodiments of this specification may be appropriately combined.

[0093] In addition, unless expressly stated in the claims, the order of the processing elements and sequences, the use of alphanumeric characters, or the use of other names described in this specification are not intended to limit the order of the processes and methods of this specification. Although the above disclosure discusses some of the invention embodiments currently considered useful through various examples, it should be understood that such details are for illustrative purposes only, and the appended claims are not limited to the disclosed embodiments. On the contrary, the claims are intended to cover all modifications and equivalent combinations that are consistent with the spirit and scope of the embodiments of this specification. For example, although the system components described above can be implemented by hardware devices, they can also be implemented only by software solutions, such as installing the described system on an existing server or mobile device.

[0094] Similarly, it should be noted that, in order to simplify the presentation of this specification and thus facilitate understanding of one or more embodiments of the invention, the foregoing descriptions of the embodiments of this specification sometimes combine multiple features into a single embodiment, figure, or description thereof. However, this disclosure method does not imply that the subject matter of this specification requires more features than those recited in the claims. In fact, an embodiment may have fewer features than all of the features of a single disclosed embodiment.

[0095] Finally, it should be understood that the embodiments described in this specification are intended only to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, by way of example and not limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly described and illustrated in this specification.

Claims

1. A big data processing system based on the industrial Internet platform, It is characterized in that include: A data collection module, comprising a plurality of equipment data collection units and a plurality of area data collection units, wherein the plurality of equipment data collection units are used to collect status data of a plurality of industrial equipment, and the plurality of area data collection units are used to collect status data of operators in a plurality of target areas; A data transmission module, comprising a plurality of edge servers, each of which runs at least one virtual host, each of which corresponds to at least one device data acquisition unit and / or at least one regional data acquisition unit, the virtual host being used to receive status data of industrial equipment collected by the corresponding at least one device data acquisition unit and / or status data of operators in a target area collected by at least one regional data acquisition unit, perform data preprocessing, generate a device heartbeat data packet corresponding to the industrial equipment and / or a regional heartbeat data packet corresponding to the target area, and upload the device heartbeat data packet and / or the regional heartbeat data packet to a data processing cloud platform; The data processing module includes the data processing cloud platform, which is used to perform data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial equipment and the status of the operators in the multiple target areas.

2. According to the big data processing system based on the industrial Internet platform according to claim 1, It is characterized in that The data transmission module further includes a plurality of server monitoring units, one of the server monitoring units corresponds to one of the edge servers, the server monitoring unit includes a server monitoring component and at least one virtual host monitoring component, one of the virtual host monitoring components corresponds to a virtual host of the edge server corresponding to the server monitoring unit, wherein the server monitoring component is used to obtain status data of the corresponding edge server, and the virtual host monitoring component is used to obtain status data of the corresponding virtual host; The data transmission module also includes a virtual host scheduling server, which is used to dynamically adjust the correspondence between the multiple virtual hosts and the multiple device data acquisition units and / or multiple regional data acquisition units based on the status data of the edge server and the status data of the virtual host.

3. According to the big data processing system based on the industrial Internet platform according to claim 2, It is characterized in that The state data of the industrial equipment includes various types of state information of the industrial equipment, and the state data of the operator in the target area includes image information of the target area; The data preprocessing performed by the virtual host includes: For each of the corresponding equipment data acquisition units, based on various types of status information of the industrial equipment collected by the equipment data acquisition unit in the current monitoring period, an information matrix of the industrial equipment is generated, the information matrix of the industrial equipment is denoised by a denoising model to generate an information matrix of the denoised industrial equipment, and the information matrix of the denoised industrial equipment is completed by a completion model to generate an information matrix of the completed industrial equipment; For each corresponding regional data acquisition unit, the image screening model is used to screen the region based on a preset condition set. The image information of the target area collected by the domain data acquisition unit is screened.

4. According to the big data processing system based on the industrial Internet platform according to claim 3, It is characterized in that The virtual host performs data preprocessing further comprising: Determine the failure probability of the industrial equipment based on the completed information matrix of the industrial equipment, and when the failure probability of the industrial equipment is greater than a preset failure probability threshold, use the industrial equipment as a target industrial equipment; Determining an optimal data collection frequency of a device data collection unit corresponding to the target industrial equipment according to the failure probability of the target industrial equipment; Based on the optimal data collection frequency of the device data collection unit corresponding to the target industrial device, predict the future load of the virtual host through a load prediction model, and upload the predicted future load of the virtual host to the virtual host scheduling server; The virtual host scheduling server dynamically adjusts the correspondence between the plurality of virtual hosts and the plurality of device data collection units and / or the plurality of regional data collection units based on the status data of the edge server and the status data of the virtual host, including: The virtual host where future loads will be uploaded will be used as the target virtual host; Based on the status data of each edge server, the status data of each virtual host and the future load of the target virtual host, a virtual host to be scheduled is determined from the multiple virtual hosts, and the virtual host to be scheduled is scheduled.

5. According to the big data processing system based on the industrial Internet platform according to claim 4, It is characterized in that The virtual host scheduling server determines a virtual host to be scheduled from the plurality of virtual hosts based on the status data of each edge server, the status data of each virtual host and the future load of the virtual host, including: Determine a target edge server based on the status data of each edge server, and all virtual hosts running on the target edge server are used as virtual hosts to be scheduled; Based on the future load of the target virtual host and the status data of the target virtual host, it is determined whether the target virtual host is a virtual host to be scheduled.

6. According to the big data processing system based on the industrial Internet platform according to claim 5, It is characterized in that The virtual host scheduling server schedules the virtual host to be scheduled, including: For each virtual host to be scheduled, determine an optimal edge server from edge servers other than the target edge server and the edge server where the virtual host to be scheduled is located among the multiple edge servers, and migrate the virtual host to be scheduled to the optimal edge server; The migrated virtual host is also used to send the optimal data collection frequency to the device data collection unit corresponding to the target industrial equipment, and the device data collection unit is also used to collect the target industrial equipment according to the optimal data collection frequency. Status data of the device.

7. A big data processing system based on an industrial Internet platform according to any one of claims 1 to 5, It is characterized in that The virtual host uploads the device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform, including: Based on the correspondence between the multiple virtual hosts and the multiple device data acquisition units and the multiple regional data acquisition units and the relevant information of the virtual hosts, the device heartbeat data packet and / or the regional heartbeat data packet are encrypted, and the encrypted device heartbeat data packet and / or the regional heartbeat data packet are uploaded to the data processing cloud platform.

8. A big data processing system based on an industrial Internet platform according to any one of claims 1 to 5, It is characterized in that The data processing module performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial devices and the status of the operators in the multiple target areas, including: Establishing a device relationship map, wherein the device relationship map is used to record the association relationship between the plurality of industrial devices; For each of the industrial devices, at least one associated industrial device is determined based on the device relationship map, and the state of the industrial device is determined based on the device heartbeat data packet of the industrial device and the device heartbeat data packet of the at least one associated industrial device.

9. A big data processing system based on an industrial Internet platform according to any one of claims 1 to 5, It is characterized in that The data processing module performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial devices and the status of the operators in the multiple target areas, including: Establishing a regional relationship map, wherein the regional relationship map is used to record the association relationship between the multiple target areas; For each of the target areas, at least one associated target area is determined based on the area relationship map, and the status of the operator in the target area is determined based on the area heartbeat data packet of the target area and the area heartbeat data packet of the associated at least one target area.

10. A big data processing method based on industrial Internet platform, It is characterized in that include: Collect status data of multiple industrial equipment; Collect status data of operators in multiple target areas; The virtual host receives the status data of at least one corresponding industrial device and / or the status data of an operator in at least one target area, performs data preprocessing, generates a device heartbeat data packet corresponding to the industrial device and / or a regional heartbeat data packet corresponding to the target area, and uploads the device heartbeat data packet and / or the regional heartbeat data packet to the data processing cloud platform; The data processing cloud platform performs data analysis based on the device heartbeat data packet and / or the regional heartbeat data packet to determine the status of the multiple industrial equipment and the status of the operators in the multiple target areas.

Images