Method for evaluating running state of underground mine roadway tunneling equipment

Abstract

The invention discloses an operation state evaluation method for underground mine roadway tunneling equipment, and belongs to the technical field of computers. Equipment operation parameters of the tunneling equipment, geological parameters of the underground mine, videos in the tunneling direction and physiological parameters and operation sets of operators in the tunneling equipment are obtained; and determining the environment action operation state of the tunneling equipment by using the equipment operation parameters, the geological parameters and the video, thereby obtaining the operation condition of the tunneling equipment under the external environment action. And determining the man-machine action state of the tunneling equipment by using the equipment operation parameters, the physiological parameters and the operation set, thereby obtaining the interaction condition between the operator and the tunneling equipment. And in combination with the environment action operation state and the man-machine action state, the equipment operation state of the tunneling equipment is determined, so that evaluation of the equipment operation state of the tunneling equipment in the working process is realized, and the safety when the tunneling equipment is used for operation is improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method for evaluating the operating status of underground mine tunneling equipment. Background Art

[0002] In underground mine tunneling operations, the traditional way to evaluate the operating status of equipment is manual inspection. Operators rely on experience and simple inspection tools such as wrenches and calipers to regularly inspect the tunneling equipment. For example, they check whether the equipment is damaged or whether parts are loose.

[0003] However, manual inspection is usually delayed, that is, operators can only inspect the tunneling equipment after it has completed its work, but it is difficult to evaluate the operating status of the tunneling equipment in a timely manner during its operation, which may pose a safety hazard. Summary of the invention

[0004] The embodiment of the present application provides an operating status evaluation method for underground mine tunnel excavation equipment, which can evaluate the operating status during the operation of the excavation equipment, timely discover the abnormality of the excavation equipment, and improve the safety of the excavation equipment during operation. The technical solution is as follows: On the one hand, a method for evaluating the operating status of underground mine tunneling equipment is provided, the method comprising: When the tunneling equipment is working, obtaining equipment operating parameters of the tunneling equipment, geological parameters of the underground mine where the tunneling equipment is located, video in the tunneling direction of the tunneling equipment, and physiological parameters and operation sets of operators in the tunneling equipment; Based on the equipment operation parameters, the geological parameters and the video, determining the environmental effect operation state of the tunneling equipment, wherein the environmental effect operation state is used to indicate the operation status of the tunneling equipment under the influence of the external environment; Determining a human-machine interaction state of the tunneling equipment based on the equipment operation parameters, the physiological parameters and the operation set, wherein the human-machine interaction state is used to represent the interaction between the operator and the tunneling equipment; Based on the environmental effect operating state and the human-machine effect state of the tunneling equipment, the equipment operating state of the tunneling equipment is determined.

[0005] On the one hand, a device for evaluating the operating status of underground mine tunneling equipment is provided, the device comprising: An acquisition module, used to acquire, when the excavation equipment is working, equipment operating parameters of the excavation equipment, geological parameters of the underground mine where the excavation equipment is located, a video in the excavation direction of the excavation equipment, and physiological parameters and operation sets of an operator in the excavation equipment; An environmental effect operation state determination module, used to determine the environmental effect operation state of the tunneling equipment based on the equipment operation parameters, the geological parameters and the video, wherein the environmental effect operation state is used to indicate the operation status of the tunneling equipment under the influence of the external environment; A human-machine interaction state determination module, used to determine the human-machine interaction state of the tunneling equipment based on the equipment operation parameters, the physiological parameters and the operation set, wherein the human-machine interaction state is used to represent the interaction between the operator and the tunneling equipment; The equipment operation state determination module is used to determine the equipment operation state of the tunneling equipment based on the environmental effect operation state and the human-machine effect state of the tunneling equipment.

[0006] On the one hand, a computer device is provided, which includes one or more processors and one or more memories, wherein at least one computer program is stored in the one or more memories, and the computer program is loaded and executed by the one or more processors to implement the operating status assessment method of the underground mine tunnel excavation equipment.

[0007] On the one hand, a computer-readable storage medium is provided, in which at least one computer program is stored, and the computer program is loaded and executed by a processor to implement the operating status evaluation method of the underground mine tunnel excavation equipment.

[0008] On the one hand, a computer program product or computer program is provided, which includes a program code, which is stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device executes the above-mentioned underground mine tunnel excavation equipment operation status assessment method.

[0009] Through the technical solution provided in the embodiment of the present application, when the tunneling equipment is working, the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine in which it is located, the video in the tunneling direction, and the physiological parameters and operation set of the operator in the tunneling equipment are obtained. The equipment operating parameters, geological parameters and the video are used to determine the environmental action operating state of the tunneling equipment, thereby obtaining the operation of the tunneling equipment under the influence of the external environment. The human-machine action state of the tunneling equipment is determined using the equipment operating parameters, physiological parameters and the operation set, thereby obtaining the interaction between the operator and the tunneling equipment. In combination with the environmental action operating state and the human-machine action state, the equipment operating state of the tunneling equipment is determined, and the equipment operating state of the tunneling equipment is evaluated during the working process, abnormalities of the tunneling equipment are discovered in time, and the safety of operations using the tunneling equipment is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the following briefly introduces the drawings required for use in the description of the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative work. Figure 1 It is a schematic diagram of an implementation environment of an operating status evaluation method for underground mine tunneling equipment provided in an embodiment of the present application; Figure 2 It is a flow chart of an operating status evaluation method for underground mine tunneling equipment provided in an embodiment of the present application; Figure 3 It is a flow chart of another method for evaluating the operating status of underground mine tunnel excavation equipment provided in an embodiment of the present application; Figure 4 It is a structural schematic diagram of an operating status evaluation device for underground mine tunnel excavation equipment provided in an embodiment of the present application; Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0011] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0012] In this application, the terms "first", "second", etc. are used to distinguish identical or similar items with basically the same effects and functions. It should be understood that there is no logical or temporal dependency between "first", "second", and "nth", nor is there any limitation on quantity and execution order.

[0013] Artificial Intelligence (AI) is the theory, method, technology and application system that uses digital computers or machines controlled by digital computers to simulate, extend and expand human intelligence, perceive the environment, acquire knowledge and use knowledge to obtain better results.

[0014] Machine Learning (ML) is a multi-disciplinary subject that involves probability theory, statistics, approximation theory, convex analysis, algorithm complexity theory, and other disciplines. It specializes in studying how computers simulate or implement human learning behavior to acquire new knowledge or skills and reorganize existing knowledge sub-models to continuously improve their performance. Machine learning is the core of artificial intelligence and the fundamental way to make computers intelligent. Its applications are spread across all areas of artificial intelligence.

[0015] Underground mines (metal and nonmetal underground mines) are mining sites and their ancillary facilities that use adits, inclined shafts, ramps, vertical shafts, etc. as entrances and exits and go deep below the surface to mine metal minerals, radioactive minerals, chemical raw materials, building materials, etc.

[0016] Mine tunnels: Mine tunnels are various passages and spaces excavated during underground mining for the purpose of transporting ore, ventilation, drainage, pedestrians and other related operations.

[0017] Excavation equipment: refers to the general term for various mechanical equipment and tools used for underground tunnel excavation operations. The main functions of these equipment are to crush rocks, transport ore and support tunnels to ensure the safety and efficiency of excavation work.

[0018] In the related art, maintenance personnel will manually inspect the status of the excavation equipment before and after using it to improve the safety of the excavation equipment. However, when the excavation equipment is excavating in the tunnel, the operator is usually focused on the excavation operation and it is difficult to notice the abnormality of the operating status of the excavation equipment, resulting in safety hazards when using the excavation equipment for excavation.

[0019] After adopting the technical solution provided in the embodiment of the present application, the operating status of the excavation equipment can be evaluated during the excavation of the excavation equipment in the tunnel, thereby timely discovering abnormalities of the excavation equipment and improving the safety of operations using the excavation equipment.

[0020] The implementation environment of the embodiment of the present application is introduced below. Figure 1 The implementation environment includes a data processing device 101 , a device controller 102 , and multiple sensors 103 .

[0021] The data processing device 101 is used to obtain data collected by multiple sensors 103 through the equipment controller 102, and process the obtained data to obtain the operating status of the excavation equipment. The data processing device 101 is usually set outside the tunnel to improve the stability of data processing.

[0022] The equipment controller 102 is used to control the excavation equipment. The equipment controller 102 is electrically connected to the multiple sensors 103 and can directly obtain the data collected by the multiple sensors 103. After obtaining the data collected by the multiple sensors 103, the equipment controller 102 will forward the data to the data processing device 101, and the data processing device 101 will process the data. This is because the data processing capability of the equipment controller 102 is usually much weaker than that of the data processing device 101 dedicated for data processing. In the embodiment of the present application, the data processing device 101 and the equipment controller 102 are connected via a wired network or a wireless network.

[0023] Multiple sensors 103 are used to collect relevant data during the operation of the tunneling equipment, providing data support for the technical solution provided in the embodiment of the present application.

[0024] The application scenario of the technical solution provided in the embodiment of the present application is described below. The technical solution provided in the embodiment of the present application can be applied in the scenario of excavating various types of tunnels, for example, it can be applied in excavating prospecting tunnels, production tunnels, and mining tunnels. The technical solution provided in the embodiment of the present application can also be applied to various types of excavation equipment equipped with the above-mentioned equipment controller 102 and multiple sensors 103, that is, as long as the relevant data provided in the embodiment of the present application can be collected, the technical solution provided in the embodiment of the present application can be applied to evaluate the operating status of the excavation equipment.

[0025] In addition, the application of the technical solution provided in the embodiments of the present application requires data processing equipment with strong computing power in order to complete data processing in a timely manner.

[0026] The following is a description of the operating status evaluation method for underground mine tunneling equipment provided in an embodiment of the present application. Figure 2 is a flow chart of an operating status evaluation method for underground mine tunneling equipment provided in an embodiment of the present application, see Figure 2 , taking the execution subject as a data processing device as an example, the method includes the following steps.

[0027] 201. When the tunneling equipment is working, the data processing equipment obtains the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine where the tunneling equipment is located, the video of the tunneling direction of the tunneling equipment, and the physiological parameters and operation set of the operator in the tunneling equipment.

[0028] Among them, the excavation equipment is a device used to excavate tunnels in a mine. The excavation equipment includes an excavation component. The excavation component refers to a component that interacts with the rock mass to destroy the rock mass. For example, the excavation component is an excavation head. The excavation equipment also includes a personnel cabin. The operator of the excavation equipment is located in the personnel cabin, that is, the operator operates the excavation equipment in the personnel cabin. The equipment operation parameters of the excavation equipment are parameters collected when the excavation equipment is working, and can be used together with other parameters to determine the equipment operation status of the excavation equipment. Geological parameters include various indicators that describe the characteristics and properties of geological bodies (rock bodies in underground mines). The excavation direction is the moving direction of the excavation equipment when it is working. In the embodiment of the present application, the video in the excavation direction includes an environmental video and an excavation component video. The environmental video is used to record the environmental conditions in the excavation direction, and the excavation component video is used to record the conditions when the excavation component is working. The physiological parameters of the operator are used to represent the physiological state of the operator. For example, the physiological parameters include body temperature, blood pressure, and blood oxygen saturation. The operation set of the operator is used to represent the operation status of the operator.

[0029] 202. The data processing device determines the environmental effect operating state of the tunneling device based on the device operating parameters, the geological parameters and the video, where the environmental effect operating state is used to represent the operating condition of the tunneling device under the influence of the external environment.

[0030] The external environment refers to the rock mass in the excavation direction, and the external environment refers to the interaction between the rock mass and the excavation components of the excavation equipment. When the excavation equipment excavates on different types of rock masses, the effect of the interaction between the excavation components and the rock mass varies. In some embodiments, the operating state of the environmental effect is represented by environmental state description information, which is information describing the operating state of the environmental effect in the form of text or latent vector.

[0031] 203. The data processing device determines the human-machine interaction state of the tunneling device based on the device operation parameters, the physiological parameters and the operation set, where the human-machine interaction state is used to represent the interaction between the operator and the tunneling device.

[0032] Among them, when the operator operates the excavation equipment in the excavation equipment, the operator's operation will affect the operating state of the excavation equipment. Accordingly, due to the large power and vibration intensity of the excavation equipment, the excavation equipment has a greater impact on the operator. When the operating state of the excavation equipment is abnormal, the impact on the operator will also change. This change can be represented by the physiological state of the operator. Therefore, by determining the physiological state of the operator, the equipment operating state of the excavation equipment can also be inferred. In some embodiments, the human-machine action state is represented by human-machine state description information, and the human-machine state description information is information that describes the human-machine action state in the form of text or latent vectors.

[0033] 204. The data processing device determines the equipment operation state of the tunneling equipment based on the environmental operation state of the tunneling equipment and the human-machine operation state.

[0034] The device operation status is used to indicate the operation status of the tunneling device, and the device operation status can be used to determine whether there is an abnormality in the operation of the tunneling device. In some embodiments, the device operation status is indicated by device operation status description information, and the device operation status description information is information describing the device operation status in the form of text or latent vector.

[0035] Through the technical solution provided in the embodiment of the present application, when the tunneling equipment is working, the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine in which it is located, the video in the tunneling direction, and the physiological parameters and operation set of the operator in the tunneling equipment are obtained. The equipment operating parameters, geological parameters and the video are used to determine the environmental action operating state of the tunneling equipment, thereby obtaining the operation of the tunneling equipment under the influence of the external environment. The human-machine action state of the tunneling equipment is determined using the equipment operating parameters, physiological parameters and the operation set, thereby obtaining the interaction between the operator and the tunneling equipment. In combination with the environmental action operating state and the human-machine action state, the equipment operating state of the tunneling equipment is determined, and the equipment operating state of the tunneling equipment is evaluated during the working process, abnormalities of the tunneling equipment are discovered in time, and the safety of operations using the tunneling equipment is improved.

[0036] The above steps 201-204 are a brief introduction to the operating status evaluation method of the underground mine tunnel excavation equipment provided in the embodiment of the present application. The operating status evaluation method of the underground mine tunnel excavation equipment provided in the embodiment of the present application will be more clearly explained in combination with some examples. Figure 3 , taking the execution subject as a data processing device as an example, the method includes the following steps.

[0037] 301. When the tunneling equipment is working, the data processing equipment obtains the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine where the tunneling equipment is located, the video of the tunneling direction of the tunneling equipment, and the physiological parameters and operation set of the operator in the tunneling equipment.

[0038] Among them, the excavation equipment is a device used to excavate tunnels in a mine. The excavation equipment includes an excavation component. The excavation component refers to a component that interacts with the rock mass to destroy the rock mass. For example, the excavation component is an excavation head. The excavation component destroys the rock mass by rotating. The embodiment of the present application does not limit the shape and size of the excavation component. The excavation equipment also includes a personnel cabin. The operator of the excavation equipment is located in the personnel cabin, that is, the operator operates the excavation equipment in the personnel cabin. The equipment operation parameters of the excavation equipment are parameters collected when the excavation equipment is working, and can be used together with other parameters to determine the equipment operation status of the excavation equipment. Geological parameters include various indicators that describe the characteristics and properties of geological bodies (rock bodies in underground mines). In some embodiments, geological parameters include rock hardness, rock density, and rock porosity. The excavation direction is the moving direction of the excavation equipment when it is working. In the embodiment of the present application, the video in the excavation direction includes an environmental video and an excavation component video. The environmental video is used to record the environmental conditions in the excavation direction, and the excavation component video is used to record the conditions when the excavation component is working. The physiological parameters of the operator are used to indicate the physiological state of the operator. For example, the physiological parameters include body temperature, blood pressure, and blood oxygen saturation. The operation set of the operator is used to indicate the operation status of the operator. In addition, the data processing equipment is not set in the excavation equipment. This is because the vibration of the excavation equipment is large during operation, which may affect the normal operation of the data processing equipment. The amount of calculation of the technical solution provided in the embodiment of the present application is large, and the equipment controller of the excavation equipment cannot meet the computing power requirements. Therefore, the equipment controller sends the collected data to the external data processing equipment, and the data processing equipment performs data processing.

[0039] In a possible implementation, when the excavation equipment is working, the data processing device obtains the equipment operation parameters of the excavation equipment, the video in the excavation direction, and the physiological parameters and operation set of the operator from the equipment controller of the excavation equipment. The data processing device obtains the geological parameters of the underground mine where the excavation equipment is located from the storage medium.

[0040] Among them, multiple sensors are arranged on the excavation equipment. The equipment operation sensor among the multiple sensors is used to collect the equipment operation parameters of the excavation equipment. The visual sensor among the multiple sensors is used to collect the video in the excavation direction. The multiple sensors are connected to the equipment controller and can send the collected data to the excavation equipment. The operator operates the excavation equipment through the equipment controller, so the equipment controller can directly obtain the operator's operation data. In addition, the personnel cabin of the excavation equipment is equipped with a physiological parameter sensor, which needs to be actively worn by the operator. If it is detected that the operator is not wearing the physiological parameter sensor, the excavation equipment cannot be started, thereby ensuring the integrity of data collection.

[0041] In some embodiments, the equipment operation parameters include a temperature parameter set, a vibration parameter set, working parameters of the excavation component, a strain parameter set, a movement parameter, and the cabin environment parameters of the personnel cabin. The temperature parameter set includes the temperatures of multiple components on the excavation equipment. For example, the temperature parameter set includes the working temperature of the excavation component and the working temperature of the engine. The working temperature is collected by a temperature sensor. The engine is used to drive the excavation equipment to move and to drive the excavation component to rotate. The vibration parameter set includes the vibration parameters of the excavation component and the vibration parameters of the engine. The vibration parameters are used to indicate the vibration conditions. The vibration parameters are collected by a vibration sensor. The working parameters include speed and torque, which are collected by a speed sensor and a torque sensor respectively. The strain parameter set includes strains at multiple positions on the excavation component, which are used to indicate the degree of deformation at multiple positions on the excavation component. The strains are collected by a strain sensor. The movement parameters include the moving speed and moving direction of the excavation equipment, which are collected by a movement sensor. The cabin environment parameters include cabin vibration data, cabin temperature, cabin pressure, and cabin gas content in the personnel cabin, which are collected by a vibration sensor, a temperature sensor, an air pressure sensor, and a gas content sensor.

[0042] In some embodiments, the video in the excavation direction includes an environmental video and an excavation component video. The environmental video is used to record the excavation status of the excavation equipment in the tunnel, and the excavation component video is used to record the working status of the excavation component of the excavation equipment.

[0043] In some embodiments, physiological parameters include body temperature, blood pressure, blood oxygen saturation, etc., which are collected through wearable physiological parameter sensors, such as smart bracelets.

[0044] 302. The data processing device determines the rock mass operation information of the tunneling device based on the device operation parameters and the geological parameters.

[0045] The rock mass operation information is used to indicate the operation of the tunneling equipment under the action of the rock mass, the geological parameters can indicate the geological conditions of the underground mine where the tunneling equipment is located, and the equipment operation parameters and geological parameters can reflect the operation of the tunneling equipment under the action of the rock mass. In some embodiments, the rock mass operation information is in the form of text or latent vector.

[0046] In a possible implementation, the equipment operation parameters include a temperature parameter set, a vibration parameter set, and the working parameters of the tunneling component. The data processing device determines the working temperature operation information of the tunneling equipment based on the temperature parameter set, the geological parameters, and the working parameters. The temperature parameter set includes the working temperature of the tunneling component and the working temperature of the engine of the tunneling equipment. The data processing device determines the working vibration operation information of the tunneling equipment based on the vibration parameter set, the geological parameters, and the working parameters. The vibration parameter set includes the vibration parameters of the tunneling component and the vibration parameters of the engine. The data processing device determines the rock mass action operation information of the tunneling equipment based on the working temperature operation information and the working vibration operation information.

[0047] The working temperature operation information is used to indicate the reasonableness of the working temperature when the excavation component interacts with the rock mass of the geological parameters at the working parameters; correspondingly, the working vibration operation information is used to indicate the reasonableness of the vibration when the excavation component interacts with the rock mass of the geological parameters at the working parameters. In some embodiments, the vibration parameters include vibration frequency and vibration intensity.

[0048] In this embodiment, the operating temperature operation information of the tunneling component is determined based on the temperature parameter set, geological parameters and the working parameters of the tunneling component, thereby realizing the recognition of the reasonable degree of the operating temperature of the tunneling component. The operating vibration operation information of the tunneling component is determined based on the vibration parameter set, geological parameters and working parameters, thereby realizing the recognition of the reasonable degree of the vibration condition of the tunneling component. The rock mass action operation information is determined based on the operating temperature operation information and the operating vibration operation information, which can reflect the influence of the rock mass on the operation of the tunneling component.

[0049] In order to explain the above implementation more clearly, the above implementation will be explained in several parts below.

[0050] The first part, the data processing equipment determines the working temperature operation information of the tunneling component based on the temperature parameter set, the geological parameter and the working parameter.

[0051] In a possible implementation, the data processing device determines a reference operating temperature range of the excavation component and a reference operating temperature range of the engine based on the geological parameter and the operating parameter. The data processing device determines component temperature operation information of the excavation component based on the reference operating temperature range of the excavation component and the operating temperature of the excavation component. The data processing device determines engine temperature operation information of the engine based on the reference operating temperature range of the engine and the operating temperature of the engine of the excavation device. The data processing device determines the operating temperature operation information of the excavation device based on the component temperature operation information and the engine temperature operation information.

[0052] Among them, the reference operating temperature range is the temperature range of the operating temperature under normal circumstances. When the engine and tunneling components remain unchanged, there is a corresponding relationship between the operating parameters, geological parameters and operating temperature of the tunneling components. The reason why the reference operating temperature range is determined instead of the reference operating temperature is because the working temperature may also be affected by air heat dissipation and water flow. The reference operating temperature range takes into account the impact of other factors on the working temperature.

[0053] For example, the data processing device inputs the geological parameter and the working parameter into the temperature range prediction model, extracts features of the geological parameter and the working parameter through the temperature range prediction model, and obtains the geological feature corresponding to the geological parameter and the working feature corresponding to the working parameter. The data processing device fuses the geological feature and the working feature through the temperature range prediction model to obtain the working temperature prediction feature. The data processing device maps the working temperature prediction feature through the temperature range prediction model to obtain the reference working temperature range of the excavation component and the reference working temperature range of the engine. When the working temperature of the excavation component is within the reference working temperature range of the excavation component, the data processing device determines the component temperature operation information of the excavation component as the first component temperature operation information, and the first component temperature operation information is used to indicate that the component temperature of the excavation component is normal, that is, the reasonableness is high. When the working temperature of the excavation component is not within the reference working temperature range of the excavation component, the data processing device determines the component temperature operation information of the excavation component as the second component temperature operation information, and the second component temperature operation information is used to indicate that the component temperature of the excavation component is abnormal, that is, the reasonableness is low. When the operating temperature of the engine is within the reference operating temperature range of the engine, the data processing device determines the engine temperature operating information of the engine as the first engine temperature operating information, and the first engine temperature operating information is used to indicate that the engine temperature of the engine is normal, that is, the reasonableness is relatively high. When the operating temperature of the engine is not within the reference operating temperature range of the engine, the data processing device determines the engine temperature operating information of the engine as the second engine temperature operating information, and the second engine temperature operating information is used to indicate that the engine temperature of the engine is abnormal, that is, the reasonableness is relatively low. The data processing device splices the component temperature operating information and the engine temperature operating information to obtain the operating temperature operating information of the tunneling equipment.

[0054] Among them, the temperature range prediction model is a regression model, which is trained based on multiple sample geological parameters, multiple sample working parameters and corresponding annotated reference temperature ranges. The embodiment of the present application does not limit the structure and training method of the temperature range prediction model.

[0055] In order to more clearly illustrate the technical solution provided by the above example, the method of obtaining two reference operating temperature ranges by using the temperature range prediction model in the above example is explained below.

[0056] In some embodiments, the data processing device inputs the geological parameter and the working parameter into a temperature range prediction model, and performs multiple full connections on the geological parameter and the working parameter through the temperature range prediction model to obtain the geological characteristics corresponding to the geological parameter and the working characteristics corresponding to the working parameter. The data processing device fuses the geological characteristics and the working characteristics through the temperature range prediction model to obtain the working temperature prediction characteristics. The data processing device performs full connection and normalization on the working temperature prediction characteristics through the first prediction head of the temperature range prediction model to obtain the reference working temperature range of the tunneling component. The data processing device performs full connection and normalization on the working temperature prediction characteristics through the second prediction head of the temperature range prediction model to obtain the reference working temperature range of the engine.

[0057] Among them, the first prediction head is used to predict the reference operating temperature range of the tunneling component, and the second prediction head is used to predict the reference operating temperature range of the engine. When training the temperature range prediction model, the corresponding labeled reference temperature range will be used to train the first prediction head and the second prediction head respectively.

[0058] The second part, the data processing equipment determines the working vibration operation information of the tunneling equipment based on the vibration parameter set, the geological parameter and the working parameter.

[0059] In a possible implementation, the data processing device determines a reference vibration parameter range of the excavation component and a reference vibration parameter range of the engine based on the geological parameter and the working parameter. The data processing device determines component vibration operation information of the excavation component based on the reference vibration parameter range of the excavation component and the vibration parameters of the excavation component. The data processing device determines engine vibration operation information of the engine based on the reference vibration parameter range of the engine and the vibration parameters of the engine of the excavation device. The data processing device determines the working vibration operation information of the excavation device based on the component vibration operation information and the engine vibration operation information.

[0060] The reference vibration parameter range is the vibration parameter range of the vibration parameters under normal circumstances. For example, the vibration parameters include vibration intensity and vibration frequency, and the reference vibration parameter range includes a reference vibration intensity range and a reference vibration frequency range. When the engine and the tunneling component remain unchanged, there is a corresponding relationship between the working parameters, geological parameters and vibration parameters of the tunneling component. The reason why the reference vibration parameter range is determined instead of the reference vibration parameter is that the working text may also be affected by the working angle of the tunneling component, etc. The reference vibration parameter range takes into account the influence of other factors on the vibration parameters.

[0061] For example, the data processing device inputs the geological parameter and the working parameter into the vibration parameter range prediction model, extracts features of the geological parameter and the working parameter through the vibration parameter range prediction model, and obtains the geological features corresponding to the geological parameter and the working features corresponding to the working parameter. The data processing device fuses the geological features and the working features through the vibration parameter range prediction model to obtain the vibration parameter prediction features. The data processing device maps the vibration parameter prediction features through the vibration parameter range prediction model to obtain the reference vibration parameter range of the excavation component and the reference vibration parameter range of the engine. When the vibration parameters of the excavation component are within the reference vibration parameter range of the excavation component (the vibration intensity is within the reference vibration intensity range, and the vibration frequency is within the reference vibration frequency range), the data processing device determines the component vibration operation information of the excavation component as the first component vibration operation information, and the first component vibration operation information is used to indicate that the component vibration of the excavation component is normal, that is, the rationality is high. In the case where the vibration parameters of the excavation component are not within the reference vibration parameter range of the excavation component (the vibration intensity is not within the reference vibration intensity range, and / or the vibration frequency is not within the reference vibration frequency range), the data processing device determines the component vibration operation information of the excavation component as the second component vibration operation information, and the second component vibration operation information is used to indicate that the component vibration of the excavation component is abnormal, that is, the rationality is low. In the case where the vibration parameters of the engine are within the reference vibration parameter range of the engine, the data processing device determines the engine vibration operation information of the engine as the first engine vibration operation information, and the first engine vibration operation information is used to indicate that the engine vibration of the engine is normal, that is, the rationality is high. In the case where the vibration parameters of the engine are not within the reference vibration parameter range of the engine, the data processing device determines the engine vibration operation information of the engine as the second engine vibration operation information, and the second engine vibration operation information is used to indicate that the engine vibration of the engine is abnormal, that is, the rationality is low. The data processing device splices the component vibration operation information and the engine vibration operation information to obtain the working vibration operation information of the excavation equipment.

[0062] Among them, the vibration parameter range prediction model is a regression model, which is trained based on multiple sample geological parameters, multiple sample working parameters and corresponding annotated reference vibration parameter ranges. The embodiment of the present application does not limit the structure and training method of the vibration parameter range prediction model.

[0063] In order to more clearly illustrate the technical solution provided by the above example, the method of obtaining two reference vibration parameter ranges by using the vibration parameter range prediction model in the above example is explained below.

[0064] In some embodiments, the data processing device inputs the geological parameter and the working parameter into a vibration parameter range prediction model, and performs multiple full connections on the geological parameter and the working parameter through the vibration parameter range prediction model to obtain the geological characteristics corresponding to the geological parameter and the working characteristics corresponding to the working parameter. The data processing device fuses the geological characteristics and the working characteristics through the vibration parameter range prediction model to obtain the vibration parameter prediction characteristics. The data processing device performs full connection and normalization on the vibration parameter prediction characteristics through the first prediction head of the vibration parameter range prediction model to obtain the reference vibration parameter range of the tunneling component. The data processing device performs full connection and normalization on the vibration parameter prediction characteristics through the second prediction head of the vibration parameter range prediction model to obtain the reference vibration parameter range of the engine.

[0065] Among them, the first prediction head is used to predict the reference vibration parameter range of the tunneling component, and the second prediction head is used to predict the reference vibration parameter range of the engine. When training the vibration parameter range prediction model, the corresponding labeled reference vibration parameter range will be used to train the first prediction head and the second prediction head respectively.

[0066] Part 3: The data processing device determines the rock mass operation information of the tunneling equipment based on the working temperature operation information and the working vibration operation information.

[0067] In a possible implementation, the data processing device splices the working temperature operation information and the working vibration operation information to obtain the rock mass operation information of the tunneling equipment.

[0068] 303. The video includes an environmental video and a tunneling component video. The data processing device determines the tunneling component status information of the tunneling device based on the device operating parameters and the tunneling component video. The tunneling component video is used to record the working condition of the tunneling component of the tunneling device.

[0069] The component status information of the tunneling component is used to describe the component status of the tunneling component.

[0070] In a possible implementation, the equipment operation parameters include the working parameters and strain parameter set of the tunneling component, and the data processing device performs video recognition on the tunneling component video to obtain the component surface state and component connection state of the tunneling component, and the component surface state is used to represent the surface condition of the tunneling component. The data processing device determines the tunneling component state information of the tunneling equipment based on the working parameters, the strain parameter set, the component surface state, and the component connection state, and the strain parameter set includes strains at multiple positions on the tunneling component.

[0071] Among them, the component surface state is used to indicate the surface condition of the excavation component, and the component connection state is used to indicate the connection condition between multiple sub-components in the excavation component.

[0072] In this implementation, the surface state and component connection state of the excavation component are obtained by performing video recognition on the excavation component video. The excavation component state information of the excavation equipment is determined by using the working parameters, the strain parameter set, the component surface state and the component connection state, and the excavation component state information can more accurately reflect the component state of the excavation component.

[0073] In order to explain the above implementation more clearly, the above implementation is explained in several parts below.

[0074] In the first part, the data processing equipment performs video recognition on the tunneling component video to obtain the component surface status and component connection status of the tunneling component.

[0075] In a possible implementation, the data processing device inputs the tunneling component video into a first video recognition model, extracts features from the tunneling component video through the first video recognition model, and obtains a first video feature of the tunneling component video. The data processing device maps the first video feature through the first video recognition model to obtain a component surface state and a component connection state of the tunneling component.

[0076] For example, the data processing device inputs the tunneling component video into the first video recognition model, and convolves multiple video frames of the tunneling component video through the first video recognition model to obtain video frame features of each video frame. The data processing device encodes the video frame features of multiple video frames based on the attention mechanism through the first video recognition model to obtain the first video feature of the tunneling component video. The data processing device fully connects and normalizes the first video feature through the first classification head of the first video recognition model to obtain the component surface state of the tunneling component. The data processing device fully connects and normalizes the first video feature through the second classification head of the first video recognition model to obtain the component connection state of the tunneling component.

[0077] Among them, the surface state of the component is one of multiple candidate surface states, and the multiple candidate surface states include intact surface, slight surface damage, surface damage and surface rupture. The component connection state is one of multiple candidate connection states, and the multiple candidate connection states include intact connection, damaged connection and disconnected connection. Correspondingly, determining the component surface state and the component connection state can be regarded as a classification based on the first video feature. The first video recognition model not only has video recognition capabilities, but also has multi-target classification capabilities. The embodiment of the present application does not limit the structure and training method of the first video recognition model.

[0078] In order to more clearly illustrate the technical solution provided in the above example, the method of using the first sorting head in the above example to determine the surface state of the component is described below.

[0079] In some embodiments, the data processing device performs full connection and normalization on the first video feature through the first classification head of the first video recognition model to obtain a first probability set of the tunneling component, the first probability set including multiple probabilities, one probability corresponding to one candidate surface state. The data processing device determines the candidate surface state corresponding to the highest probability in the first probability set as the surface state of the component.

[0080] It should be noted that the method of using the second classification head to determine the connection status of the component in the above example and the method of determining the surface status of the component belong to the same inventive concept. The implementation process is described above and will not be repeated here.

[0081] In the second part, the data processing equipment determines the state information of the excavation component of the excavation equipment based on the working parameters, the strain parameter set, the surface state of the component and the connection state of the component. The strain parameter set includes strains at multiple positions on the excavation component.

[0082] In a possible implementation, the data processing device determines the working strain information of the tunneling component based on the working parameter and the strain parameter set. The data processing device determines the working damage information of the tunneling component based on the surface state and the connection state. The data processing device determines the tunneling component state information of the tunneling device based on the working strain information and the working damage information.

[0083] Among them, the tunneling component will inevitably produce strain when working. The working strain information determined by combining the working parameters and the strain parameter set can be used to reflect whether the strain of the tunneling component is reasonable, that is, whether there is an abnormality. The surface state and the connection state can reflect the damage of the tunneling component from different dimensions. Therefore, the working damage information determined by combining the surface state and the connection state can be used to reflect the damage of the tunneling component.

[0084] For example, the data processing device determines the deformation degree of the tunneling component based on the multiple strains in the strain parameter set. The data processing device determines the working strain information of the tunneling component based on the deformation degree and the working parameter. The data processing device splices the surface state and the connection state to obtain the working damage information of the tunneling component. The data processing device splices the working strain information and the working damage information to obtain the tunneling component state information of the tunneling device.

[0085] In order to more clearly illustrate the technical solution provided by the above example, the manner in which the data processing equipment in the above example determines the working strain information of the tunneling component based on the deformation degree and the working parameters is explained below.

[0086] In some embodiments, the data processing device determines the deformation degree and the working parameter to belong to the strain information determination model, and extracts the feature of the working parameter through the strain information determination model to obtain the working parameter feature of the working parameter. The data processing device fuses the working parameter feature and the deformation degree through the strain information determination model to obtain the working strain information determination feature. The data processing device maps the working strain information determination feature through the strain information determination model to obtain the working strain information.

[0087] The working strain information includes normal strain and abnormal strain. Accordingly, the strain information determination model is a binary classification model, and the embodiment of the present application does not limit the structure and training method of the strain information determination model.

[0088] 304. The data processing device determines the excavation effect information of the excavation device based on the geological parameters and the environmental video, and the environmental video is used to record the excavation situation of the excavation device in the tunnel.

[0089] The excavation effect information is used to indicate the excavation effect of the excavation equipment in the underground mine.

[0090] In a possible implementation, the geological parameters include rock hardness, rock density and rock porosity, and the data processing device performs video recognition on the environmental video to obtain the rock size and rock shape of the broken rock in the excavation direction of the excavation equipment. The data processing device determines the size crushing effect information of the broken rock based on the rock hardness, the rock porosity and the rock size. The data processing device determines the shape crushing effect information of the broken rock based on the rock density and the rock shape. The data processing device determines the excavation effect information of the excavation equipment based on the size crushing effect information and the shape crushing effect information.

[0091] The size crushing effect information is used to describe the size crushing effect of the excavation component on the rock mass, and the shape crushing effect information is used to describe the shape crushing effect of the excavation component on the rock mass.

[0092] In this implementation mode, video recognition is performed on the environmental video to obtain the rock size and rock shape of the broken rock in the excavation direction of the excavation equipment. The size crushing effect information of the broken rock is determined by combining the rock hardness, rock porosity and rock size. The shape crushing effect information of the broken rock is determined by combining the rock density and rock shape. The final excavation effect information is obtained by combining the size crushing effect information and the shape crushing effect information of the broken rock, and the accuracy of the excavation effect information is high.

[0093] In order to explain the above implementation more clearly, the above implementation is explained in several parts below.

[0094] In the first part, the data processing equipment performs video recognition on the environmental video to obtain the rock size and rock shape of the broken rock mass in the excavation direction of the excavation equipment.

[0095] The data processing device inputs the environment video into the second video recognition model, extracts features of the environment video through the second video recognition model, and obtains the second video features of the environment video. The data processing device maps the second video features through the second video recognition model to obtain the rock mass size and rock mass shape of the tunneling assembly.

[0096] For example, the data processing device inputs the environment video into the second video recognition model, and convolves multiple video frames of the environment video through the second video recognition model to obtain video frame features of each video frame. The data processing device encodes the video frame features of multiple video frames based on the attention mechanism through the second video recognition model to obtain the second video feature of the environment video. The data processing device fully connects and normalizes the second video feature through the first classification head of the second video recognition model to obtain the rock size of the excavation component. The data processing device fully connects and normalizes the second video feature through the second classification head of the second video recognition model to obtain the rock shape of the excavation component.

[0097] Among them, the rock mass size is one of multiple candidate surface states, and the multiple candidate surface states include intact surface, slightly damaged surface, damaged surface, and cracked surface. The rock mass shape is one of multiple candidate connection states, and the multiple candidate connection states include intact connection, damaged connection, and disconnected connection. Accordingly, determining the rock mass size and rock mass shape can be regarded as a classification based on the second video feature. The second video recognition model not only has video recognition capabilities, but also has multi-target classification capabilities. The embodiment of the present application does not limit the structure and training method of the second video recognition model.

[0098] The second part, the data processing equipment determines the size crushing effect information of the crushed rock mass based on the hardness of the rock mass, the porosity of the rock mass and the size of the rock mass.

[0099] In a possible implementation, the data processing device determines a reference rock size range of the crushed rock mass based on the hardness of the rock mass and the porosity of the rock mass, the reference rock size range including a plurality of reference rock sizes, one reference rock size corresponding to one reference size crushing effect information. The data processing device determines the reference size crushing effect information corresponding to the rock size in the reference rock size range as the size crushing effect information of the crushed rock mass.

[0100] The multiple reference rock mass sizes and the corresponding reference size crushing effect information are set by technicians according to actual conditions, and the embodiments of the present application do not limit this. In some embodiments, the reference size crushing effect information is good effect, average effect and poor effect.

[0101] The third part, the data processing equipment determines the shape crushing effect information of the crushed rock mass based on the density of the rock mass and the shape of the rock mass.

[0102] In a possible implementation, the data processing device determines a reference rock shape set of the broken rock mass based on the rock mass density, the reference rock shape set including a plurality of reference rock shapes, one reference rock shape corresponding to one reference shape crushing effect information. The data processing device determines the reference shape crushing effect information corresponding to the rock shape in the reference rock shape set as the shape crushing effect information of the broken rock mass.

[0103] The multiple reference rock shapes and the corresponding reference shape crushing effect information are set by technicians according to actual conditions, and the embodiments of the present application do not limit this. In some embodiments, the reference shape crushing effect information is good effect, average effect and poor effect.

[0104] Part 4: The data processing device determines the excavation effect information of the excavation equipment based on the size crushing effect information and the shape crushing effect information.

[0105] In a possible implementation, the data processing device fuses the size crushing effect information and the shape crushing effect information to obtain the excavation effect information of the excavation equipment.

[0106] 305. The data processing device determines the environmental action operation state of the tunneling equipment based on the rock mass action operation information, the tunneling component status information and the tunneling effect information. The environmental action operation state is used to represent the operation status of the tunneling equipment under the action of the external environment.

[0107] The external environment refers to the rock mass in the excavation direction, and the external environment refers to the interaction between the rock mass and the excavation components of the excavation equipment. When the excavation equipment excavates on different types of rock masses, the effect of the interaction between the excavation components and the rock mass varies. In some embodiments, the operating state of the environmental effect is represented by environmental state description information, which is information describing the operating state of the environmental effect in the form of text or latent vector.

[0108] In a possible implementation, the data processing device splices the rock mass action operation information, the tunneling component state information, and the tunneling effect information to obtain first operation state determination information. The data processing device inputs the first operation state determination information into the environment action operation state determination model, extracts features of the first operation state determination information through the environment action operation state determination model, and obtains first operation state determination features. The data processing device decodes the first operation state determination features through the environment action operation state determination model to obtain the environment action operation state of the tunneling device.

[0109] The environment effect operation state determination model is a regression model, which can map the first operation state determination information into the environment effect operation state.

[0110] For example, the data processing device splices the rock mass action operation information, the tunneling component status information, and the tunneling effect information to obtain the first operation state determination information. The data processing device inputs the first operation state determination information into the environment action operation state determination model, and performs multiple full connections on the first operation state determination information through the environment action operation state determination model to obtain the first operation state determination feature. The data processing device uses the environment action operation state determination model to perform multiple rounds of iterative decoding on the first operation state determination feature based on the attention mechanism to obtain the environment action operation state of the tunneling equipment.

[0111] 306. The data processing device determines the physiological state information of the operator in the tunneling device based on the device operating parameters and the physiological parameters.

[0112] Among them, the physiological state information is used to describe the physiological state of the operator. The physiological state information is obtained by considering the cabin environment and physiological parameters of the personnel cabin. Compared with the physiological state information obtained by only considering the physiological parameters, it is more in line with the actual situation.

[0113] In a possible implementation, the equipment operation parameters include cabin environment parameters, and the data processing device determines the environmental description information of the personnel cabin of the tunneling equipment based on the cabin environment parameters, and the cabin environment parameters include cabin vibration data, cabin temperature, cabin pressure, and cabin gas content in the personnel cabin. The data processing device determines the physiological state information of the operator based on the environmental description information and the physiological parameters.

[0114] For example, the data processing device inputs the cabin vibration data, cabin temperature, cabin pressure and cabin gas content into the environment description information generation model, and extracts the features of the cabin vibration data, cabin temperature, cabin pressure and cabin gas content through the environment description information generation model to obtain the cabin environment features. The data processing device decodes the cabin environment features through the environment description information generation model to obtain the environment description information. The data processing device inputs the environment description information and the physiological parameters into the physiological state information determination model, and extracts the features of the environment description information and the physiological parameters through the physiological state information determination model to obtain the environment description features of the environment description information and the physiological features of the physiological parameters. The data processing device fuses the environment description features and the physiological features through the physiological state information determination model to obtain the environment physiological features. The data processing device decodes the environment physiological features through the physiological state information determination model to obtain the physiological state information of the operator.

[0115] In order to more clearly illustrate the implementation method provided by the above example, the following describes the method of determining the model through the physiological state information in the above example, decoding the physiological characteristics of the environment, and obtaining the physiological state information of the operator.

[0116] In some embodiments, the data processing device uses the physiological state determination model to perform multiple rounds of iterative decoding on the physiological characteristics of the environment based on the attention mechanism to obtain the physiological state information of the operator.

[0117] 307. The data processing device determines operation state information of the operator based on the physiological state information and the operation set, where the operation state information is used to indicate the controllability of the operator over the executed operation.

[0118] In a possible implementation, the data processing device determines the degree of physiological abnormality of the operator based on the physiological state information and the reference physiological state information of the operator, the reference physiological state information is determined based on the equipment operation parameters and the reference physiological parameters of the operator, the reference physiological parameters are the average physiological parameters of the operator when using the tunneling equipment. The data processing device determines the degree of operation abnormality of the operator based on the operation set and the operation mode of the operator, the operation mode is used to represent the operation habits of the operator. The data processing device determines the operation state information of the operator based on the degree of physiological abnormality and the degree of operation abnormality of the operator.

[0119] The reference physiological state information can be regarded as the baseline of the operator's physiological state. The method of determining the reference physiological state information based on the equipment operation parameters and the reference physiological parameters belongs to the same inventive concept as the method of determining the physiological state information in the above step 306, and the implementation process is not repeated. The operation mode is used to represent the operation habits of the operator. The operation mode can be used to identify abnormalities in the operation of the operator. That is, under normal circumstances, the operation habits of the operator will not change. Once an operation that violates the operation habits occurs, it may indicate that an abnormal situation has occurred.

[0120] In order to explain the above implementation more clearly, the above implementation is explained in several parts below.

[0121] The first part, a data processing device determines the degree of physiological abnormality of the operator based on the physiological state information and the reference physiological state information of the operator.

[0122] In a possible implementation, the data processing device performs feature extraction on the physiological state information and the reference physiological state information respectively to obtain the physiological state feature of the physiological state information and the reference physiological state feature of the reference physiological state information. The data processing device determines the feature similarity between the physiological state feature and the reference physiological state feature, and determines the degree of physiological abnormality of the operator based on the feature similarity.

[0123] The feature similarity is negatively correlated with the degree of physiological abnormality, that is, the higher the feature similarity, the lower the degree of physiological abnormality; the lower the feature similarity, the higher the degree of physiological abnormality. In some embodiments, the reciprocal of the feature similarity is determined as the degree of physiological abnormality.

[0124] Part 2: The data processing device determines the degree of abnormality of the operation of the operator based on the operation set and the operation mode of the operator.

[0125] Among them, the operation set includes multiple operations on the tunneling equipment, the operation mode includes multiple groups of operation sequences, one group of operation sequences includes multiple operations, one group of operation sequences is a combination of multiple operations commonly used by operators, and the operating habits of operators can be reflected through multiple groups of operation sequences.

[0126] In a possible implementation, the data processing device performs feature extraction on the operation set and the operation mode to obtain the operation feature of the operation set and the operation mode feature of the operation mode. The data processing device determines the feature similarity between the operation feature and the operation mode feature, and determines the degree of abnormality of the operation of the operator based on the feature similarity.

[0127] The feature similarity is negatively correlated with the degree of abnormal operation, that is, the higher the feature similarity, the lower the degree of abnormal operation; the lower the feature similarity, the higher the degree of abnormal operation. In some embodiments, the reciprocal of the feature similarity is determined as the degree of abnormal operation.

[0128] Part 3: The data processing device determines the operation status information of the operator based on the physiological abnormality level and the operation abnormality level of the operator.

[0129] In a possible implementation, the data processing device combines the physiological abnormality level and the operational abnormality level of the operator to obtain the operational status information of the operator.

[0130] 308. The data processing device determines operation matching information of the tunneling device based on the device operating parameters and the operation set.

[0131] In a possible implementation, the equipment operation parameters include movement parameters of the tunneling equipment and working parameters of the tunneling component, the operation set includes a movement operation set and a tunneling operation set, and the data processing device determines the movement operation matching information of the tunneling equipment based on the movement parameters and the movement operation set, the movement operation matching information is used to indicate the matching situation between the movement operation and the movement parameters, and the movement operation is used to control the tunneling equipment to move. The data processing device determines the tunneling operation matching information of the tunneling equipment based on the working parameters and the tunneling operation set, the tunneling operation matching information is used to indicate the matching situation between the tunneling operation and the working parameters, and the tunneling operation is used to control the tunneling component. The data processing device determines the operation matching information of the tunneling equipment based on the movement operation matching information and the tunneling operation matching information.

[0132] Among them, the operation matching information is used to reflect the matching degree between the operation status of the tunneling equipment and the operation of the operator. Generally speaking, the higher the matching degree, the better the controllability of the tunneling equipment and the lower the probability of failure. Correspondingly, the lower the matching degree, the worse the controllability of the tunneling equipment and the higher the probability of failure.

[0133] In this embodiment, the mobile operation matching information is determined by using the mobile parameters and the mobile operation set, and the excavation operation matching information is determined by using the working parameters and the excavation operation set. The operation matching information can be determined by combining the mobile operation matching information and the excavation operation matching information, thereby determining the operation matching information, thereby achieving the determination of the matching degree between the operation and the operation status of the excavation equipment.

[0134] In order to explain the above implementation more clearly, the above implementation will be explained in several parts below.

[0135] In the first part, a data processing device determines the mobile operation matching information of the tunneling device based on the mobile parameter and the mobile operation set.

[0136] The moving parameters include moving speed and moving direction, the moving operation set includes multiple moving operations, and the moving operations are used to control the moving speed and moving direction of the tunneling equipment.

[0137] In a possible implementation, the data processing device determines a reference movement parameter of the tunneling device based on the movement operation set, the reference movement parameter corresponding to the movement operation set, and determines movement operation matching information of the tunneling device based on the reference movement parameter and the movement parameter.

[0138] The reference movement parameter is the expected movement parameter after executing the movement operation set on the tunneling equipment.

[0139] In this embodiment, the reference movement parameter of the tunneling equipment is determined by using the movement operation set, and the movement operation matching information is determined by using the reference movement parameter and the movement parameter, so that the accuracy of the movement operation matching information is relatively high.

[0140] For example, the data processing device inputs the mobile operation set into the mobile parameter prediction model, extracts features of the mobile operation set through the mobile parameter prediction model, and obtains the operation set features of the mobile operation set. The data processing device maps the operation set features through the mobile parameter prediction model to obtain the reference mobile parameter. The data processing device determines the mobile operation matching information based on the difference information between the reference mobile parameter and the mobile parameter.

[0141] The difference information is used to indicate the degree of difference between the reference movement parameter and the movement parameter, and the movement operation matching information is used to indicate the degree of matching between the movement operation set and the movement parameter. Generally speaking, the degree of difference is negatively correlated with the degree of matching, that is, the higher the degree of difference, the lower the degree of matching; the lower the degree of difference, the higher the degree of matching. The movement parameter determination model is a regression model, which is obtained by multiple rounds of training based on multiple sample movement operation sets and the annotated movement parameters corresponding to each sample movement operation set. The embodiment of the present application does not limit the structure and training method of the movement parameter determination model.

[0142] In the second part, the data processing device determines the tunneling operation matching information of the tunneling device based on the working parameters and the tunneling operation set.

[0143] The working parameters include rotation speed and torque, the excavation operation set includes multiple excavation operations, and the excavation operations are used to control the rotation speed and torque of the excavation components.

[0144] In a possible implementation, the data processing device determines a reference working parameter of the excavation component based on the excavation operation set, the reference working parameter corresponding to the excavation operation set. The data processing device determines excavation operation matching information of the excavation component based on the reference working parameter and the working parameter.

[0145] The reference working parameter is the expected working parameter after the excavation component performs the excavation operation set.

[0146] In this embodiment, the excavation operation set is used to determine the reference working parameters of the excavation component, and the excavation operation matching information is determined using the reference working parameters and the working parameters, so that the excavation operation matching information has a high accuracy.

[0147] For example, the data processing device inputs the excavation operation set into the working parameter prediction model, extracts features of the excavation operation set through the working parameter prediction model, and obtains the operation set features of the excavation operation set. The data processing device maps the operation set features through the working parameter prediction model to obtain the reference working parameter. The data processing device determines the excavation operation matching information based on the difference information between the reference working parameter and the working parameter.

[0148] The difference information is used to indicate the degree of difference between the reference working parameter and the working parameter, and the excavation operation matching information is used to indicate the degree of matching between the excavation operation set and the working parameter. Generally speaking, the degree of difference is negatively correlated with the degree of matching, that is, the higher the degree of difference, the lower the degree of matching; the lower the degree of difference, the higher the degree of matching. The working parameter determination model is a regression model, which is obtained by multiple rounds of training based on multiple sample excavation operation sets and the annotated working parameters corresponding to each sample excavation operation set. The embodiment of the present application does not limit the structure and training method of the working parameter determination model.

[0149] Part three: The data processing device determines the operation matching information of the tunneling device based on the movement operation matching information and the tunneling operation matching information.

[0150] In a possible implementation, the data processing device splices the movement operation matching information and the excavation operation matching information to obtain the operation matching information of the excavation device.

[0151] 309. The data processing device determines the human-machine interaction state of the tunneling device based on the physiological state information, the operation state information and the operation matching information. The human-machine interaction state is used to represent the interaction between the operator and the tunneling device.

[0152] Among them, when the operator operates the excavation equipment in the excavation equipment, the operator's operation will affect the operating state of the excavation equipment. Accordingly, due to the large power and vibration intensity of the excavation equipment, the excavation equipment has a greater impact on the operator. When the operating state of the excavation equipment is abnormal, the impact on the operator will also change. This change can be represented by the physiological state of the operator. Therefore, by determining the physiological state of the operator, the equipment operating state of the excavation equipment can also be inferred. In some embodiments, the human-machine action state is represented by human-machine state description information, and the human-machine state description information is information that describes the human-machine action state in the form of text or latent vectors.

[0153] In a possible implementation, the data processing device determines the personnel status of the operator based on the physiological status information and the operation status information. The data processing device determines the operation status of the tunneling equipment based on the operation matching information. The data processing device determines the human-machine interaction status of the tunneling equipment based on the personnel status and the operation status.

[0154] The personnel status is used to indicate the status of the operator under the influence of the tunneling equipment. For example, the personnel status includes normal, slightly abnormal, and abnormal. The operation status is used to indicate the response status to the operation. For example, the operation status includes timely response, delayed response, and abnormal response.

[0155] For example, the data processing device splices the physiological state information and the operation state information to obtain personnel state determination information. The data processing device inputs the personnel state determination information into the personnel state determination model, and extracts features from the personnel state determination information through the personnel state determination model to obtain personnel state determination features. The data processing device decodes the personnel state determination features through the personnel state determination model to obtain the personnel state of the operator. The data processing device inputs the operation matching information into the operation state determination model, and extracts features from the operation matching information through the operation state determination model to obtain operation state determination features. The data processing device decodes the operation state determination features through the operation state determination model to obtain the operation state of the tunneling equipment. The data processing device splices the personnel state and the operation state to obtain the human-machine interaction state of the tunneling equipment.

[0156] 310. The data processing device determines the equipment operation state of the tunneling equipment based on the environmental operation state of the tunneling equipment and the human-machine operation state.

[0157] The device operation status is used to indicate the operation status of the tunneling device, and the device operation status can be used to determine whether there is an abnormality in the operation of the tunneling device. In some embodiments, the device operation status is indicated by device operation status description information, and the device operation status description information is information describing the device operation status in the form of text or latent vector.

[0158] In a possible implementation, a data processing device combines the environmental action operating state and the human-machine action state of the tunneling device to obtain the device operating state of the tunneling device.

[0159] In some embodiments, when the operating status of the equipment meets the preset conditions, it means that there is no abnormality in the operation of the tunneling equipment; when the operating status of the equipment does not meet the preset conditions, it means that there is an abnormality in the operation of the tunneling equipment. The preset conditions are set by technicians according to actual conditions, and the embodiments of the present application do not limit this.

[0160] All the above optional technical solutions can be arbitrarily combined to form optional embodiments of the present application, which will not be described one by one here.

[0161] Through the technical solution provided in the embodiment of the present application, when the tunneling equipment is working, the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine in which it is located, the video in the tunneling direction, and the physiological parameters and operation set of the operator in the tunneling equipment are obtained. The equipment operating parameters, geological parameters and the video are used to determine the environmental action operating state of the tunneling equipment, thereby obtaining the operation of the tunneling equipment under the influence of the external environment. The human-machine action state of the tunneling equipment is determined using the equipment operating parameters, physiological parameters and the operation set, thereby obtaining the interaction between the operator and the tunneling equipment. In combination with the environmental action operating state and the human-machine action state, the equipment operating state of the tunneling equipment is determined, and the equipment operating state of the tunneling equipment is evaluated during the working process, abnormalities of the tunneling equipment are discovered in time, and the safety of operations using the tunneling equipment is improved.

[0162] Figure 4 is a schematic diagram of the structure of an operating status evaluation device for underground mine tunneling equipment provided in an embodiment of the present application, see Figure 4 The device includes: an acquisition module 401, an environment action operation state determination module 402, a human-machine action state determination module 403 and an equipment operation state determination module 404.

[0163] The acquisition module 401 is used to obtain the equipment operating parameters of the excavation equipment, the geological parameters of the underground mine where the excavation equipment is located, the video in the excavation direction of the excavation equipment, and the physiological parameters and operation set of the operator in the excavation equipment when the excavation equipment is working.

[0164] The environmental effect operation status determination module 402 is used to determine the environmental effect operation status of the tunneling equipment based on the equipment operation parameters, the geological parameters and the video. The environmental effect operation status is used to represent the operation status of the tunneling equipment under the influence of the external environment.

[0165] The human-machine interaction state determination module 403 is used to determine the human-machine interaction state of the tunneling equipment based on the equipment operation parameters, the physiological parameters and the operation set. The human-machine interaction state is used to represent the interaction between the operator and the tunneling equipment.

[0166] The equipment operation status determination module 404 is used to determine the equipment operation status of the tunneling equipment based on the environmental operation status of the tunneling equipment and the human-machine operation status.

[0167] In a possible implementation, the video includes an environment video and a tunneling component video, and the environment action operation status determination module 402 is used to determine the rock action operation information of the tunneling equipment based on the equipment operation parameters and the geological parameters. Based on the equipment operation parameters and the tunneling component video, the tunneling component status information of the tunneling equipment is determined, and the tunneling component video is used to record the working conditions of the tunneling component of the tunneling equipment. Based on the geological parameters and the environment video, the tunneling effect information of the tunneling equipment is determined, and the environment video is used to record the tunneling conditions of the tunneling equipment in the tunnel. Based on the rock action operation information, the tunneling component status information and the tunneling effect information, the environment action operation status of the tunneling equipment is determined.

[0168] In a possible implementation, the equipment operation parameters include a temperature parameter set, a vibration parameter set, and the working parameters of the tunneling component. The environment action operation state determination module 402 is used to determine the working temperature operation information of the tunneling equipment based on the temperature parameter set, the geological parameters, and the working parameters. The temperature parameter set includes the working temperature of the tunneling component and the working temperature of the engine of the tunneling equipment. Based on the vibration parameter set, the geological parameters, and the working parameters, the working vibration operation information of the tunneling equipment is determined. The vibration parameter set includes the vibration parameters of the tunneling component and the vibration parameters of the engine. Based on the working temperature operation information and the working vibration operation information, the rock action operation information of the tunneling equipment is determined.

[0169] In a possible implementation, the equipment operation parameters include the working parameters and strain parameter set of the tunneling component, and the environment action operation state determination module 402 is used to perform video recognition on the tunneling component video to obtain the component surface state and component connection state of the tunneling component, and the component surface state is used to represent the surface condition of the tunneling component. Based on the working parameters, the strain parameter set, the component surface state and the component connection state, the tunneling component state information of the tunneling equipment is determined, and the strain parameter set includes strains at multiple positions on the tunneling component.

[0170] In a possible implementation, the geological parameters include rock hardness, rock density and rock porosity, and the environment action operation state determination module 402 is used to perform video recognition on the environment video to obtain the rock size and rock shape of the broken rock in the excavation direction of the excavation equipment. Based on the rock hardness, the rock porosity and the rock size, the size crushing effect information of the broken rock is determined. Based on the rock density and the rock shape, the shape crushing effect information of the broken rock is determined. Based on the size crushing effect information and the shape crushing effect information, the excavation effect information of the excavation equipment is determined.

[0171] In a possible implementation, the human-machine action state determination module 403 is used to determine the physiological state information of the operator in the tunneling equipment based on the equipment operation parameters and the physiological parameters. Based on the physiological state information and the operation set, the operation state information of the operator is determined, and the operation state information is used to indicate the controllability of the operator over the executed operation. Based on the equipment operation parameters and the operation set, the operation matching information of the tunneling equipment is determined. Based on the physiological state information, the operation state information and the operation matching information, the human-machine action state of the tunneling equipment is determined.

[0172] In a possible implementation, the equipment operation parameters include cabin environment parameters, and the human-machine action state determination module 403 is used to determine the environmental description information of the personnel cabin of the tunneling equipment based on the cabin environment parameters, and the cabin environment parameters include cabin vibration data, cabin temperature, cabin pressure, and cabin gas content in the personnel cabin. Based on the environmental description information and the physiological parameters, the physiological state information of the operator is determined.

[0173] In a possible implementation, the equipment operation parameters include the movement parameters of the tunneling equipment and the working parameters of the tunneling component, the operation set includes the movement operation set and the tunneling operation set, and the human-machine action state determination module 403 is used to determine the movement operation matching information of the tunneling equipment based on the movement parameters and the movement operation set, the movement operation matching information is used to indicate the matching situation between the movement operation and the movement parameters, and the movement operation is used to control the tunneling equipment to move. Based on the working parameters and the tunneling operation set, the tunneling operation matching information of the tunneling equipment is determined, the tunneling operation matching information is used to indicate the matching situation between the tunneling operation and the working parameters, and the tunneling operation is used to control the tunneling component. Based on the movement operation matching information and the tunneling operation matching information, the operation matching information of the tunneling equipment is determined.

[0174] In a possible implementation, the human-machine action state determination module 403 is used to determine the degree of physiological abnormality of the operator based on the physiological state information and the reference physiological state information of the operator, wherein the reference physiological state information is determined based on the equipment operation parameters and the reference physiological parameters of the operator, and the reference physiological parameters are the average physiological parameters of the operator when using the excavation equipment. The degree of operation abnormality of the operator is determined based on the operation set and the operation mode of the operator, and the operation mode is used to represent the operation habits of the operator. The operation state information of the operator is determined based on the degree of physiological abnormality and the degree of operation abnormality of the operator.

[0175] In a possible implementation, the human-machine action state determination module 403 is used to determine the personnel state of the operator based on the physiological state information and the operation state information, determine the operation state of the tunneling equipment based on the operation matching information, and determine the human-machine action state of the tunneling equipment based on the personnel state and the operation state.

[0176] It should be noted that: the operating status evaluation device for underground mine tunnel excavation equipment provided in the above embodiment only uses the division of the above functional modules as an example when evaluating the operating status of the excavation equipment. In actual applications, the above functions can be assigned to different functional modules as needed, that is, the personnel structure of the computer equipment is divided into different functional modules to complete all or part of the functions described above. In addition, the operating status evaluation device for underground mine tunnel excavation equipment provided in the above embodiment and the operating status evaluation method embodiment of underground mine tunnel excavation equipment belong to the same concept. The specific implementation process is detailed in the method embodiment and will not be repeated here.

[0177] Through the technical solution provided in the embodiment of the present application, when the tunneling equipment is working, the equipment operating parameters of the tunneling equipment, the geological parameters of the underground mine in which it is located, the video in the tunneling direction, and the physiological parameters and operation set of the operator in the tunneling equipment are obtained. The equipment operating parameters, geological parameters and the video are used to determine the environmental action operating state of the tunneling equipment, thereby obtaining the operation of the tunneling equipment under the influence of the external environment. The human-machine action state of the tunneling equipment is determined using the equipment operating parameters, physiological parameters and the operation set, thereby obtaining the interaction between the operator and the tunneling equipment. In combination with the environmental action operating state and the human-machine action state, the equipment operating state of the tunneling equipment is determined, and the equipment operating state of the tunneling equipment is evaluated during the working process, abnormalities of the tunneling equipment are discovered in time, and the safety of operations using the tunneling equipment is improved.

[0178] Figure 5 It is a structural diagram of a computer device provided in an embodiment of the present application. The computer device can be implemented as the above-mentioned data processing device. The computer device 500 may have relatively large differences due to different configurations or performances, and may include one or more processors (Central Processing Units, CPU) 501 and one or more memories 502, wherein the one or more memories 502 store at least one computer program, and the at least one computer program is loaded and executed by the one or more processors 501 to implement the methods provided in the above-mentioned various method embodiments. Of course, the computer device 500 may also have components such as a wired or wireless network interface, a keyboard, and an input and output interface for input and output. The computer device 500 may also include other components for implementing device functions, which will not be repeated here.

[0179] In an exemplary embodiment, a computer-readable storage medium is also provided, such as a memory including a computer program, and the computer program can be executed by a processor to complete the operating status evaluation method of underground mine tunneling equipment in the above embodiment. For example, the computer-readable storage medium can be a read-only memory (ROM), a random access memory (RAM), a compact disc read-only memory (CD-ROM), a magnetic tape, a floppy disk, an optical parameter storage device, etc.

[0180] In an exemplary embodiment, a computer program product or a computer program is also provided, which includes a program code, and the program code is stored in a computer-readable storage medium. A processor of a computer device reads the program code from the computer-readable storage medium, and the processor executes the program code, so that the computer device executes the above-mentioned underground mine tunnel excavation equipment operation status assessment method.

[0181] In some embodiments, the computer program involved in the embodiments of the present application may be deployed and executed on a computer device, or on multiple computer devices located at one location, or on multiple computer devices distributed at multiple locations and interconnected by a communication network. Multiple computer devices distributed at multiple locations and interconnected by a communication network may constitute a blockchain system.

[0182] A person skilled in the art will understand that all or part of the steps to implement the above embodiments may be accomplished by hardware or by instructing related hardware through a program, and the program may be stored in a computer-readable storage medium, and the above-mentioned storage medium may be a read-only memory, a disk or an optical disk, etc.

[0183] The above are only optional embodiments of the present application and are not intended to limit the present application. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present application should be included in the protection scope of the present application.

Claims

1. A method for evaluating the operating status of underground mine tunneling equipment, characterized in that: The method comprises: When the tunneling equipment is working, obtaining equipment operating parameters of the tunneling equipment, geological parameters of the underground mine where the tunneling equipment is located, video in the tunneling direction of the tunneling equipment, and physiological parameters and operation sets of operators in the tunneling equipment; Based on the equipment operation parameters, the geological parameters and the video, determining the environmental effect operation state of the tunneling equipment, wherein the environmental effect operation state is used to indicate the operation status of the tunneling equipment under the influence of the external environment; Determining a human-machine interaction state of the tunneling equipment based on the equipment operation parameters, the physiological parameters and the operation set, wherein the human-machine interaction state is used to represent the interaction between the operator and the tunneling equipment; Based on the environmental effect operating state and the human-machine effect state of the tunneling equipment, the equipment operating state of the tunneling equipment is determined.

2. The method according to claim 1, characterized in that The video includes an environment video and a tunneling component video, and determining the environment effect operation state of the tunneling equipment based on the equipment operation parameters, the geological parameters and the video includes: Determining rock mass operation information of the tunneling equipment based on the equipment operation parameters and the geological parameters; Determine the state information of the excavation component of the excavation equipment based on the equipment operation parameters and the excavation component video, wherein the excavation component video is used to record the working condition of the excavation component of the excavation equipment; Determine excavation effect information of the excavation equipment based on the geological parameters and the environmental video, wherein the environmental video is used to record the excavation situation of the excavation equipment in the tunnel; Based on the rock mass action operation information, the tunneling component status information and the tunneling effect information, the environmental action operation status of the tunneling equipment is determined.

3. The method according to claim 2, characterized in that The equipment operation parameters include a temperature parameter set, a vibration parameter set, and working parameters of the tunneling component. The determining of rock mass operation information of the tunneling equipment based on the equipment operation parameters and the geological parameters includes: Determining the operating temperature operation information of the tunneling equipment based on the temperature parameter set, the geological parameters and the operating parameters, the temperature parameter set including the operating temperature of the tunneling assembly and the operating temperature of the engine of the tunneling equipment; Determining the working vibration operation information of the tunneling equipment based on the vibration parameter set, the geological parameters and the working parameters, the vibration parameter set including the vibration parameters of the tunneling assembly and the vibration parameters of the engine; Based on the working temperature operating information and the working vibration operating information, rock mass action operating information of the tunneling equipment is determined.

4. The method according to claim 2, characterized in that: The equipment operation parameters include working parameters and a set of strain parameters of the tunneling component, and determining the tunneling component state information of the tunneling equipment based on the equipment operation parameters and the tunneling component video includes: Performing video recognition on the tunneling component video to obtain a component surface state and a component connection state of the tunneling component, wherein the component surface state is used to represent the surface condition of the tunneling component; Based on the working parameters, the strain parameter set, the component surface state and the component connection state, the excavation component state information of the excavation equipment is determined, and the strain parameter set includes strains at multiple positions on the excavation component.

5. The method according to claim 2, characterized in that: The geological parameters include rock hardness, rock density and rock porosity. The determining of the tunneling effect information of the tunneling equipment based on the geological parameters and the environmental video includes: Performing video recognition on the environmental video to obtain the rock mass size and rock mass shape of the broken rock mass in the excavation direction of the excavation equipment; Determining size crushing effect information of the crushed rock mass based on the hardness of the rock mass, the porosity of the rock mass and the size of the rock mass; Determining shape crushing effect information of the crushed rock mass based on the rock mass density and the rock mass shape; Based on the size crushing effect information and the shape crushing effect information, excavation effect information of the excavation equipment is determined.

6. The method according to claim 1, characterized in that The determining the human-machine interaction state of the tunneling equipment based on the equipment operation parameters, the physiological parameters and the operation set includes: Determining physiological status information of an operator in the tunneling equipment based on the equipment operating parameters and the physiological parameters; Determining operation state information of the operator based on the physiological state information and the operation set, wherein the operation state information is used to indicate a controllable degree of the operation performed by the operator; Determining operation matching information of the tunneling equipment based on the equipment operation parameters and the operation set; Based on the physiological state information, the operation state information and the operation matching information, the human-machine interaction state of the tunneling equipment is determined.

7. The method according to claim 6, characterized in that The equipment operation parameters include cabin environment parameters, and determining the physiological state information of the operator in the tunneling equipment based on the equipment operation parameters and the physiological parameters includes: Determining environmental description information of a personnel cabin of the tunneling equipment based on the cabin environmental parameters, wherein the cabin environmental parameters include cabin vibration data, cabin temperature, cabin pressure, and cabin gas content in the personnel cabin; Based on the environmental description information and the physiological parameters, the physiological state information of the operator is determined.

8. The method according to claim 6, characterized in that The equipment operation parameters include movement parameters of the tunneling equipment and working parameters of the tunneling component, the operation set includes a movement operation set and a tunneling operation set, and determining the operation matching information of the tunneling equipment based on the equipment operation parameters and the operation set includes: Based on the movement parameters and the movement operation set, determining movement operation matching information of the tunneling device, wherein the movement operation matching information is used to indicate a matching condition between the movement operation and the movement parameters, and the movement operation is used to control the tunneling device to move; Determine, based on the working parameters and the tunneling operation set, tunneling operation matching information of the tunneling device, wherein the tunneling operation matching information is used to indicate a matching condition between the tunneling operation and the working parameters, and the tunneling operation is used to control the tunneling component; Based on the movement operation matching information and the excavation operation matching information, operation matching information of the excavation equipment is determined.

9. The method according to claim 6, characterized in that The determining the operation state information of the operator based on the physiological state information and the operation set includes: Determining the degree of physiological abnormality of the operator based on the physiological state information and reference physiological state information of the operator, wherein the reference physiological state information is determined based on the equipment operation parameters and the reference physiological parameters of the operator, and the reference physiological parameters are average physiological parameters of the operator when using the tunneling equipment; Determining the degree of abnormality of the operation of the operator based on the operation set and the operation mode of the operator, wherein the operation mode is used to represent the operation habits of the operator; Based on the degree of physiological abnormality and the degree of operational abnormality of the operator, the operation status information of the operator is determined.

10. The method according to claim 6, characterized in that The determining the human-machine interaction state of the tunneling equipment based on the physiological state information, the operation state information and the operation matching information includes: Determining the personnel status of the operator based on the physiological status information and the operating status information; Determining the operation state of the tunneling equipment based on the operation matching information; Based on the personnel status and the operation status, the human-machine interaction status of the tunneling equipment is determined.

Images