Method and device for processing mining intelligent helmet data based on APP mobile terminal

Abstract

The invention provides a method and device for processing mining intelligent helmet data based on an APP mobile terminal. The method comprises the following steps: in response to passing of identity verification of a wearer of the mining intelligent helmet, acquiring post type information of the wearer; displaying a user interface and a function module matched with the post type information of the wearer on a display interface of an APP mobile terminal associated with the mining intelligent helmet; acquiring sensor information acquired by the mining intelligent helmet; acquiring communication interaction information, wherein the communication interaction information is generated based on the sensor information in combination with artificial intelligence; and determining a presentation form of the communication interaction information according to the function module, and rendering the communication interaction information on the user interface according to the presentation form. According to the application, the operation steps of the wearer can be simplified, the universality of the mining intelligent helmet and the APP mobile terminal associated with the mining intelligent helmet is ensured, and different intelligent and personalized use experiences can be provided for wearers of different types of posts.

Description

Technical Field

[0001] The present application relates to the field of intelligent mining, and in particular to a method and device for processing data of smart mining helmets based on an APP mobile terminal. Background Art

[0002] The coal industry is a critical cornerstone of China's energy system, playing an irreplaceable role in ensuring national energy security, promoting economic development, and fostering social progress. However, with the acceleration of the global energy transition, the coal industry faces multiple challenges: improving production safety, optimizing resource utilization efficiency, and achieving green, low-carbon development. Traditional production methods, due to their high labor intensity, numerous safety hazards, and low management efficiency, are no longer able to meet the demands of modern mining construction.

[0003] Against this backdrop, the application of cutting-edge technologies such as intelligent systems, big data, and 5G has become crucial for the development of the coal industry. Intelligent technologies enhance automated operations, big data enables precise analysis and prediction, and 5G ensures efficient information transmission within and outside mines. As key nodes connecting frontline production personnel with these advanced systems, intelligent coal mine terminals are becoming increasingly important in improving mine safety, production efficiency, and intelligent management. Summary of the Invention

[0004] The embodiments of the present application provide a method and device for processing data of a mining smart helmet based on an APP mobile terminal.

[0005] According to a first aspect of an embodiment of the present application, a method for processing data of a mining smart helmet based on an APP mobile terminal is provided, comprising:

[0006] In response to the identity verification of the wearer of the mining smart helmet being passed, obtaining the job type information of the wearer;

[0007] On the display interface of the APP mobile terminal associated with the mining smart helmet, a user interface and function modules matching the job type information of the wearer are displayed;

[0008] Acquire sensor information collected by the mining smart helmet, the sensor information including at least surrounding environment image information, location information, physiological sign information of the wearer, posture information of the wearer, thermal infrared image information, gas sensor data, and environmental sensor data;

[0009] Acquiring communication interaction information, where the communication interaction information is generated by the APP mobile terminal or a cloud server communicating therewith based on the sensor information combined with artificial intelligence (AI) technology;

[0010] The presentation form of the communication interaction information is determined according to the functional module, and the communication interaction information is rendered on the user interface according to the presentation form.

[0011] According to a second aspect of an embodiment of the present application, a device for processing data of a mining smart helmet based on an APP mobile terminal is provided, comprising:

[0012] A first acquisition module is configured to acquire job type information of a wearer of the mining smart helmet in response to the wearer's identity verification being passed;

[0013] A display module is used to display a user interface and function modules matching the job type information of the wearer on the display interface of the APP mobile terminal associated with the mining smart helmet;

[0014] A second acquisition module is used to acquire sensor information collected by the mining smart helmet, wherein the sensor information includes at least surrounding environment image information, location information, physiological sign information of the wearer, posture information of the wearer, thermal infrared image information, gas sensor data, and environmental sensor data;

[0015] A third acquisition module is used to acquire communication interaction information, where the communication interaction information is generated by the APP mobile terminal or a cloud server communicating therewith based on the sensor information combined with artificial intelligence (AI) technology;

[0016] a determination module, configured to determine a presentation form of the communication interaction information according to the functional module;

[0017] A rendering module is used to render the communication interaction information on the user interface according to the presentation form.

[0018] According to a third aspect of an embodiment of the present application, there is provided an APP mobile terminal, including:

[0019] at least one processor;

[0020] a memory communicatively connected to the at least one processor; wherein,

[0021] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect.

[0022] According to a fourth aspect of an embodiment of the present application, a storage medium is provided, which stores instructions. When the instructions are executed on an APP mobile terminal, the APP mobile terminal executes the method described in the first aspect above.

[0023] According to a fifth aspect of an embodiment of the present application, it includes at least one of a program or an instruction, wherein when the at least one of the program or the instruction is executed by a processor, the steps of the method described in the first aspect are implemented.

[0024] According to the technical solution of the present application, based on the job type of the wearer of the mining smart helmet, a user interface and functional modules matching the job type can be displayed on the display interface of the APP mobile terminal, which can facilitate the wearer's use, avoid the user interface elements and functional modules from being too complex and causing complicated user operations, thereby simplifying the wearer's operating steps and greatly improving the wearer's experience. In addition, when generating corresponding communication interaction information based on the sensor information collected by the mining smart helmet, the communication interaction information can be rendered on the user interface in a presentation form corresponding to the functional module according to different functional modules, that is, the presentation form of the communication interaction information will be different for different functional modules, which can further improve the intelligence and personalization of the APP mobile terminal. While ensuring the versatility of the mining smart helmet and its associated APP mobile terminal, it can also provide different intelligent and personalized user experiences for wearers of different types of positions.

[0025] Additional aspects and advantages of the present application will be given in part in the description below, and in part will become apparent from the description below, or will be learned through practice of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0026] The above and / or additional aspects and advantages of the present application will become apparent and easily understood from the following description of the embodiments in conjunction with the accompanying drawings, in which:

[0027] Figure 1 A flow chart of a method for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application;

[0028] Figure 2 A flow chart of a method for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application;

[0029] Figure 3 A block diagram of a device for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application;

[0030] Figure 4 It is a block diagram of an APP mobile terminal according to an embodiment of the present application. DETAILED DESCRIPTION

[0031] The following describes in detail embodiments of the present application, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present application, and should not be construed as limiting the present application.

[0032] It should be noted that the collection, storage, use, processing, transmission, provision and disclosure of user personal information involved in the technical solution of this disclosure are in compliance with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0033] It is worth noting that the information (including but not limited to user device information, user personal information, etc.), data (including but not limited to data used for analysis, stored data, displayed data, etc.) and signals involved in this application are all authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0034] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned. They should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0035] In the context of intelligent coal mines, the coal industry's focus on underground mobile devices is changing. The performance and functions of the equipment itself are no longer the only focus. Instead, new directions such as platformization, networking, intelligence, and human-computer interaction are taking their place. Underground mobile devices no longer exist as just a single tool, but have become a key link between front-line workers and intelligent mine systems and mining command networks. To this end, this application uses underground mining smart helmets as a hardware platform. While realizing basic tool functions, it has added AR display, visible light / infrared audio and video communication, personnel identity authentication and navigation, environmental parameter detection and backtracking, and personnel physiological status monitoring functions to fully meet the life safety protection, efficient production, and management coordination needs of underground front-line personnel.

[0036] The following describes a method and device for processing mining smart helmet data based on an APP mobile terminal in an embodiment of the present application with reference to the accompanying drawings.

[0037] Figure 1 This is a flow chart of the method for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application. Figure 1 As shown, the method for processing mining smart helmet data based on the APP mobile terminal may include but is not limited to the following steps.

[0038] In step 101, in response to the identity verification of the wearer of the mining smart helmet being passed, the job type information of the wearer is obtained.

[0039] In some embodiments, the smart helmet for mining may include a variety of sensors, which may include but are not limited to: image acquisition devices (such as cameras), UWB modules, biosensors, environmental sensors, gas sensors, sensors for detecting posture (such as six-axis sensors or nine-axis sensors, etc.), infrared thermal imagers, etc. Among them, the image acquisition device can be used to collect image information of the surrounding environment. The UWB module can be used to interact with the base station to obtain location information to achieve the purpose of positioning. The biosensor may include but is not limited to a physiological sign detector for detecting the physiological characteristics of the wearer. Optionally, the biosensor may also include a fingerprint sensor. The environmental sensor can be used to detect the wearer's surrounding environment information (such as temperature and / or humidity information). The gas sensor can be used to detect the concentration of hazardous gases.

[0040] In some embodiments, each mining smart helmet can be associated with an APP mobile terminal. For example, the mining smart helmet and the APP mobile terminal can communicate via wired communication or wireless communication. The wireless communication method may include but is not limited to: WiFi, Bluetooth, etc.

[0041] In some embodiments, the APP mobile terminal has an identity login interface, and the wearer of the mining smart helmet can log in through the identity login interface. After the identity authentication is passed, the wearer can use the APP mobile terminal and the mining smart helmet.

[0042] In some embodiments, the mining smart helmet has a one-key login control. When the wearer of the mining smart helmet triggers the one-key login control, the wearer can log in through the sensor on the mining smart helmet (such as an image acquisition device and / or a biosensor, etc.), and after the identity authentication is passed, the wearer can use the APP mobile terminal and the mining smart helmet. For example, the image acquisition device on the mining smart helmet can be used to collect the wearer's facial image information, so that the mining smart helmet or the APP mobile terminal can authenticate the wearer based on the facial image information. For another example, the biosensor on the mining smart helmet can be used to collect the wearer's fingerprint image information, so that the mining smart helmet or the APP mobile terminal can authenticate the wearer based on the fingerprint image information.

[0043] In an embodiment of the present application, when the wearer's identity is verified, the wearer's job type information can be obtained based on the wearer's identity information. In an embodiment of the present application, the job type information may include but is not limited to production positions, maintenance and inspection positions, fixed positions, and fire and rescue positions. The wearers of production positions may be, for example, coal mining machine drivers, tunneling machine drivers, gas extraction workers, and bolt support workers. The wearers of maintenance and inspection positions may be, for example, gas inspectors, hydraulic support workers, maintenance electricians, and locomotive repairers. Fixed positions may be, for example, comprehensive mining and control operators, chute operators, and dispatchers in underground dispatch rooms. Fire and rescue positions may be, for example, mine rescue team members and fire prevention and extinguishing team members.

[0044] Optionally, in some embodiments, the mining smart helmet and / or mobile app terminal can support NFC login and authentication functions. For example, the mining smart helmet and / or mobile app terminal provides personnel authentication and login functions, calling the NFC module to quickly identify the wearer and collect and upload login information.

[0045] In step 102, a user interface and functional modules matching the wearer's job type information are displayed on the display interface of the APP mobile terminal associated with the mining smart helmet.

[0046] In some embodiments, based on the wearer's job type information, interface configuration information and functional configuration information that match the job type information can be selected from a preset database, and corresponding user interfaces and functional modules can be generated based on the interface configuration information and functional configuration information. The user interface and functional modules are displayed on the display interface of the APP mobile terminal; the functional modules may include general functional modules and specific functional modules associated with the wearer's job type information. The general functional modules may include, but are not limited to, login and identity authentication modules.

[0047] For example, if the wearer's job type is a production job, the specific functional modules associated with this job type may include, but are not limited to, a dust explosion detection module, and may also include a bone conduction voice communication module, wherein the bone conduction voice communication module may be based on network-based audio and video calls, or may be based on mobile data (such as 5G or 6G, etc.) mobile communication, etc. If the wearer's job type is a maintenance and inspection job, the specific functional modules associated with this job type may include, but are not limited to, a vital sign status detection module (including physiological characteristic monitoring and / or posture monitoring), and may also include a remote intelligent module (such as first-person infrared thermal imaging, mobile monitoring of underground equipment by the surface platform), and may also include a path navigation module (such as for UWB personnel positioning and / or escape routes, etc.). If the wearer's job type is a fixed job, the specific functional modules associated with this job type may include, but are not limited to, an optical waveguide AR display module (such as for full-view immersive display), and may include a device interface access module (such as for "one-touch connection" (such as NFC) of the device operation interface). The wearer's job type is a fire and rescue job. The specific functional modules associated with this job type may include, but are not limited to, an intercom functional module (such as that can be used for network-based wireless intercom), an infrared thermal imaging functional module (such as that used for thermal imaging identification of people in distress), and a path navigation module (such as UWB personnel positioning and / or escape route analysis). Optionally, the specific functional modules associated with this fire and rescue job may also include a one-button call for help functional module and / or a one-button broadcast functional module. The one-button call for help functional module can be used to call for help from the well surface, and the one-button broadcast functional module can broadcast text and / or voice alarms to all equipment underground (such as mining smart helmets, APP mobile terminals, or other devices that can receive broadcasts).

[0048] It is worth noting that production positions are characterized by high dust and high noise levels; maintenance and inspection positions are characterized by long moving distances and high work intensity; fixed positions are characterized by hazardous gas environments and low visibility; and firefighting and rescue positions are characterized by emergency rescue in complex environments. The embodiments of the present application can provide wearers of different types of positions with corresponding user interfaces and functional modules based on the pain points of different positions. That is, based on the wearer's position type, a corresponding personalized intelligent user interface and functional modules are provided, thereby facilitating the wearer's use and avoiding the complexity of user operations caused by excessive user interface elements and functional modules. This can simplify the wearer's operating steps and greatly enhance the wearer's user experience.

[0049] In step 103, sensor information collected by the mining smart helmet is obtained.

[0050] In an embodiment of the present application, the sensor information may include at least but not limited to surrounding image information, location information, physiological sign information of the wearer, posture information of the wearer, thermal infrared image information, gas sensor data, environmental sensor data, etc. The location information can be determined based on the interaction between the UWB module of the APP mobile terminal and the base station deployed in the mine. For example, a base station can be deployed in the mine, and the UWB module in the APP mobile terminal can interact with the base station to obtain the location information of the mining smart helmet associated with the APP mobile terminal.

[0051] In an embodiment of the present application, the mining smart helmet is communicatively connected with its associated APP mobile terminal. The mining smart helmet collects corresponding data through its own multiple sensors and can send the collected sensor information to the APP mobile terminal, so that the APP mobile terminal can perform subsequent processing based on the sensor information.

[0052] In step 104, communication interaction information is obtained.

[0053] In some embodiments, the communication interaction information is generated by the APP mobile terminal or the cloud server communicating therewith based on sensor information combined with artificial intelligence (AI) technology.

[0054] In one possible implementation, the communication interaction information can be generated by an app mobile terminal based on sensor information combined with AI technology. For example, the app mobile terminal can deploy an AI algorithm. Upon receiving sensor information collected by the mining smart helmet, the app mobile terminal can process the sensor information using the AI ​​algorithm and generate corresponding communication interaction information based on the processing results. For example, the communication interaction information can be information related to emergency risk avoidance events, but is not limited to this.

[0055] In another possible implementation, the communication interaction information can be generated by a cloud server based on sensor information combined with AI technology. For example, the cloud server can deploy an AI algorithm. An app-based mobile terminal receives sensor information collected by the mining smart helmet and transmits this sensor information to the cloud server. The cloud server can then process this sensor information using the AI ​​algorithm and generate corresponding communication interaction information based on the processing results. For example, this communication interaction information can be related to emergency risk avoidance events, but is not limited to this.

[0056] For example, taking the case where sensor information includes surrounding environment image information and the AI ​​algorithm may include a landslide prediction neural network model, the landslide warning neural network model may be used to predict landslides on the surrounding environment image information. If the landslide prediction result output by the landslide warning neural network model is that a landslide may occur (such as the predicted probability of occurrence is greater than a certain threshold), communication interaction information matching the prediction result may be generated, such as relevant information for landslide warning, such as the location, area, severity, probability of occurrence, and countdown prediction value for the occurrence of the landslide.

[0057] For another example, taking the case where the sensor information includes the wearer's physiological signs information, the AI ​​algorithm may include a body state analysis model. The body state analysis model may be used to predict the body state of the wearer's physiological signs information. If the prediction result output by the body state analysis model is that an abnormal body state may occur, communication interaction information matching the prediction result may be generated, such as relevant information regarding the abnormal body state, such as the specific abnormal body state that may occur, its severity, and mitigation methods.

[0058] It should be noted that the two examples of implementation methods for generating communication interaction information given above are only for the convenience of those skilled in the art to understand how to generate communication interaction information based on sensor information combined with AI technology, and cannot be simply understood as the only two examples of implementation methods for generating communication interaction information. In other words, the specific functions of the AI ​​algorithm can be determined based on the actual business scenarios and actual business needs of the smart helmet for mining, and then the sensor information collected by the smart helmet for mining can be processed based on the specific functions of the AI ​​algorithm to generate corresponding communication interaction information.

[0059] In step 105, the presentation form of the communication interaction information is determined according to the functional module, and the communication interaction information is rendered on the user interface according to the presentation form.

[0060] In the embodiment of the present application, different job types may correspond to different functional modules; for different functional modules, the communication interaction information may have different presentation forms.

[0061] For example, the wearer's job type is a production job. The specific functional modules associated with this job type may include but are not limited to a dust explosion detection module, and may also include a bone conduction voice communication module, wherein the bone conduction voice communication module may be based on network audio and video calls, or may be based on mobile data (such as 5G or 6G, etc.) mobile communications, etc., then the presentation form of the communication interaction information may be voice or video, that is, the communication interaction information is presented in the form of voice or video through the bone conduction voice communication module.

[0062] Exemplarily, the wearer's job type is a maintenance and inspection job. The specific functional modules associated with this job type may include but are not limited to a vital sign status detection module (including physiological characteristic monitoring and / or posture monitoring), and may also include a remote intelligent module (such as first-person infrared thermal imaging, mobile monitoring of underground equipment by the surface platform), and may also include a path navigation module (such as for UWB personnel positioning and / or escape routes, etc.). If the communication interaction information includes infrared thermal imaging, the communication interaction information may be presented in the form of infrared thermal imaging; if the communication interaction information includes an escape route map, the communication interaction information may be presented in the form of map navigation; but it is not limited to this.

[0063] For example, the wearer's job type is a fixed job, and the specific functional modules associated with this job type may include but are not limited to an optical waveguide AR display module (such as for full-view immersive display), and may include a device interface access module (such as for a "touch connection" of the device operation interface (such as NFC)), then the communication interaction information can be presented based on AR technology.

[0064] For example, if the wearer's job type is a fire rescue job, the specific functional modules associated with this job type may include, but are not limited to, an intercom function module (such as a network-based wireless intercom), an infrared thermal imaging function module (such as for thermal imaging identification of people in distress), and a path navigation module (such as UWB personnel positioning and / or escape route analysis). If the communication interaction information includes an escape route map, the communication interaction information may be presented in the form of map navigation; but this is not limited to this.

[0065] In the above embodiment, based on the job type of the wearer of the smart mining helmet, a user interface and functional modules matching the job type can be displayed on the display interface of the APP mobile terminal, which can facilitate the wearer's use, avoid the user interface elements and functional modules from being too complex and causing complicated user operations, thereby simplifying the wearer's operating steps and greatly improving the wearer's experience. In addition, when generating corresponding communication interaction information based on the sensor information collected by the smart mining helmet, the communication interaction information can be rendered on the user interface in a presentation form corresponding to the functional module according to different functional modules, that is, the presentation form of the communication interaction information will be different for different functional modules, which can further improve the intelligence and personalization of the APP mobile terminal. While ensuring the versatility of the smart mining helmet and its associated APP mobile terminal, it can also provide different intelligent and personalized user experiences for wearers of different types of positions.

[0066] Figure 2This is a flow chart of the method for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application. Figure 2 As shown, the method for processing mining smart helmet data based on the APP mobile terminal may include but is not limited to the following steps.

[0067] In step 201, in response to the identity verification of the wearer of the mining smart helmet being passed, the job type information of the wearer is obtained.

[0068] Optional implementations of step 201 can be found in Figure 1 Optional implementation of step 101, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0069] In step 202, a user interface and functional modules matching the wearer's job type information are displayed on the display interface of the APP mobile terminal associated with the mining smart helmet.

[0070] Optional implementations of step 202 can be found in Figure 1 Optional implementation of step 102, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0071] In step 203, sensor information collected by the mining smart helmet is obtained.

[0072] Optional implementations of step 203 can be found in Figure 1 Optional implementation of step 103, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0073] In step 204, communication interaction information is obtained.

[0074] In some embodiments, the communication interaction information is generated by the APP mobile terminal or the cloud server communicating therewith based on sensor information combined with AI technology.

[0075] Optional implementations of step 204 can be found in Figure 1 Optional implementation of step 104, and Figure 1 Other related parts in the embodiments involved will not be described in detail here.

[0076] In step 205, the urgency of the communication event associated with the communication interaction information is determined.

[0077] In some embodiments, when the APP mobile terminal or cloud server generates communication interaction information based on sensor information combined with AI technology, the purpose of this communication interaction can be determined, and the purpose of this communication interaction can be understood as a communication event associated with the communication interaction information. Based on the actual business scenarios and actual business needs of the smart helmet for mining, the corresponding AI algorithm can be used to process the sensor information. When generating the corresponding communication interaction information, the purpose and urgency of this communication interaction can also be determined. Among them, the urgency can be determined based on the processing results of the sensor information by the AI ​​algorithm. For example, taking the use of the AI ​​algorithm to process the sensor information and predicting that a landslide may occur as an example, the urgency of this communication event can be highly urgent (such as the highest level).

[0078] In step 206, the presentation form of the communication interaction information is determined according to the urgency and the functional module.

[0079] In some embodiments, the communication interaction information can be presented in the form of text, voice, or video. In some embodiments, the communication interaction information can also be presented in a confirmed form or a non-confirmation form. Non-confirmation communication interaction information can be understood as communication interaction information that does not require the wearer to manually confirm its acceptance. Confirmation communication interaction information can be understood as communication interaction information that requires the wearer to manually confirm its acceptance.

[0080] In some embodiments, when the urgency level is a first urgency level and the functional module includes a system notification functional module, the communication interaction information is determined to be presented in text; or, when the urgency level is a second urgency level and the functional module includes an audio and video control module, the communication interaction information is determined to be presented in voice or video; or, when the urgency level is a third urgency level and the functional module includes a system notification functional module and an audio and video control module, the communication interaction information is determined to be presented in at least two of text, voice, and video. The first urgency level is less than the second urgency level, and the second urgency level is less than the third urgency level. The voice or video is generated by the voice and video control module based on the communication interaction information.

[0081] Exemplarily, when the urgency level is the second urgency level and the functional module includes an audio and video control module, if the communication interaction information is text information, the text can be converted into corresponding voice or video through the audio and video control module. For example, the text can be converted into corresponding voice based on TTS speech synthesis technology, or the text can be converted into corresponding video based on a video generation model, so that the communication interaction information is presented in the form of voice or video. If the communication interaction information is voice or video, the communication interaction information can be directly rendered on the user interface in the form of voice or video through the audio and video control module.

[0082] Exemplarily, when the urgency level is the third urgency level and the functional module includes a system notification functional module and an audio and video control module, if the type of the communication interaction information is text information, the text can be converted into corresponding voice or video through the audio and video control module, and the presentation form of the communication interaction information can be determined to be text + voice, or the presentation form of the communication interaction information can be determined to be text + video.

[0083] In step 207, the communication interaction information is rendered on the user interface according to the presentation format.

[0084] By implementing the embodiments of the present application, the presentation form of the communication interaction information can be determined based on the urgency of the communication event and the functional module associated with the communication interaction information, so that the communication interaction information can be rendered in this presentation form on the user interface, thereby effectively providing early warning alarms, and further effectively eliminating safety hazards, thereby improving the management efficiency and success rate of emergency events.

[0085] Optionally, in some embodiments, when the emergency warning event associated with the communication event is a specific warning event, an emergency avoidance route can be obtained. The emergency avoidance route can be generated by the APP mobile terminal or the cloud server based on the sensor information collected by multiple mining smart helmets in the mine combined with AI technology; on the display interface of the APP mobile terminal, the emergency avoidance route is displayed based on AR technology to guide the wearer to a safe area.

[0086] For example, based on the collected location information and / or image information of multiple mining smart helmets, combined with the current mine map and the helmet dispatch area of ​​the emergency warning event, an emergency avoidance route for the currently scheduled mining smart helmet can be generated, and the emergency avoidance route can be displayed on the display interface of the APP mobile terminal associated with the scheduled mining smart helmet based on AR technology to guide the wearer to a safe area. Among them, the scheduled mining smart helmet can be understood as a mining smart helmet that obtains communication interaction information, and the helmet dispatch area of ​​the emergency warning event can be understood as the area where the scheduled mining smart helmet is located.

[0087] For example, suppose there are areas A, B, and C in a mine. There are three workers in area A (e.g., a1, a2, and a3 can be used to represent the three workers in area A) and they all wear smart mining helmets. There are two workers in area B (e.g., b1 and b2 can be used to represent the two workers in area B) and they all wear smart mining helmets. The smart mining helmet worn by worker a1 determines that the emergency warning event triggered is a specific warning event based on the image acquisition device and / or gas sensor on the helmet. The helmet dispatch area can be determined to be area A, and the safe area can be determined to be area B. On the display interface of the APP mobile terminal associated with the smart mining helmet worn by worker a1, an emergency avoidance route can be displayed based on AR technology to guide worker a1 to the safe area B.

[0088] Optionally, in some embodiments, the mining smart helmet and / or its associated APP mobile terminal may have a one-button broadcast function module. When a specific warning event is determined to have occurred and a trigger operation for the one-button broadcast function module is received by the wearer, a text and / or voice broadcast warning can be sent to other devices in the helmet dispatch area where the mining smart helmet is located (such as the mining smart helmet, APP mobile terminal, or other devices that can receive broadcasts). In this way, the safety of all personnel in the mine can be further guaranteed.

[0089] Optionally, in some embodiments, it is determined that the login identity of the APP mobile terminal associated with the mining smart helmet has switched; based on the user interface and function module that matches the wearer's job type information after the login identity is switched, the user interface and function module on the display interface of the APP mobile terminal are seamlessly switched. In other words, when other wearers wear the mining smart helmet and the login identity authentication of the other wearers is passed, the corresponding user interface and function module can be matched based on the job type of the other wearers, and then the user interface and function module on the display interface of the APP mobile terminal are seamlessly switched. That is: for the same mining smart helmet, the user interface and function module on the display interface of the APP mobile terminal can be seamlessly switched based on the job type of different wearers, so as to adapt to the wearing of different staff members, improve the usability and adaptability of the equipment, and at the same time avoid the user interface elements and function modules from being too complicated, thereby simplifying the wearer's operation steps and greatly improving the wearer's experience.

[0090] Optionally, in some embodiments, the wearer's usage data for the APP mobile terminal and / or the mining smart helmet before the login identity switch, and the wearer's usage data for the APP mobile terminal and / or the mining smart helmet after the login identity switch, are stored in their respective independent spaces for data isolation. This can protect data integrity and security, prevent data leakage and abuse, ensure sustainable data development, and improve concurrency performance.

[0091] Optionally, in some embodiments, when the APP mobile terminal determines that the wearer has read the communication interaction information, it can send the read status of the communication interaction information to a cloud server (such as a dispatch center) to facilitate the safe tracking of the wearer.

[0092] Figure 3 This is a block diagram of a device for processing mining smart helmet data based on an APP mobile terminal provided in an embodiment of the present application. Figure 3 As shown, the device for processing mining smart helmet data based on the APP mobile terminal may include a first acquisition module 301, a display module 302, a second acquisition module 303, a third acquisition module 304, a determination module 305 and a rendering module 306.

[0093] The first acquisition module 301 is configured to acquire the job type information of the wearer in response to the identity verification of the wearer of the mining smart helmet being passed.

[0094] The display module 302 is used to display a user interface and functional modules that match the wearer's job type information on the display interface of the APP mobile terminal associated with the mining smart helmet.

[0095] The second acquisition module 303 is configured to acquire sensor information collected by the mining smart helmet. The sensor information includes at least surrounding image information, location information, wearer's physiological information, wearer's posture information, thermal infrared image information, gas sensor data, and environmental sensor data. In some embodiments, the location information is determined based on the interaction between the UWB module of the mobile terminal of the app and a base station deployed in the mine.

[0096] The third acquisition module 304 is used to obtain communication interaction information. The communication interaction information is generated by the APP mobile terminal or the cloud server communicating with it based on sensor information combined with artificial intelligence (AI) technology.

[0097] The determination module 305 is configured to determine the presentation form of the communication interaction information according to the functional module.

[0098] The rendering module 306 is configured to render the communication interaction information on the user interface according to the presentation format.

[0099] In some embodiments, the display module 302 is used to: based on the wearer's job type information, select interface configuration information and function configuration information that match the job type information from a preset database; generate corresponding user interfaces and function modules according to the interface configuration information and function configuration information; display the user interface and function modules on the display interface of the APP mobile terminal; wherein the function modules include general function modules and specific function modules associated with the job type information.

[0100] In some embodiments, the determination module 305 is used to: determine the urgency of the communication event associated with the communication interaction information; and determine the presentation form of the communication interaction information based on the urgency and the functional module.

[0101] In some embodiments, the determination module 305 is used to: when the urgency level is the first urgency level and the functional module includes a system notification functional module, determine that the presentation form of the communication interaction information is text; or, when the urgency level is the second urgency level and the functional module includes an audio and video control module, determine that the presentation form of the communication interaction information is voice or video; or, when the urgency level is the third urgency level and the functional module includes a system notification functional module and an audio and video control module, determine that the presentation form of the communication interaction information is at least two of text, voice, and video; wherein the first urgency level is less than the second urgency level, the second urgency level is less than the third urgency level, and the voice or video is generated by the voice and video control module based on the communication interaction information.

[0102] In some embodiments, the device for processing mining smart helmet data based on an APP mobile terminal may also include a fourth acquisition module. Specifically, the fourth acquisition module is configured to obtain an emergency escape route when the emergency warning event associated with the communication event is a specific warning event. The emergency escape route is generated by the APP mobile terminal or a cloud server based on sensor information collected by multiple mining smart helmets in the mine and combined with AI technology. The rendering module 306 is further configured to display the emergency escape route on the display interface of the APP mobile terminal using AR technology to guide the wearer to a safe area.

[0103] In some embodiments, the apparatus for processing mining smart helmet data based on an APP mobile terminal may further include a switching module. The switching module is configured to: determine whether a login identity of the APP mobile terminal associated with the mining smart helmet has switched; and seamlessly switch the user interface and functional modules on the display interface of the APP mobile terminal based on the user interface and functional modules that match the job type information of the wearer after the login identity switch.

[0104] In some embodiments, the device for processing mining smart helmet data based on an APP mobile terminal may further include a storage module. The storage module is configured to store the wearer's usage data for the APP mobile terminal and / or the mining smart helmet before the login identity is switched, and the wearer's usage data for the APP mobile terminal and / or the mining smart helmet after the login identity is switched, in corresponding independent spaces for data isolation.

[0105] It should be noted that the above explanation of the method embodiment for processing data of a smart helmet for mining based on an APP mobile terminal is also applicable to the device for processing data of a smart helmet for mining based on an APP mobile terminal in this embodiment, and will not be repeated here.

[0106] In some embodiments, both the APP mobile terminal and the mining smart helmet can belong to the devices in the mining smart helmet control system. This application can adopt the device-server architecture, and the device side (such as the APP mobile terminal) is turned on and off to register and cancel on the server side (such as the cloud server in this article). The server side connects to other systems that need to be coordinated, and the device side only interacts with the server side of this system. The management personnel can operate the server on the WEB side for management. For example, the device-side functional modules may include a personnel login module, a personnel positioning module, a personnel physiological status monitoring module, a helmet equipment management module, an AR display management module, an audio and video communication module, an alarm module, etc.

[0107] The personnel login module provides personnel identity authentication and login functionality, invoking the NFC module to quickly identify users and collect and upload login information. The personnel positioning module uses the device's UWB module to interact with the base station, collecting and uploading real-time location information. The personnel physiological status monitoring module monitors and uploads the wearer's physiological parameters in real time, triggering alarms in abnormal situations. The helmet device management module is responsible for hardware status monitoring and functional control of the helmet device, including battery power, headlight, and various sensor management. The AR display management module controls the content of the AR display device, displaying mission information, alarm prompts, and sensor data. The audio and video communication module supports real-time visible light / infrared thermal imaging audio and video calls and message transmission, enabling efficient communication up and down the well. The alarm module is responsible for receiving and processing emergency communications and alarms initiated by the server, triggering audible and visual alarms and returning user-read information to the server.

[0108] Exemplarily, the functional modules of the server side may include personnel management, personnel location management, personnel physiological status management, helmet equipment management, audio and video communication management, and alarm services. Among them, personnel management can maintain basic personnel information and login status, and manage personnel equipment permissions and activity areas. Personnel location management can receive and display the location information of underground personnel in real time, providing support for emergency response and command and dispatch. Personnel physiological status management can record and analyze personnel physiological data, handle abnormal conditions in a timely manner, and provide historical data query. Helmet equipment management can monitor the operating status of helmet equipment, support remote diagnosis and fault handling. Audio and video communication management can handle audio and video communication requests between device ends and between device ends and control center, ensuring efficient multi-terminal connection and message transmission. The alarm service can initiate text or sound alarms for one or more helmet devices, and track and confirm that the user has read them.

[0109] According to an embodiment of the present application, the present application also provides an electronic device and a readable storage medium.

[0110] like Figure 4 , is a block diagram of an APP mobile terminal according to an embodiment of the present application. The APP mobile terminal is intended to represent various forms of digital computers, such as workstations and personal digital assistants. The APP mobile terminal can also represent various forms of mobile devices, such as personal digital processing, cellular phones, smart phones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely examples and are not intended to limit the implementation of the present application described and / or required herein.

[0111] like Figure 4 As shown, the APP mobile terminal includes: one or more processors 401, a memory 402, and interfaces for connecting various components, including high-speed interfaces and low-speed interfaces. The various components are interconnected using different buses and can be installed on a common motherboard or installed in other ways as needed. The processor can process instructions executed in the APP mobile terminal, including instructions stored in or on the memory to display graphical information of the GUI on an external input / output device (such as a display device coupled to the interface). In other embodiments, if necessary, multiple processors and / or multiple buses can be used together with multiple memories and multiple memories. Similarly, multiple APP mobile terminals can be connected, and each device provides some necessary operations (for example, a multi-processor system). Figure 4 A processor 401 is taken as an example.

[0112] Memory 402 is the non-transitory computer-readable storage medium provided in this application. The memory stores instructions executable by at least one processor, causing the at least one processor to execute the method for processing data of a smart mining helmet based on an APP mobile terminal provided in this application. The non-transitory computer-readable storage medium of this application stores computer instructions for causing a computer to execute the method for processing data of a smart mining helmet based on an APP mobile terminal provided in this application.

[0113] The memory 402 is a non-transient computer-readable storage medium that can be used to store non-transient software programs, non-transient computer executable programs and modules, such as the program instructions / modules corresponding to the method for processing mining smart helmet data based on the APP mobile terminal in the embodiment of the present application (for example, the attached Figure 3 The processor 401 executes various functional applications and data processing of the server by running the non-transient software programs, instructions, and modules stored in the memory 402, thereby implementing the method for processing mining smart helmet data based on the APP mobile terminal in the above method embodiment.

[0114] The memory 402 may include a program storage area and a data storage area, wherein the program storage area may store an operating system, an application required for at least one function; the data storage area may store data created according to the use of the APP mobile terminal, etc. In addition, the memory 402 may include a high-speed random access memory, and may also include a non-transient memory, such as at least one disk storage device, a flash memory device, or other non-transient solid-state storage device. In some embodiments, the memory 402 may optionally include a memory remotely located relative to the processor 401, and these remote memories may be connected to the APP mobile terminal via a network. Examples of the above-mentioned network include, but are not limited to, the Internet, an intranet, a local area network, a mobile communication network, and a combination thereof.

[0115] The APP mobile terminal may further include: an input device 403 and an output device 404. The processor 401, the memory 402, the input device 403 and the output device 404 may be connected via a bus or other means. Figure 4 The bus connection is taken as an example.

[0116] The input device 403 can receive input digital or character information, and generate key signal input related to user settings and function control of the APP mobile terminal, such as a touch screen, a keypad, a mouse, a trackpad, a touchpad, an indicator stick, one or more mouse buttons, a trackball, a joystick and other input devices. The output device 404 may include a display device, an auxiliary lighting device (e.g., an LED) and a tactile feedback device (e.g., a vibration motor), etc. The display device may include, but is not limited to, a liquid crystal display (LCD), a light emitting diode (LED) display and a plasma display. In some embodiments, the display device may be a touch screen.

[0117] Various implementations of the systems and techniques described herein can be realized in digital electronic circuit systems, integrated circuit systems, dedicated ASICs (application specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include being implemented in one or more computer programs that can be executed and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0118] These computer programs (also referred to as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented using high-level procedural and / or object-oriented programming languages, and / or assembly / machine languages. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., a magnetic disk, an optical disk, a memory, a programmable logic device (PLD)) for providing machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal for providing machine instructions and / or data to a programmable processor.

[0119] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0120] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer with a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include: a local area network (LAN), a wide area network (WAN), the Internet, and a blockchain network.

[0121] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact via a communication network. This client-server relationship is established by computer programs running on the respective computers, establishing a client-server relationship. The server may be a cloud server, also known as a cloud computing server or cloud host, a host product within the cloud computing service ecosystem that addresses the management difficulties and limited scalability of traditional physical hosts and VPS services ("Virtual Private Servers" or simply "VPS"). The server may also be a server in a distributed system or a server integrated with blockchain.

[0122] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this application can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this application can be achieved. This is not a limitation herein.

[0123] The above specific embodiments do not constitute a limitation on the scope of protection of this application. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this application shall be included within the scope of protection of this application.

Claims

1. A method for processing mining smart helmet data based on an APP mobile terminal, characterized in that: include: In response to the identity verification of the wearer of the mining smart helmet being passed, obtaining the job type information of the wearer; On the display interface of the APP mobile terminal associated with the mining smart helmet, a user interface and function modules matching the job type information of the wearer are displayed; Acquire sensor information collected by the mining smart helmet, the sensor information including at least surrounding environment image information, location information, physiological sign information of the wearer, posture information of the wearer, thermal infrared image information, gas sensor data, and environmental sensor data; Acquiring communication interaction information, where the communication interaction information is generated by the APP mobile terminal or a cloud server communicating therewith based on the sensor information combined with artificial intelligence (AI) technology; The presentation form of the communication interaction information is determined according to the functional module, and the communication interaction information is rendered on the user interface according to the presentation form.

2. The method according to claim 1, wherein The display interface of the APP mobile terminal associated with the mining smart helmet displays a user interface and functional modules that match the wearer's job type information, including: Based on the job type information of the wearer, selecting interface configuration information and function configuration information matching the job type information from a preset database; Generate a corresponding user interface and functional modules according to the interface configuration information and the functional configuration information; The user interface and the functional modules are displayed on the display interface of the APP mobile terminal; wherein the functional modules include general functional modules and specific functional modules associated with the position type information.

3. The method according to claim 1, wherein The determining of the presentation form of the communication interaction information according to the functional module includes: determining an urgency of a communication event associated with the communication interaction information; The presentation form of the communication interaction information is determined according to the urgency and the functional module.

4. The method according to claim 3, wherein The determining, based on the urgency and the functional module, of a presentation form of the communication interaction information includes: When the urgency level is the first urgency level and the functional module includes a system notification functional module, determining that the presentation form of the communication interaction information is text; or When the urgency level is the second urgency level and the functional module includes an audio and video control module, determining that the presentation form of the communication interaction information is voice or video; or When the urgency level is the third urgency level and the functional module includes the system notification functional module and the audio and video control module, determining that the presentation form of the communication interaction information is at least two of text, voice, and video; The first urgency level is less than the second urgency level, the second urgency level is less than the third urgency level, and the voice or video is generated by the voice and video control module based on the communication interaction information.

5. The method according to claim 3 or 4, wherein: The method further comprises: When the emergency warning event associated with the communication event is a specific warning event, an emergency avoidance route is obtained, where the emergency avoidance route is generated by the APP mobile terminal or the cloud server based on sensor information collected by multiple mining smart helmets in the mine in combination with AI technology; On the display interface of the APP mobile terminal, the emergency avoidance route is displayed based on AR technology to guide the wearer to a safe area.

6. The method according to claim 1, wherein The location information is determined based on the interaction between the UWB module of the APP mobile terminal and a base station deployed in the mine.

7. The method according to claim 1, wherein The method further comprises: Determining that a login identity of an APP mobile terminal associated with the mining smart helmet has been switched; Based on the user interface and functional modules that match the wearer's job type information after the login identity is switched, the user interface and functional modules on the display interface of the APP mobile terminal are seamlessly switched.

8. The method according to claim 7, wherein The method further comprises: The usage data of the wearer for the APP mobile terminal and / or the mining smart helmet before the login identity is switched, and the usage data of the wearer for the APP mobile terminal and / or the mining smart helmet after the login identity is switched, are stored in their respective corresponding independent spaces for data isolation.

9. A device for processing mining smart helmet data based on an APP mobile terminal, characterized in that: include: A first acquisition module is configured to acquire job type information of a wearer of the mining smart helmet in response to the wearer's identity verification being passed; A display module is used to display a user interface and function modules matching the job type information of the wearer on the display interface of the APP mobile terminal associated with the mining smart helmet; A second acquisition module is used to acquire sensor information collected by the mining smart helmet, wherein the sensor information includes at least surrounding environment image information, location information, physiological sign information of the wearer, posture information of the wearer, thermal infrared image information, gas sensor data, and environmental sensor data; A third acquisition module is used to acquire communication interaction information, where the communication interaction information is generated by the APP mobile terminal or a cloud server communicating therewith based on the sensor information combined with artificial intelligence (AI) technology; a determination module, configured to determine a presentation form of the communication interaction information according to the functional module; A rendering module is used to render the communication interaction information on the user interface according to the presentation form.

10. An APP mobile terminal, characterized in that: include: at least one processor; a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 8.

Images