Unsafe person identification method and device based on AR glasses, equipment and medium

Abstract

The invention provides an unsafe person recognition method, device and equipment based on AR glasses and a medium, and the method comprises a server side, a client side and a webpage side, and the method comprises the steps: obtaining face image information through the client side, and transmitting the face image information to the server side; the server side carries out face recognition on the face image information according to a face image information database and sends a recognition result to the webpage side; and the webpage end carries out visual processing on the identification result, and sends a visual processing result to the client for display, so that the problem that a thermal power plant security administrator cannot discriminate personnel information one by one completely and cannot form a systematic and normative record is solved.

Description

Technical Field

[0001] The present invention relates to the field of augmented reality technology, and in particular to an unsafe person identification method, device, equipment and medium based on AR glasses. Background Art

[0002] Safety management is the top priority of thermal power plant operation management. In addition to routine safety control and safety system development, safety inspectors are also required to patrol and inspect the plant regularly to ensure early detection or avoidance of various safety issues. Usually, safety inspectors identify unsafe personnel or unsafe behaviors with their naked eyes in the plant, but due to the large number of personnel in the plant and the frequent entry of personnel from outsourced units, safety managers are unable to fully identify personnel information one by one, telephone communication efficiency is also very low, and once unsafe personnel information or behavior is discovered, there is no way to form a systematic and standardized record.

[0003] AR (Augmented Reality) is an advanced technology that cleverly integrates virtual information with the real world. It provides a new immersive experience by superimposing computer-generated images, sounds, touch and other sensory information in real scenes. By wearing advanced AR glasses, safety management, maintenance management and remote calls can be performed in the complex environment of thermal power plants, which can significantly and comprehensively improve the efficiency and accuracy of daily management work in thermal power plants.

[0004] Therefore, it is urgent to propose a method for identifying unsafe personnel based on AR glasses to solve the technical problem that safety managers in thermal power plants cannot fully identify personnel information one by one and cannot form systematic and standardized records. Summary of the invention

[0005] In order to overcome the problems existing in the related art, the present disclosure provides a method, device, equipment and medium for identifying unsafe personnel based on AR glasses, so as to solve the technical problem in the related art that the safety manager in the thermal power plant is unable to completely identify the personnel information one by one and cannot form a systematic and standardized record.

[0006] One or more embodiments of this specification provide a method for identifying unsafe persons based on AR glasses, including a server, a client, and a web page. The method includes the following steps:

[0007] Acquire facial image information through the client, and transmit the facial image information to the server;

[0008] The server performs face recognition on the face image information according to the face image information database, and sends the recognition result to the web page end;

[0009] The web - end visualizes the recognition result and sends the visualized result to the client for display.

[0010] Preferably, the server performs face recognition on the face image information according to the face image information database, which specifically includes the following steps:

[0011] The server performs batch face recognition on the face image information through the XSSFWorkbook of the POI component. Among them, the face image information is converted into tabular data in the client and then transmitted to the server;

[0012] Store the recognition result in the face image information database.

[0013] Preferably, it further includes the following steps:

[0014] Transmit the personnel information data in the EPR system to the server to update the face image information database of the server.

[0015] Preferably, it further includes the following steps:

[0016] Obtain other image information through the client, identify the unsafe behaviors included in the other image information through the server, and display the unsafe behaviors on the web - end.

[0017] One or more embodiments of this specification provide a recognition device for unsafe personnel based on AR glasses, including a server, a client, and a web - end. The device includes an image acquisition module, a recognition module, and a visualization module;

[0018] The image acquisition module is used to obtain face image information through the client and transmit the face image information to the server;

[0019] The recognition module is used for the server to perform face recognition on the face image information according to the face image information database and send the recognition result to the web - end;

[0020] The visualization module is used for the web - end to visualize the recognition result and send the visualized result to the client for display.

[0021] Preferably, the recognition module is further configured for the server to perform batch face recognition on the face image information through the XSSFWorkbook of the POI component. Among them, after the face image information is converted into tabular data in the client, it is transmitted to the server;

[0022] Store the recognition result in the face image information database.

[0023] Preferably, the recognition module is further configured such that the server performs batch face recognition on the face image information through the XSSFWorkbook of the POI component, wherein the face image information is converted into tabular data in the client and then transmitted to the server;

[0024] The recognition result is stored in the face image information database.

[0025] Preferably, it further includes an unsafe behavior recognition module, which is used to obtain other image information through the client, recognize the unsafe behavior included in the other image information through the server, and display the unsafe behavior on the web page.

[0026] One or more embodiments of this specification provide a computer device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, it implements the method for recognizing unsafe personnel based on an AR glasses as described above.

[0027] One or more embodiments of this specification provide a computer-readable storage medium. The computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, it implements the steps of the method for recognizing unsafe personnel based on an AR glasses as described above.

[0028] The method, device, equipment, and medium for recognizing unsafe personnel based on an AR glasses provided by the present disclosure have the following advantages: By obtaining face image information through the client, it can capture the face picture in real time in the actual scene where the wearer is located, ensuring the acquisition of the latest on-site face image data at present. Transmitting the face image information to the server, through establishing a stable transmission link, the data can be sent out quickly and reliably, avoiding data accumulation or loss in the client. The server performs face recognition on the face image information according to the face image information database, accurately distinguishing whether it is an unsafe person, and sending the recognition result to the web page, which can accurately judge the identity of the person corresponding to the acquired face image, reducing the probability of misjudgment and missed judgment. The web page performs visualization processing on the recognition result, converting the abstract recognition and judgment result into an intuitive and easy-to-understand form, and sending the visualization processing result to the client for display, realizing the fusion of virtual recognition information and the real on-site scene in the wearer's field of vision. In a complex real scene, it can quickly locate and know which personnel have unsafe factors, facilitating the timely adoption of corresponding measures, enabling the recognition result to directly serve the actual security guarantee work, and improving the practicability and on-site operability of the entire recognition system. BRIEF DESCRIPTION OF THE DRAWINGS

[0029] To more clearly illustrate the technical solutions in one or more embodiments of this specification or the prior art, the following will briefly introduce the accompanying drawings required for the description of the embodiments or the prior art. Obviously, the accompanying drawings in the following description are only some embodiments recorded in this specification. For those of ordinary skill in the art, without creative efforts, other drawings can also be obtained based on these drawings.

[0030] Figure 1 A flowchart of a method for identifying unsafe personnel based on an AR glasses provided for one or more embodiments of this specification;

[0031] Figure 2 A step flowchart of a method for identifying unsafe personnel provided for one or more embodiments of this specification;

[0032] Figure 3 A structural schematic diagram of a device for identifying unsafe personnel based on an AR glasses provided for one or more embodiments of this specification;

[0033] Figure 4 A structural schematic diagram of a computer device provided for one or more embodiments of this specification. Detailed implementation manners

[0034] In order to enable those skilled in the art to better understand the technical solutions in one or more embodiments of this specification, the following will clearly and completely describe the technical solutions in one or more embodiments of this specification in conjunction with the accompanying drawings in one or more embodiments of this specification. Obviously, the described embodiments are only a part of the embodiments of this specification, rather than all the embodiments. Based on one or more embodiments of this specification, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of this invention.

[0035] The following will make a detailed description of the present invention in conjunction with the detailed implementation manners and the accompanying drawings of the specification.

[0036] Method embodiments

[0037] According to an embodiment of the present invention, a method for identifying unsafe personnel based on an AR glasses is provided, as Figure 1As shown in the figure, it is a schematic flowchart of the method for identifying unsafe personnel based on AR glasses provided in this embodiment. According to the method for identifying unsafe personnel based on AR glasses in the embodiments of the present invention, it includes a server, a client, and a web terminal. To achieve personnel safety identification and management based on AR glasses, it is necessary to separately set up: the server system of the AR glasses, the client system installed on the AR glasses hardware, and the web terminal system. The server system is responsible for data recording, processing, receiving, and computing, etc. The client system is responsible for image recognition, transmission, and display, etc. The web terminal system is responsible for data visualization management, etc. Only when the three systems cooperate across terminals can an integrated and closed-loop full-process personnel information security management be formed. As Figure 2 shown, it is a step flowchart of the method for identifying unsafe personnel provided in this embodiment.

[0038] There are many choices for the hardware of AR glasses. As an intelligent wearable device integrating augmented reality technology, AR glasses include multiple functions such as image recognition, superimposed display, and interactive operation. Scenario brand models such as Apple Vision, Meizu StarV View, INMO AIR 2, etc. can all help implement the management method of the present invention. Considering the convenience and efficiency of use, the present invention adopts an AR glasses hardware product based on the Android system. The development environment of the Android system is relatively mature, and the support for image transmission and voice transmission is also relatively smooth. Various support tools or components can be quickly found during the implementation process of the combination of software and hardware.

[0039] The method for identifying unsafe personnel based on AR glasses includes the following steps:

[0040] S110. Obtain face image information through the client and transmit the face image information to the server.

[0041] Safe personnel in thermal power plants can wear AR glasses to conduct safety patrols in the plant area. Once they encounter active personnel, they can immediately identify the personnel information. Once they find a situation where someone enters the plant area irregularly, they can record the image information at any time and automatically upload it to the server.

[0042] S120. The server performs face recognition on the face image information according to the face image information database and sends the recognition result to the web terminal.

[0043] Today, with the rapid development of AI, the technology of face recognition has become increasingly mature. Whether free or commercial, the kernels of various face recognition tools are similar. They all perform face comparison through feature extraction, including methods such as principal component analysis (PCA) and linear discriminant analysis (LDA). In this embodiment, the commercial face recognition interface of iFlytek is used. By calling the face recognition calculation of iFlytek, the recognition result is fed back to the AR glasses client, and the client displays the current face information.

[0044] S130. The web end visualizes the recognition result and sends the visualized result to the client for display.

[0045] The web end system of the AR glasses system is open to the management personnel and all employees of the thermal power plant, and automatically displays various safety information recorded by the glasses server. This can play a good role in overall supervision and warning. The saved safety records can also be summarized into standardized data, providing strong support for the personnel incentive or assessment management of the thermal power plant.

[0046] It further includes the following step S140. Other image information is obtained through the client, and the server identifies the unsafe behaviors included in the other image information and displays the unsafe behaviors on the web end. That is, the same type of operation can also be performed on other unsafe behaviors. For example, when situations such as not wearing safety equipment, leakage of thermal power plant equipment, and unsafe placement of goods are found, image records and text records can also be formed at any time.

[0047] The method provided in this embodiment obtains face image information through the client, can capture the face picture in real time in the actual scene where the wearer is located, ensures obtaining the latest on-site face image data at present, transmits the face image information to the server, and through establishing a stable transmission link, can send the data quickly and reliably, avoiding data accumulation or loss at the client; the server performs face recognition on the face image information according to the face image information database, accurately distinguishes whether it is an unsafe person, and sends the recognition result to the web end, can accurately judge the identity of the person corresponding to the obtained face image, reducing the probability of misjudgment and missed judgment; the web end visualizes the recognition result, converts the abstract recognition and judgment result into an intuitive and easy-to-understand form, and sends the visualized result to the client for display, realizing the integration of virtual recognition information and the real on-site scene in the wearer's field of vision, quickly locating and knowing which personnel have unsafe factors in the complex real scene, facilitating timely taking corresponding measures, enabling the recognition result to directly serve the actual safety guarantee work, and improving the practicability and on-site operability of the entire recognition system.

[0048] In one embodiment, the server performs face recognition on the face image information according to the face image information database, specifically including the following steps:

[0049] The server performs batch face recognition on the face image information through the XSSFWorkbook of the POI component. Among them, after converting the face image information into tabular data on the client side, for example, performing Base64 format processing to convert it into a string, and transmitting it to the server through an https request, and storing the recognition result in the face image information database.

[0050] The method provided in this embodiment uses the XSSFWorkbook of the POI component to realize batch recognition of face image information, significantly improving the recognition efficiency. Through specific format conversion and transmission, it ensures the stability, confidentiality and compatibility of data transmission, reduces the data transmission risk, performs recognition based on the face image information database and stores the results, improves the recognition accuracy with the help of rich data resources, and is convenient for data management and subsequent analysis and utilization, enhancing the performance and practicability of the overall face image recognition system, and effectively assisting the accurate screening and management of unsafe personnel in related application scenarios such as security monitoring.

[0051] As can be seen from the above, all identity recognition of face information is based on the face database. Therefore, the accuracy of the face database is extremely important. However, relying solely on manpower to maintain the accuracy of the database is inefficient and prone to errors. Therefore, it is also necessary to improve the management method of the face database. In one embodiment, the following step S150 is further included:

[0052] Transmit the personnel information data in the EPR system to the server to update the face image information database of the server.

[0053] Generally, thermal power plants will use ERP software products to assist in the operation and management of the power plant. The ERP system will include operation ticket information, work ticket information, work order information and defect order information. These information all contain corresponding personnel information data. By using the webservice interface method, the server of the AR glasses is connected to the ERP system, and the data of the two tickets and work orders are pushed to the database asynchronously, so that the security permission information of the face database can always be in the latest state, saving manpower and avoiding operation errors.

[0054] Device embodiment

[0055] According to an embodiment of the present invention, there is provided an identification device for unsafe personnel based on AR glasses, as Figure 3As shown in the figure, it is a schematic structural diagram of an identification device for unsafe personnel based on an AR glasses provided in this embodiment. The identification device for unsafe personnel based on an AR glasses according to an embodiment of the present invention includes a server, a client, and a web page. The device includes an image acquisition module 31, an identification module 32, and a visualization module 33.

[0056] The image acquisition module 31 is configured to obtain face image information through the client and transmit the face image information to the server.

[0057] The identification module 32 is configured to perform face recognition on the face image information by the server according to the face image information database and send the recognition result to the web page.

[0058] The visualization module 33 is configured to perform visualization processing on the recognition result by the web page and send the visualization processing result to the client for display.

[0059] For the device provided in this embodiment, the image acquisition module 31 obtains face image information through the client, can capture the face picture in real time in the actual scene where the wearer is located, ensures to obtain the latest on-site face image data at present, transmits the face image information to the server, and can send the data quickly and reliably by establishing a stable transmission link, avoiding data accumulation or loss on the client; the identification module 32 performs face recognition on the face image information by the server according to the face image information database, accurately distinguishes whether it is an unsafe person, and sends the recognition result to the web page, can accurately judge the identity of the person corresponding to the obtained face image, and reduces the probability of misjudgment and missed judgment; the visualization module 33 performs visualization processing on the recognition result by the web page, converts the abstract recognition and judgment result into an intuitive and easy-to-understand form, sends the visualization processing result to the client for display, realizes the fusion of virtual recognition information and the real on-site scene in the wearer's field of view, can quickly locate and know which personnel have unsafe factors in a complex real scene, is convenient to take corresponding measures in time, enables the recognition result to directly serve the actual safety guarantee work, and improves the practicability and on-site operability of the entire recognition system.

[0060] In one embodiment, the identification module 32 is further configured to perform batch face recognition on the face image information by the server through the XSSFWorkbook of the POI component. Among them, the face image information is converted into table data in the client and then transmitted to the server, and the recognition result is stored in the face image information database.

[0061] The device provided in this embodiment uses the XSSFWorkbook of the POI component to achieve batch recognition of face image information, significantly improving the recognition efficiency. Through specific format conversion and transmission, it ensures the stability, confidentiality, and compatibility of data transmission, reduces the data transmission risk, conducts recognition based on the face image information database and stores the results, improves the recognition accuracy with the help of rich data resources, facilitates data management and subsequent analysis and utilization, enhances the performance and practicality of the overall face image recognition system, and effectively assists in the precise screening and management of unsafe personnel in related application scenarios such as security monitoring.

[0062] In one embodiment, it further includes a database update module 34, which is used to transmit the personnel information data in the EPR system to the server to update the face image information database of the server.

[0063] The device provided in this embodiment can keep the security permission information of the face database always in the latest state, saving manpower and avoiding operation errors.

[0064] In one embodiment, it further includes an unsafe behavior recognition module 35, which is used to obtain other image information through the client, identify the unsafe behaviors included in the other image information through the server, and display the unsafe behaviors on the web page.

[0065] The device provided in this embodiment can obtain other image information through the client, accurately identify the unsafe behaviors therein by using the server, and display them on the web page, realizing the efficient discovery and intuitive presentation of unsafe behaviors, helping to detect and respond to potential risks in a timely manner, and improving the safety control level in related scenarios.

[0066] The embodiment of the present invention is a corresponding device embodiment to the above method embodiment. The specific operations of each module processing step can be understood with reference to the description of the method embodiment and will not be elaborated here.

[0067] As Figure 4 shown, the present invention also provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, it implements the method for recognizing unsafe personnel based on AR glasses in the above embodiment, or when the computer program is executed by a processor, it implements the method for recognizing unsafe personnel based on AR glasses in the above embodiment.

[0068] Those of ordinary skill in the art can understand that all or part of the processes in the methods of the above embodiments can be completed by instructing relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the embodiments of the above methods. Among them, any reference to a memory, storage, database, or other medium used in the various embodiments provided in this application can include non-volatile and / or volatile memories. Non-volatile memory can include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), Rambus direct RAM (RDRAM), direct memory bus dynamic RAM (DRDRAM), and Rambus dynamic RAM (RDRAM), etc.

[0069] The various embodiments in this specification are described in a progressive manner. For the same or similar parts among the various embodiments, reference can be made to each other. The key point of each embodiment is to illustrate the differences from other embodiments. In particular, for the device or system embodiments, since they are basically similar to the method embodiments, they are described relatively simply. For the relevant parts, reference can be made to the partial description of the method embodiments. The device and system embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the solution of this embodiment. Those of ordinary skill in the art can understand and implement it without creative efforts.

[0070] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the various embodiments of the present invention, and the content not described in detail in the specification of the present invention belongs to the well-known technology in the art.

Claims

1. A method for identifying unsafe persons based on AR glasses, characterized in that: Comprising a server, a client and a web page, the method comprises the following steps: Acquire facial image information through the client, and transmit the facial image information to the server; The server performs face recognition on the face image information according to the face image information database, and sends the recognition result to the web page end; The web page visualizes the recognition result and sends the visualization result to the client for display.

2. The method for identifying unsafe persons according to claim 1, characterized in that: The server performs face recognition on the face image information according to the face image information database, specifically comprising the following steps: The server performs batch face recognition on the face image information through the XSSFWorkbook of the POI component, wherein the face image information is converted into table data in the client and then transmitted to the server; The recognition result is stored in a face image information database.

3. The method for identifying unsafe persons according to claim 1, characterized in that: The following steps are also included: The personnel information data in the EPR system is transmitted to the server and the facial image information database on the server is updated.

4. The method for identifying unsafe persons according to claim 1, characterized in that: The following steps are also included: Other image information is obtained through the client, unsafe behaviors contained in the other image information are identified through the server, and the unsafe behaviors are displayed on the web page.

5. An unsafe person identification device based on AR glasses, characterized in that: The device comprises a server, a client and a web page, and the device comprises an image acquisition module, a recognition module and a visualization module; The image acquisition module is used to acquire face image information through the client and transmit the face image information to the server; The recognition module is used for the server to perform face recognition on the face image information according to the face image information database and send the recognition result to the web page; The visualization module is used to visualize the recognition result on the web page and send the visualization result to the client for display.

6. The unsafe person identification device according to claim 5, characterized in that: The recognition module is further configured to perform batch face recognition on the face image information through the XSSFWorkbook of the POI component at the server, wherein the face image information is converted into table data in the client and then transmitted to the server; The recognition result is stored in a face image information database.

7. The unsafe person identification device according to claim 5, characterized in that: It also includes a database update module, which is used to transmit personnel information data in the EPR system to the server and update the facial image information database of the server.

8. The unsafe person identification device according to claim 5, characterized in that: It also includes an unsafe behavior identification module, which is used to obtain other image information through the client, identify the unsafe behavior contained in the other image information through the server, and display the unsafe behavior on the web page.

9. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for identifying unsafe persons based on AR glasses as described in any one of claims 1 to 4 is implemented.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the steps of the method for identifying unsafe persons based on AR glasses as described in any one of claims 1 to 4 are implemented.

Images