A safety qualification integrated verification system and method based on neural network technology

Abstract

The present invention discloses an integrated security qualification verification system and method based on neural network technology, which relates to the field of image recognition technology and solves the problem of low efficiency of manual verification in the prior art. The present invention realizes accurate verification of the identity of personnel at the work site and intelligent inspection of safety equipment by establishing a personnel information database, extracting work ticket information, collecting and comparing on-site faces, processing document images and verifying information, multi-source image recognition verification, and deep learning model training. The advantage of this process is that it can greatly improve the accuracy and efficiency of the qualification review of operating personnel, ensuring that only personnel with corresponding qualifications and safety equipment can enter the work site, thereby enhancing the safety and compliance of on-site operations. At the same time, through the application of deep learning models, the automation and intelligence level of safety inspections is further improved, providing strong technical support for safety management in industries such as electricity.

Description

Technical Field

[0001] The present invention relates to the field of image recognition technology, and in particular to a security qualification integrated verification system and method based on neural network technology. Background Art

[0002] With the rapid development and intelligent transformation of the power industry, the safety management of the work site has become particularly important. In the power industry, operators must have the corresponding qualifications and skills to ensure that they can complete various tasks safely and effectively. However, the traditional qualification verification method mainly relies on manual inspection of documents and manual records, which has obvious limitations. Due to the intervention of human factors, negligence may occur during the inspection process, resulting in insufficient comprehensive and detailed qualification verification. In addition, the manual verification process is cumbersome and time-consuming, and cannot quickly respond to dynamic changes in on-site operations, which may have serious consequences in emergency situations. Therefore, the power industry urgently needs a more efficient and accurate qualification verification method to improve the safety management level of the work site.

[0003] Although the power industry attaches great importance to the safety management of the operation site, the existing qualification verification process still has many shortcomings. First, the efficiency of manual verification is low and cannot meet the needs of large-scale operation sites, especially during peak operation hours, which can easily cause work backlogs and affect the progress of operations. Secondly, due to the instability of human factors, the error rate in the verification process is high, which may lead to personnel without corresponding qualifications entering the operation site, increasing the risk of safety accidents. In addition, manually recorded information is difficult to quickly query and trace, and once an accident occurs, it is difficult to quickly locate the problem and the person responsible. These problems not only affect the efficiency of operations, but also pose challenges to the safety management of the power industry. There is an urgent need to solve these problems through technological innovation to realize the intelligent and automated safety management of the power industry.

[0004] In view of this, there is a need for an integrated security qualification verification system and method based on neural network technology. Summary of the invention

[0005] In view of the low efficiency of manual verification and the problem of security risks in the prior art, the present invention provides a security qualification integrated verification system and method based on neural network technology, which can automatically verify the work qualifications of personnel by using verification methods such as face recognition and text recognition, thus avoiding the problem of low efficiency of manual verification. The specific technical solution is as follows:

[0006] A safety qualification integrated verification system based on neural network technology, comprising:

[0007] A personnel information database, wherein the personnel information database stores personnel photos, names, work certificate numbers, and positions;

[0008] An operation information extraction unit is connected to a personnel information database. The operation information extraction unit includes a work ticket collection module and an information extraction module. The work ticket collection module is connected to the information extraction module. The work ticket collection module is used to obtain the work ticket and extract the operator information on the work ticket. The information extraction module calls up and stores the corresponding operator's photo, name, work certificate number, and position from the personnel information database based on the operator information of the work ticket collection module.

[0009] The person-certificate-ticket verification unit includes a personnel information verification module, which collects a face image on site based on a preset first camera, and compares the face image with the person photo stored in the information extraction module to check whether the person is consistent. If not, an alarm signal is issued, and if consistent, a signal of successful matching is issued;

[0010] Management terminal, used to receive alarm signals.

[0011] Preferably, the person-certificate-ticket verification unit further includes a certificate information verification module, which includes an image extraction module, a text extraction module and an information verification module, and starts working after receiving a successful match signal output from the personnel information verification module;

[0012] The image extraction module uses the face detection function of OpenCV to locate the face area in the document image, and cuts out the face image from the original image based on the detected face area and saves it;

[0013] The text extraction module uses PaddleOCR or Tesseract OCR to perform text recognition on the ID card image. The recognition result will contain all the text information in the image.

[0014] The information verification module compares the facial image extracted by the image extraction module with the personnel photo in the personnel information database. If the personnel photo is not matched successfully, a first alarm signal is output; if the personnel photo is matched successfully, the name, work ID number, position and other information corresponding to the person in the personnel information database are extracted, and then this information is compared with the information extracted from the text extraction module. If the match is not successful, a second alarm signal is output; if the match is successful, a match success signal is output.

[0015] Preferably, the person-certificate-ticket verification unit further includes a work ticket information verification module, which starts working after receiving a successful matching signal output from the information verification module. The working process is as follows:

[0016] Collect the facial images in the personnel information verification module, the certificate information verification module and the information extraction module, and use the image recognition algorithm to identify whether the three images are of the same person. If so, pass the verification and output a verification pass signal. If not, output a third alarm signal.

[0017] Preferably, it also includes a security verification unit, which is connected to the person-certificate-ticket verification unit. The security verification unit uses a pre-trained security inspection model to perform security verification, and then automatically identifies whether the person is wearing corresponding protective tools.

[0018] Preferably, the training process of the security check model is as follows:

[0019] Collect images with and without protective gear;

[0020] Use annotation tools to annotate images and mark the protective measures and personnel locations in each image;

[0021] Select a deep learning model and use the PyTorch framework for model training;

[0022] Set hyperparameters, start training the model, and evaluate model performance using the validation set;

[0023] Preferably, the security verification unit starts to work only after receiving the verification pass signal output by the person-certificate-ticket verification unit.

[0024] A safety qualification integrated verification method based on neural network technology includes the following steps:

[0025] Step 1: Establish a personnel information database, in which personnel photos, names, work ID numbers, and positions are stored;

[0026] Step 2: Obtain the work ticket and extract the operator information on the work ticket, call the corresponding operator's photo, name, work certificate number, position from the personnel information database and store them separately;

[0027] Step 3: Collect the face image on site, and compare the face image with the person photo stored separately in step 2 to check whether the person is consistent. If they are consistent, proceed to step 4. If they are inconsistent, output an alarm signal and end the process;

[0028] Step 4: Collect the ID image on site, extract the face area on the ID image, and extract the text information on the ID, and check whether the image and text information of the face area correspond to each other in the personnel information database. If so, proceed to step 5, if not, output an alarm signal;

[0029] Step 5: Collect the facial images extracted in steps 2, 3, and 4, and use the image recognition algorithm to identify whether the three images are of the same person. If so, pass the verification and proceed to step 6. If not, output an alarm signal;

[0030] Step 6: Train the safety inspection model based on the deep learning model, obtain pictures of on-site personnel and automatically identify whether they are wearing the corresponding protective tools.

[0031] Preferably, step 4 is as follows:

[0032] Capture document images on-site based on pre-set cameras;

[0033] Use OpenCV's face detection function to locate the face area in the ID image, crop the face image from the original image based on the detected face area, and save it;

[0034] Use PaddleOCR or Tesseract OCR to perform text recognition on the ID card image. The recognition result will contain all the text information in the image.

[0035] The extracted facial image is compared with the personnel photo in the personnel information database. If the personnel photo is not matched successfully, the first alarm signal is output. If the personnel photo is matched successfully, the name, work ID number, position and other information corresponding to the person in the personnel information database are extracted, and then this information is compared with the information extracted in the text extraction module. If the comparison is not successful, the second alarm signal is output. If the comparison is successful, step five is performed.

[0036] A computer-readable storage medium, the computer-readable storage medium comprising a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the integrated security qualification verification method based on neural network technology as described in any one of claims 7 to 8.

[0037] A processor, the processor is used to run a program, wherein when the program is running, the integrated security qualification verification method based on neural network technology as described in any one of claims 7 to 8 is executed.

[0038] Compared with the prior art, the present invention has the following beneficial effects:

[0039] The present invention realizes accurate verification of the identity of personnel at the work site and intelligent inspection of safety equipment by establishing a personnel information database, extracting work ticket information, collecting and comparing on-site faces, processing document images and checking information, multi-source image recognition and verification, and training deep learning models. The advantage of this process is that it can greatly improve the accuracy and efficiency of the qualification review of operators, reduce human errors and fraudulent behavior, and ensure that only personnel with corresponding qualifications and safety equipment can enter the work site, thereby enhancing the safety and compliance of on-site operations. At the same time, through the application of deep learning models, the automation and intelligence level of safety inspections is further improved, providing strong technical support for safety management in industries such as electricity. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the following is a brief introduction to the drawings required for the specific embodiments or the description of the prior art. In all the drawings, similar elements or parts are generally identified by similar reference numerals. In the drawings, the elements or parts are not necessarily drawn according to the actual scale.

[0041] Figure 1 is a flow chart of the method of the present invention;

[0042] Figure 2 It is a specific flow chart of step four of the present invention. DETAILED DESCRIPTION

[0043] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0044] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0045] It should also be understood that the terms used in the present specification are only for the purpose of describing specific embodiments and are not intended to limit the present invention. As used in the present specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0046] It should be further understood that the term "and / or" used in the present description and the appended claims refers to any and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0047] In one embodiment of the present invention, a security qualification integrated verification system based on neural network technology is provided, comprising:

[0048] A personnel information database, wherein the personnel information database stores personnel photos, names, work certificate numbers, positions, etc.

[0049] An operation information extraction unit is connected to a personnel information database. The operation information extraction unit includes a work ticket collection module and an information extraction module. The work ticket collection module is connected to the information extraction module. The work ticket collection module is used to obtain the work ticket and extract the operator information on the work ticket. The information extraction module calls up and stores the corresponding operator's photo, name, work certificate number, and position from the personnel information database based on the operator information of the work ticket collection module.

[0050] Person-ID-Ticket Verification Unit: This unit is used to verify whether the personnel information, ID information and work ticket information are correct, including:

[0051] 1. Personnel information verification module: Based on the pre-set first camera, the face image is collected on site, and the face image is compared with the person photo stored in the information extraction module to check whether the person is consistent. This module is installed with a face recognition model for comparing the collected photo with the photo stored in the information extraction module and outputting the comparison result.

[0052] 2. The certificate information verification module collects the certificate image on site based on the pre-set second camera. The certificate information verification module includes an image extraction module, a text extraction module and an information verification module; wherein the image extraction module uses the face detection function of OpenCV to locate the face area in the certificate image, and cuts out the face image from the original image according to the detected face area, and saves it. The text extraction module uses PaddleOCR or Tesseract OCR to perform text recognition on the ID card image, and the recognition result will contain all the text information in the image. The information verification module compares the face image extracted by the image extraction module with the personnel photo in the personnel information database. If the personnel photo is not successfully matched, that is, the photo on the certificate does not exist in the personnel information database, it indicates that the face on the work permit does not belong to the current working system, and the first alarm signal can be output. If the personnel photo is matched successfully, the name, work ID number, position and other information corresponding to the person in the personnel information database will be extracted, and then this information will be compared with the information extracted from the text extraction module. If the comparison is successful, it means that the information displayed on the certificate is correct. If the comparison is not successful, such as the name or position does not match, it means that there are text costs or printing errors, and a second alarm signal can be output at this time.

[0053] 3. Work ticket information verification module. This module is designed to verify whether the certificate holder, relevant certificates and work ticket information match. That is to say, this module is a further verification module based on the successful matching of the first two modules. It collects facial images from the personnel information verification module, certificate information verification module and information extraction module, and uses the image recognition algorithm to identify whether the three images are of the same person. If so, it passes the verification and outputs a verification pass signal. If not, it outputs a third alarm signal.

[0054] The security verification unit is connected to the person-certificate-ticket verification unit. After receiving the verification pass signal output by the person-certificate-ticket verification unit, a pre-trained security inspection model is used to perform security verification.

[0055] The safety inspection model aims to obtain pictures of on-site personnel and automatically identify whether they are wearing appropriate protective equipment, such as helmets.

[0056] The training process of the safety check model is as follows:

[0057] S1: Collect images with and without protective gear. This step can use existing datasets, such as the Helmet-Asian and Helmet-Europe datasets, which contain 19,000+ and 3,000+ images respectively, and are suitable for target detection model training.

[0058] S2: Use labeling tools (such as LabelImg) to label the images and mark the protective measures and personnel locations in each image.

[0059] S3: You can choose deep learning models such as YOLOv5 or YOLOv8. These models perform well in object detection tasks and are particularly suitable for real-time detection scenarios.

[0060] S4: Use the PyTorch framework for model training. In this, transfer learning can be used to start from a pre-trained model to speed up training and improve accuracy.

[0061] S5: Set hyperparameters (such as learning rate, batch size, etc.) and start training the model. Use the validation set to evaluate the model performance, focusing on indicators such as mAP (average precision) and F1 score to ensure the effectiveness of the model in practical applications.

[0062] S6: Collect real-time video streams or static images and use the trained model to identify whether relevant personnel are wearing protective measures.

[0063] In one embodiment of the present invention, a security qualification integrated verification method based on neural network technology is also provided. Figure 1 As shown, the following steps are included:

[0064] Step 1: Establish a personnel information database, which stores personnel photos, names, work ID numbers, positions, etc.;

[0065] Step 2: Obtain the work ticket and extract the operator information on the work ticket, call the corresponding operator's photo, name, work certificate number, position from the personnel information database and store them separately;

[0066] Step 3: Collect the face image on site, and compare the face image with the person photo stored separately in step 2 to check whether the person is consistent. If they are consistent, proceed to step 4. If they are inconsistent, output an alarm signal and end the process;

[0067] Step 4: Collect the ID image on site, extract the face area on the ID image, and extract the text information on the ID, and check whether the image and text information of the face area exist one-to-one in the personnel information database. If so, proceed to step 5, if not, output an alarm signal.

[0068] like Figure 2 As shown, step four is as follows:

[0069] Capture document images on-site based on pre-set cameras;

[0070] Use OpenCV's face detection function to locate the face area in the ID image, crop the face image from the original image based on the detected face area, and save it;

[0071] Use PaddleOCR or Tesseract OCR to perform text recognition on the ID card image. The recognition result will contain all the text information in the image.

[0072] The extracted face image is compared with the person's photo in the person information database. If the person's photo is not successfully matched, a first alarm signal is output. If the person's photo is successfully matched, the name, work certificate number, position and other information corresponding to the person in the person information database are extracted, and then the information is compared with the information extracted in the text extraction module. If the comparison is not successful, a second alarm signal is output. If the comparison is successful, step five is performed;

[0073] Step 5: Collect the facial images extracted in steps 2, 3, and 4, and use the image recognition algorithm to identify whether the three images are of the same person. If so, pass the verification and enter step 6. If not, output an alarm signal.

[0074] Step 6: Train the safety inspection model based on the deep learning model, obtain the pictures of the on-site personnel and automatically identify whether they are wearing the corresponding protective tools, including helmets; the details of this step are as follows:

[0075] S1: Collect images with and without protective gear. This step can use existing datasets, such as the Helmet-Asian and Helmet-Europe datasets, which contain 19,000+ and 3,000+ images respectively, and are suitable for target detection model training.

[0076] S2: Use labeling tools (such as LabelImg) to label the images and mark the protective measures and personnel locations in each image.

[0077] S3: You can choose deep learning models such as YOLOv5 or YOLOv8. These models perform well in object detection tasks and are particularly suitable for real-time detection scenarios.

[0078] S4: Use the PyTorch framework for model training. In this, transfer learning can be used to start from a pre-trained model to speed up training and improve accuracy.

[0079] S5: Set hyperparameters (such as learning rate, batch size, etc.) and start training the model. Use the validation set to evaluate the model performance, focusing on indicators such as mAP (average precision) and F1 score to ensure the effectiveness of the model in practical applications.

[0080] S6: Collect real-time video streams or static images and use the trained model to identify whether relevant personnel are wearing protective measures.

[0081] In summary, the present invention realizes accurate verification of the identity of personnel at the work site and intelligent inspection of safety equipment through the steps of establishing a personnel information database, extracting work ticket information, collecting and comparing on-site faces, processing document images and checking information, multi-source image recognition and verification, and deep learning model training. The advantage of this process is that it can greatly improve the accuracy and efficiency of the qualification review of operators, reduce human errors and fraudulent behaviors, and ensure that only personnel with corresponding qualifications and safety equipment can enter the work site, thereby enhancing the safety and compliance of on-site operations. At the same time, through the application of deep learning models, the automation and intelligence level of safety inspections is further improved, providing strong technical support for safety management in industries such as electricity.

[0082] Those of ordinary skill in the art will appreciate that the units of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition of each example has been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of the present invention.

[0083] In the embodiments provided by the present invention, it should be understood that the division of units is only a logical function division, and there may be other division methods in actual implementation, for example, multiple units can be combined into one unit, one unit can be split into multiple units, or some features can be ignored, etc.

[0084] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0085] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: U disk, read-only memory (ROM, Read-0nlyMemory), random access memory (RAM, RandomAccessMemory), mobile hard disk, magnetic disk or optical disk, etc., which can store program code.

[0086] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or replace some or all of the technical features therein by equivalents. These modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention, and they should all be included in the scope of the claims and specification of the present invention.

Claims

1. A security qualification integrated verification system based on neural network technology, characterized in that: include: A personnel information database, wherein the personnel information database stores personnel photos, names, work certificate numbers, and positions; An operation information extraction unit is connected to a personnel information database, and includes a work ticket collection module and an information extraction module. The work ticket collection module is connected to the information extraction module. The work ticket collection module is used to obtain the work ticket and extract the operator information on the work ticket. The information extraction module retrieves and stores the personnel photo, name, work certificate number, and position of the corresponding operator from the personnel information database based on the operator information of the work ticket collection module; The person-certificate-ticket verification unit includes a personnel information verification module, which collects a face image on site based on a preset first camera, and compares the face image with the person photo stored in the information extraction module to check whether the person is consistent. If not, an alarm signal is issued, and if consistent, a signal of successful matching is issued; Management terminal, used to receive alarm signals.

2. According to claim 1, a security qualification integrated verification system based on neural network technology is characterized in that: The person-certificate-ticket verification unit also includes a certificate information verification module, which includes an image extraction module, a text extraction module and an information verification module, and starts working after receiving a successful matching signal output from the personnel information verification module; The image extraction module uses the face detection function of OpenCV to locate the face area in the document image, and cuts out the face image from the original image based on the detected face area and saves it; The text extraction module uses PaddleOCR or Tesseract OCR to perform text recognition on the ID card image. The recognition result will contain all the text information in the image. The information verification module compares the facial image extracted by the image extraction module with the personnel photo in the personnel information database. If the personnel photo is not matched successfully, a first alarm signal is output; if the personnel photo is matched successfully, the name, work ID number, position and other information corresponding to the person in the personnel information database are extracted, and then this information is compared with the information extracted from the text extraction module. If the match is not successful, a second alarm signal is output; if the match is successful, a match success signal is output.

3. According to claim 2, a security qualification integrated verification system based on neural network technology is characterized in that: The person-certificate-ticket verification unit also includes a work ticket information verification module, which starts working after receiving a successful match signal output from the information verification module. The working process is as follows: Collect the facial images in the personnel information verification module, the certificate information verification module and the information extraction module, and use the image recognition algorithm to identify whether the three images are of the same person. If so, pass the verification and output a verification pass signal. If not, output a third alarm signal.

4. According to claim 3, a security qualification integrated verification system based on neural network technology is characterized in that: It also includes a safety verification unit, which is connected to the person-certificate-ticket verification unit. The safety verification unit uses a pre-trained safety inspection model to perform safety verification, and then automatically identifies whether the person is wearing the corresponding protective tools.

5. According to claim 4, a security qualification integrated verification system based on neural network technology is characterized in that: The training process of the safety check model is as follows: Collect images with and without protective gear; Use annotation tools to annotate images and mark the protective measures and personnel locations in each image; Select a deep learning model and use the PyTorch framework for model training; Set the hyperparameters, start training the model, and evaluate the model performance using the validation set.

6. The integrated security qualification verification system based on neural network technology according to claim 5 is characterized in that: The security verification unit starts working only after receiving the verification pass signal output by the person-certificate-ticket verification unit.

7. A security qualification integrated verification method based on neural network technology, characterized in that: The following steps are involved: Step 1: Establish a personnel information database, which stores personnel photos, names, work ID numbers, and positions; Step 2: Obtain the work ticket and extract the operator information on the work ticket, call the corresponding operator's photo, name, work certificate number, position from the personnel information database and store them separately; Step 3: Collect the face image on site, and compare the face image with the person photo stored separately in step 2 to check whether the person is consistent. If they are consistent, proceed to step 4. If they are inconsistent, output an alarm signal and end the process; Step 4: Collect the ID image on site, extract the face area on the ID image, and extract the text information on the ID, and check whether the image and text information of the face area correspond to each other in the personnel information database. If so, proceed to step 5, if not, output an alarm signal; Step 5: Collect the facial images extracted in steps 2, 3, and 4, and use the image recognition algorithm to identify whether the three images are of the same person. If so, pass the verification and proceed to step 6. If not, output an alarm signal; Step 6: Train the safety inspection model based on the deep learning model, obtain pictures of on-site personnel and automatically identify whether they are wearing the corresponding protective tools.

8. The method for integrated security qualification verification based on neural network technology according to claim 7 is characterized in that: Step 4 is as follows: Capture document images on-site based on pre-set cameras; Use OpenCV's face detection function to locate the face area in the ID image, crop the face image from the original image based on the detected face area, and save it; Use PaddleOCR or Tesseract OCR to perform text recognition on the ID card image. The recognition result will contain all the text information in the image. The extracted facial image is compared with the personnel photo in the personnel information database. If the personnel photo is not matched successfully, the first alarm signal is output. If the personnel photo is matched successfully, the name, work ID number, position and other information corresponding to the person in the personnel information database are extracted, and then this information is compared with the information extracted in the text extraction module. If the comparison is not successful, the second alarm signal is output. If the comparison is successful, step five is performed.

9. A computer-readable storage medium, characterized in that: The computer-readable storage medium includes a stored program, wherein when the program is running, the device where the computer-readable storage medium is located is controlled to execute the integrated security qualification verification method based on neural network technology as described in any one of claims 7 to 8.

10. A processor, characterized in that: The processor is used to run a program, wherein when the program is running, the integrated security qualification verification method based on neural network technology described in any one of claims 7 to 8 is executed.

Images