A wind farm work ticket inspection method and system based on image text recognition

Abstract

The present invention discloses a wind farm work ticket inspection method and system based on image text recognition. The method extracts part of key information from photos of on-site work tickets based on image recognition technology to identify text content therein, and can also be associated with a troubleshooting plan in a fault library according to the identified key work content, providing a reference basis for operators, and then combining NFC ink screen photos of the work site for boarding and clocking in supervision, so that management personnel can track and count the completion of work tickets, and can also improve the review and execution efficiency of work tickets. On the one hand, the present invention can effectively constrain the site to carry out work in accordance with the requirements and specifications, forming an effective supervision mechanism for the work, and on the other hand, provide troubleshooting plan support for on-site operators, and can reduce the workload of reviewers, improve work efficiency, effectively constrain the site to carry out work in accordance with the requirements and specifications, forming an effective supervision mechanism for the work, and has high compatibility.

Description

Technical Field

[0001] The present invention relates to the technical field of wind power work spot inspection, and in particular to a wind farm work ticket spot inspection method and system based on image text recognition. Background Art

[0002] A wind farm work ticket refers to an organizational measure to ensure safety when performing maintenance, repairs, troubleshooting, and other work on site in order to effectively ensure the personal and equipment safety of the staff and prevent accidents. Whether the work ticket is executed in accordance with the work procedures not only affects the safe operation of the equipment but also the personal safety of the executors. Therefore, the management of the work ticket is crucial. In actual use, the execution of the work ticket is generally supervised, spot-checked, and assessed by the leaders of the wind farm or the leaders of the wind power production department. This not only consumes a lot of manpower, but also makes it difficult to trace the execution process of the operators. Summary of the invention

[0003] The purpose of the present invention is to solve the deficiencies in the prior art and provide a wind farm work ticket inspection method and system based on image text recognition. Based on image recognition technology, some key information of the photos taken of the on-site work tickets is identified to extract the text content. At the same time, according to the identified key work content, it can also be associated with the troubleshooting plan of the fault library to provide a reference basis for the operating personnel. Combined with the NFC ink screen photos of the work site, boarding and clocking supervision are carried out, which is convenient for managers to track and count the completion of the work tickets, and can also improve the review and execution efficiency of the work tickets.

[0004] To achieve the above object, the technical solution provided by the present invention is: a method for checking wind farm work tickets based on image text recognition, comprising the following steps:

[0005] S1. photographing the wind farm work ticket, and uploading the photographed wind farm work ticket to a photo storage system;

[0006] S2. Use the PaddleOCR model framework to identify the uploaded wind farm work ticket. If the wind farm work ticket fails to be identified, the uploaded photo is judged to be unqualified, and it is necessary to return to step S1 to re-photograph and upload to the photo storage system; if the wind farm work ticket is successfully identified, execute step S3;

[0007] S3. Determine whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, remind the user to upload the image again. If the ticket number does not exist and the punch-in result is failed or none, extract the key information in the text information of the wind farm work ticket to generate a key content image and upload it to the punch-in system. At the same time, use a text similarity algorithm to calculate the similarity coefficient between the key information in the wind farm work ticket and the troubleshooting solution in the fault library, and extract multiple troubleshooting solutions based on the similarity coefficient as a reference for troubleshooting.

[0008] S4. Use NFC to sense the key content image to the ink screen of the corresponding wind turbine tower, take a photo of the ink screen, and upload the photo of the ink screen to the clocking-in system;

[0009] S5. Use the PaddleOCR model framework to recognize the text information of the uploaded ink screen photo;

[0010] S6. Compare the ticket number in the text information of the key content image and the ticket number in the text information of the ink screen photo to see if they are consistent. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane. If the ticket numbers are inconsistent, it is determined that the staff member has failed to board the plane.

[0011] Further, in step S2, the following operations are specifically performed:

[0012] The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket and identify whether there are words on the work ticket. If the words on the work ticket cannot be recognized, it is considered that the recognition of the wind farm work ticket has failed, and the uploaded photo is judged to be unqualified. It is necessary to return to step S1 to re-photograph and upload it to the photo storage system; if the words on the work ticket are recognized, the wind farm work ticket is successfully recognized, and step S3 is executed.

[0013] Further, in step S3, the following operations are specifically performed:

[0014] Determine whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, a reminder is given to upload the image again. If the ticket number does not exist and the punch-in result is failed or none, the key information in the wind farm work ticket is extracted and the extracted key information is displayed in a table using the third-party python library prettyTable. With the help of the ImageDraw module of the image processing library PIL, the table is drawn to the specified position of the image to generate a key content image, which is named after the ticket number of the work ticket and uploaded to the punch-in system. At the same time, based on the key information in the identified wind farm work ticket, the fault library troubleshooting plan is associated based on the python natural language processing library Gensim and the TF-IDF algorithm, and multiple troubleshooting plans are extracted according to the text similarity coefficient as a reference for the troubleshooting plan of the on-site workers.

[0015] Further, in step S4, the following operations are specifically performed:

[0016] Download the key content picture from the clock-in system to the mobile phone, sense the key content picture to the ink screen of the corresponding wind turbine tower through NFC, take a photo of the ink screen to obtain the ink screen photo and upload the ink screen photo to the clock-in system.

[0017] Furthermore, the key information, text information of key content pictures and text information of ink screen photos all include ticket number, work ticket type, work supervisor, work shift members, equipment or system code, equipment or system name, work location and content, planned start time and planned completion time.

[0018] The present invention also provides a wind farm work ticket inspection system based on image text recognition, comprising:

[0019] A work ticket photo acquisition module is used to acquire work ticket photos corresponding to the wind farm;

[0020] The key content acquisition module of the work ticket photo recognizes the text content of the wind farm work ticket photo based on the PaddleOCR model framework, extracts key information to generate key content pictures, and uploads them to the punch-in system after naming them with the work ticket number;

[0021] The fault library troubleshooting solution association module associates the fault library troubleshooting solution with key information to provide troubleshooting solution support for on-site operators;

[0022] NFC ink screen photo acquisition module, which acquires ink screen photos through NFC and uploads them to the clock-in system;

[0023] NFC ink screen photo content acquisition module, which uses the PaddleOCR framework to recognize text information in the uploaded ink screen photo;

[0024] The boarding punch-in result determination module compares the recognized ink screen photo information with the original wind farm work ticket photo information extracted by the system. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane, otherwise it is determined that the staff member has failed to board the plane.

[0025] Furthermore, the work ticket photo key content acquisition module specifically performs the following operations:

[0026] The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket photo, and the key information in the text content is extracted. The third-party python library prettyTable is used to display the extracted key information in a table. With the help of the ImageDraw module of the image processing library PIL, the table is drawn to the specified position of the image to generate a key content picture.

[0027] Furthermore, the fault library troubleshooting solution association module specifically performs the following operations:

[0028] According to the key information in the identified wind farm work ticket, the Python-based natural language processing library Gensim is used to associate the fault library troubleshooting solutions using the TF-IDF algorithm, and multiple troubleshooting solutions are extracted according to the text similarity coefficient as a reference for the on-site workers' troubleshooting solutions.

[0029] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0030] On the one hand, the present invention can effectively constrain the site to carry out work in accordance with the requirements and specifications, forming an effective supervision mechanism for the work; on the other hand, it can provide investigation plan support for on-site workers; at the same time, it can reduce the workload of auditors, improve work efficiency, effectively constrain the site to carry out work in accordance with the requirements and specifications, forming an effective supervision mechanism for the work, and has high compatibility. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 This is a flow chart of the wind farm work ticket inspection method.

[0032] Figure 2 Key content images generated for the present invention.

[0033] Figure 3 This is a photo of the ink screen. DETAILED DESCRIPTION

[0034] The present invention will be further described below in conjunction with specific embodiments.

[0035] Example 1

[0036] See also Figures 1 to 3 As shown, the wind farm work ticket inspection method based on image text recognition provided by this embodiment includes the following steps:

[0037] S1. photographing the wind farm work ticket, and uploading the photographed wind farm work ticket to a photo storage system;

[0038] S2. Use the PaddleOCR model framework to identify the uploaded wind farm work ticket. If the wind farm work ticket fails to be identified, the uploaded photo is judged to be unqualified, and it is necessary to return to step S1 to re-photograph and upload to the photo storage system; if the wind farm work ticket is successfully identified, execute step S3, and specifically perform the following operations:

[0039] The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket and identify whether there are words on the work ticket. If the words on the work ticket cannot be recognized, it is considered that the recognition of the wind farm work ticket has failed, and the uploaded photo is judged to be unqualified. It is necessary to return to step S1 to re-photograph and upload it to the photo storage system; if the words on the work ticket are recognized, the wind farm work ticket is successfully recognized, and step S3 is executed.

[0040] S3. Determine whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, remind the user to upload the image again. If the ticket number does not exist and the punch-in result is failed or none, extract the key information in the text information of the wind farm work ticket to generate a key content image and upload it to the punch-in system. At the same time, use the text similarity algorithm to calculate the similarity coefficient between the key information in the wind farm work ticket and the troubleshooting solution in the fault library. Extract multiple troubleshooting solutions based on the similarity coefficient. Specifically, perform the following operations:

[0041] Determine whether the ticket number GZP-1-202106-0002 of the wind farm work ticket exists in the punch-in record system. If the ticket number GZP-1-202106-0002 exists in the punch-in record system and the punch-in result is successful, then remind to upload the picture again. If the ticket number GZP-1-202106-0002 does not exist and the punch-in result is failed or none, then extract the ticket number, work ticket type, work person in charge, work shift member, equipment or system code, equipment or system name, work location and content, planned start time and planned completion time in the wind farm work ticket, and use the third-party library Pret in Python to extract the key information. tyTable displays the extracted key information in a tabular form, and uses the ImageDraw module of the image processing library PIL to draw the table to the specified position of the image to generate a key content picture, and uploads it to the punch-in system after naming it with the work ticket number GZP-1-202106-0002; at the same time, according to the key information in the identified wind farm work ticket, stop words are removed, and then the fault library troubleshooting plan is associated based on the python natural language processing library Gensim and the TF-IDF algorithm, and the top three troubleshooting plans with the largest similarity coefficients are extracted according to the text similarity coefficient as a reference for the troubleshooting plan of on-site workers.

[0042] S4. Use NFC to sense the key content image to the ink screen of the corresponding wind turbine tower. After taking a photo of the ink screen, upload the ink screen photo to the clock-in system. Specifically, perform the following operations:

[0043] When on-site workers arrive at the site for work, they use their mobile phones to download key content pictures from the clock-in system to their mobile phones. After sensing the key content pictures to the ink screen of the corresponding wind turbine tower through NFC, they take a photo of the ink screen to obtain the ink screen photo and upload the ink screen photo to the clock-in system.

[0044] S5. The check-in system uses the PaddleOCR model framework to recognize the text information of the uploaded ink screen photo;

[0045] S6. Compare the ticket number in the text information of the key content image with the ticket number in the text information of the ink screen photo to see if they are consistent. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane and the boarding punch-in column on the system interface is checked. If the ticket numbers are inconsistent, it is determined that the staff member has failed to board the plane.

[0046] Among them, the text information of the key content picture and the text information of the ink screen photo both include the ticket number, work ticket type, work supervisor, work team members, equipment or system code, equipment or system name, work location and content, planned start time and planned completion time.

[0047] Example 2

[0048] This embodiment discloses a wind farm work ticket inspection system based on image text recognition, including:

[0049] A work ticket photo acquisition module is used to acquire work ticket photos corresponding to the wind farm;

[0050] The work ticket photo key content acquisition module recognizes the text content of the wind farm work ticket photo based on the PaddleOCR model framework, extracts key information to generate key content pictures, and uploads them to the punch-in system after naming them with the work ticket number. The specific operations are as follows:

[0051] The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket photo, and the key information in the text content is extracted. The third-party python library prettyTable is used to display the extracted key information in a table. With the help of the ImageDraw module of the image processing library PIL, the table is drawn to the specified position of the image to generate a key content picture.

[0052] The fault library troubleshooting solution association module associates the fault library troubleshooting solution with key information and provides troubleshooting solution support for on-site operators. Specifically, the following operations are performed:

[0053] According to the key information in the identified wind farm work ticket, the Python-based natural language processing library Gensim is used to associate the fault library troubleshooting solutions using the TF-IDF algorithm, and multiple troubleshooting solutions are extracted according to the text similarity coefficient as a reference for the on-site workers' troubleshooting solutions.

[0054] NFC ink screen photo acquisition module, which acquires ink screen photos through NFC and uploads them to the clock-in system;

[0055] NFC ink screen photo content acquisition module, which uses the PaddleOCR framework to recognize text information in the uploaded ink screen photo;

[0056] The boarding punch-in result determination module compares the recognized ink screen photo information with the original wind farm work ticket photo information extracted by the system. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane, otherwise it is determined that the staff member has failed to board the plane.

[0057] The embodiments described above are only preferred embodiments of the present invention and are not intended to limit the scope of implementation of the present invention. Therefore, all changes made according to the shape and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A wind farm work ticket inspection method based on image text recognition, characterized in that: The following steps are involved: S1. photographing the wind farm work ticket, and uploading the photographed wind farm work ticket to a photo storage system; S2. Use the PaddleOCR model framework to identify the uploaded wind farm work ticket. If the wind farm work ticket fails to be identified, the uploaded photo is judged to be unqualified, and it is necessary to return to step S1 to re-photograph and upload to the photo storage system; if the wind farm work ticket is successfully identified, execute step S3; S3. Determine whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, remind the user to upload the image again. If the ticket number does not exist and the punch-in result is failed or none, extract the key information in the text information of the wind farm work ticket to generate a key content image and upload it to the punch-in system. At the same time, use a text similarity algorithm to calculate the similarity coefficient between the key information in the wind farm work ticket and the troubleshooting solution in the fault library, and extract multiple troubleshooting solutions based on the similarity coefficient as a reference for troubleshooting. S4. Use NFC to sense the key content image to the ink screen of the corresponding wind turbine tower, take a photo of the ink screen, and upload the photo of the ink screen to the clocking-in system; S5. Use the PaddleOCR model framework to recognize the text information of the uploaded ink screen photo; S6. Compare the ticket number in the text information of the key content image and the ticket number in the text information of the ink screen photo to see if they are consistent. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane. If the ticket numbers are inconsistent, it is determined that the staff member has failed to board the plane.

2. A method for checking wind farm work tickets based on image text recognition according to claim 1, characterized in that: In step S2, the following operations are specifically performed: The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket and identify whether there are words on the work ticket. If the words on the work ticket cannot be recognized, it is considered that the recognition of the wind farm work ticket has failed, and the uploaded photo is judged to be unqualified. It is necessary to return to step S1 to re-photograph and upload it to the photo storage system; if the words on the work ticket are recognized, the wind farm work ticket is successfully recognized, and step S3 is executed.

3. The method for checking wind farm work tickets based on image text recognition according to claim 1 is characterized in that: In step S3, the following operations are specifically performed: Determine whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, a reminder is given to upload the image again. If the ticket number does not exist and the punch-in result is failed or none, the key information in the wind farm work ticket is extracted and the extracted key information is displayed in a table using the third-party python library prettyTable. With the help of the ImageDraw module of the image processing library PIL, the table is drawn to the specified position of the image to generate a key content image, which is named after the ticket number of the work ticket and uploaded to the punch-in system. At the same time, based on the key information in the identified wind farm work ticket, the fault library troubleshooting plan is associated based on the python natural language processing library Gensim and the TF-IDF algorithm, and multiple troubleshooting plans are extracted according to the text similarity coefficient as a reference for the troubleshooting plan of the on-site workers.

4. The method for checking wind farm work tickets based on image text recognition according to claim 1 is characterized in that: In step S4, the following operations are specifically performed: Download the key content picture from the clock-in system to the mobile phone, sense the key content picture to the ink screen of the corresponding wind turbine tower through NFC, take a photo of the ink screen to obtain the ink screen photo and upload the ink screen photo to the clock-in system.

5. The method for checking wind farm work tickets based on image text recognition according to claim 1 is characterized in that: The key information, the text information of the key content picture and the text information of the ink screen photo all include the ticket number, work ticket type, work supervisor, work shift members, equipment or system code, equipment or system name, work location and content, planned start time and planned completion time.

6. A wind farm work ticket inspection system based on image text recognition, characterized in that: include: A work ticket photo acquisition module is used to acquire work ticket photos corresponding to the wind farm; The key content acquisition module of the work ticket photo recognizes the text content of the wind farm work ticket photo based on the PaddleOCR model framework, extracts key information to generate key content pictures, and uploads them to the punch-in system after naming them with the work ticket number; the uploaded wind farm work ticket is recognized by the PaddleOCR model framework. If the recognition of the wind farm work ticket fails, the uploaded photo is judged to be unqualified, and it is necessary to return to the work ticket photo acquisition module to re-acquire the work ticket photo; if the wind farm work ticket is successfully recognized, it is determined whether the ticket number of the wind farm work ticket exists in the punch-in record system. If the ticket number exists in the punch-in record system and the punch-in result is successful, it is reminded to upload the picture again. If the ticket number does not exist and the punch-in result is failed or none, the key information in the text information of the wind farm work ticket is extracted to generate key content pictures and upload them to the punch-in system; The fault library troubleshooting scheme association module associates the fault library troubleshooting scheme with key information to provide troubleshooting scheme support for on-site operators; a text similarity algorithm is used to calculate the similarity coefficient between the key information in the wind farm work ticket and the fault library troubleshooting scheme, and multiple troubleshooting schemes are extracted based on the similarity coefficient as troubleshooting references; The NFC ink screen photo acquisition module senses the key content image to the ink screen of the corresponding wind turbine tower through NFC, takes a photo of the ink screen, and uploads the ink screen photo to the clock-in system; NFC ink screen photo content acquisition module, which uses the PaddleOCR framework to recognize text information in the uploaded ink screen photo; The boarding punch-in result determination module compares the recognized ink screen photo information with the original wind farm work ticket photo information extracted by the system. If the ticket numbers are consistent, it is determined that the staff member has successfully boarded the plane, otherwise it is determined that the staff member has failed to board the plane.

7. A wind farm work ticket checking system based on image text recognition according to claim 6, characterized in that: The work ticket photo key content acquisition module specifically performs the following operations: The PaddleOCR model framework is used to recognize the text content of the wind farm work ticket photo, and the key information in the text content is extracted. The third-party python library prettyTable is used to display the extracted key information in a table. With the help of the ImageDraw module of the image processing library PIL, the table is drawn to the specified position of the image to generate a key content picture.

8. The wind farm work ticket checking system based on image text recognition according to claim 6 is characterized in that: The fault library troubleshooting scheme association module specifically performs the following operations: According to the key information in the identified wind farm work ticket, the Python-based natural language processing library Gensim is used to associate the fault library troubleshooting solutions using the TF-IDF algorithm, and multiple troubleshooting solutions are extracted according to the text similarity coefficient as a reference for the on-site workers' troubleshooting solutions.

Images