Intelligent tool container management method and system

Abstract

The invention discloses an intelligent tool container management method, system and equipment and a medium. Receiving a task instruction sent by the tool management platform, and extracting task information from the task instruction; according to the task information, the track mechanical arm, the indicator and the intelligent tool basket execute respective corresponding operations in the task information; based on the task information, carrying out final detection on the target tool, judging whether a final detection result meets a preset requirement or not, and obtaining a judgment result; and generating task feedback information according to a judgment result, and sending the task feedback information to the tool management platform. The intelligent tool container, the track-based mechanical arm, the indicator and the intelligent tool basket are utilized, and intelligent judgment and execution are carried out on the process proceeding condition in the tool receiving and returning process, so that the convenience and accuracy of tool receiving and returning are greatly improved, the conditions that tools are mistakenly received or lost and the like are reduced, and the working efficiency is improved. The labor, time and capital costs are reduced, and the tool management efficiency is improved.

Description

Technical Field

[0001] The present invention relates to the field of maintenance tool management for steam turbines and gas turbines, and particularly to an intelligent tool container management method, system, computing device, and computer storage medium. Background Art

[0002] Steam turbines and gas turbines are widely used in multiple industrial fields such as energy, aerospace, industrial manufacturing, and ocean engineering. However, due to their complex and precise structures and extremely high safety requirements in various application scenarios, it is necessary to frequently understand the operating conditions of steam turbines or gas turbines and perform repairs in a timely manner when faults or damages occur.

[0003] Due to their complex structures and a large number of parts, the types and quantities of corresponding tools required for their medium repairs or overhauls are also very large. Currently, for the tools required for maintenance, they are usually transported to the maintenance site by a specially modified container by the maintenance unit and then managed by a dedicated person. The maintenance workers are required to pick them up after going to work and return them before getting off work, and the management is carried out by registering and signing on a record book. However, in this way, since there are many sub-teams after the unit is disassembled at the maintenance site, the time required for registration, picking up, and returning is very long, accounting for 6% to 13% of the working hours in the maintenance work; when the tools are returned, it is easy to make mistakes during manual collection and statistics, resulting in the loss of tools, causing direct economic losses, or the incorrect placement of tools, reducing work efficiency. Moreover, after the maintenance is completed, a large amount of manual work is required to count and check the tools for replenishment and procurement, greatly increasing the labor cost and time cost. Summary of the Invention

[0004] To solve the above technical problems, the present invention provides an intelligent tool container management method and corresponding intelligent tool container management system, computing device, and computer storage medium.

[0005] According to one aspect of the present invention, an intelligent tool container management method is provided, and the method includes:

[0006] Receiving a task instruction sent by a tool management platform and extracting task information from the task instruction;

[0007] According to the task information, the rail manipulator, the indicator, and the intelligent tool basket respectively perform their corresponding operations in the task information;

[0008] Based on the task information, performing a final inspection on the target tool, determining whether the final inspection result meets the preset requirements, and obtaining a judgment result;

[0009] Generating task feedback information according to the judgment result and sending it to the tool management platform.

[0010] In the above solution, according to the task information, the rail manipulator, the indicator, and the intelligent tool basket respectively perform their corresponding operations in the task information, which further includes:

[0011] Based on the task information, extract the task objectives included therein; wherein, the task objective is tool requisition or tool return.

[0012] If the task objective is tool requisition, before the rail manipulator, the indicator, and the intelligent tool basket respectively perform their operations, measure the first weight of the tool placement shelf.

[0013] Obtain the position information corresponding to the target tool, control the rail manipulator to move to the storage position of the target tool in the tool placement shelf based on this position information, after reaching the storage position, grab and take a photo record of the target tool to obtain a lending image; send a tool status modification message to the tool management platform so that the tool management platform modifies the usage status corresponding to the target tool to lent, and correspondingly modifies the usage status in the indicator at the storage position; open the lid of the intelligent tool basket.

[0014] After the grabbing is completed, control the rail manipulator to hold the grabbed state and put the target tool into the opened intelligent tool basket; measure the current second weight of the tool placement shelf, calculate the first weight change value of the tool placement shelf based on the first weight and the second weight, compare the first weight change value with the initial weight, and judge whether they are the same; if so, determine that the rail manipulator grabs correctly, generate a correct grabbing message, and send it to the tool management platform; if not, determine that the rail manipulator grabs incorrectly, generate an incorrect grabbing message, and send it to the tool management platform, and display the incorrect grabbing message in the indicator at the corresponding modified storage position.

[0015] Wherein, when the rail manipulator grabs correctly, close the lid of the intelligent tool basket; in response to the requisition opening instruction generated by the user scanning the QR code of the intelligent toolbox, open the lid of the intelligent tool basket so that the user can take out the target tool.

[0016] Wherein, the task information further includes tool basic information, that is, the name, storage position, initial image, and initial weight of the target tool.

[0017] In the above solution, according to the task information, the rail manipulator, the indicator, and the intelligent tool basket respectively perform their corresponding operations in the task information, which further includes:

[0018] If the task objective is tool return, before the rail manipulator, the indicator, and the intelligent tool basket respectively perform their operations, measure the third weight of the intelligent tool basket.

[0019] In response to the tool collection and unpacking instruction generated by the user scanning the QR code of the intelligent toolbox, open the lid of the intelligent tool basket so that the user can return the target tool; after the user puts the target tool into the intelligent tool basket, send tool status modification information to the tool management platform so that the tool management platform modifies the usage status corresponding to the target tool to returned, and correspondingly modifies the usage status in the indicator of the storage location; close the lid of the intelligent tool basket;

[0020] After the user puts the target tool into the intelligent tool basket, measure the fourth weight of the intelligent tool basket; calculate the second weight change value of the intelligent tool basket based on the third weight and the fourth weight, compare the second weight change value with the initial weight, and judge whether they are the same; if so, determine that the target tool is correctly returned, generate a correct return information, and send it to the tool management platform, control the rail robotic arm to grab the target tool from the intelligent tool basket, move it to the corresponding storage location of the target tool on the tool placement shelf after taking it out, and put and photograph the target tool after arriving at the storage location to obtain a return image.

[0021] In the above solution, the final inspection of the target tool is performed based on the task information, and it is judged whether the final inspection result meets the preset requirements to obtain a judgment result, which further includes:

[0022] If the task objective is tool collection, measure the fifth weight and the sixth weight of the intelligent tool basket before and after the user takes out the target tool from the intelligent tool basket respectively; calculate the third weight change value of the intelligent tool basket based on the fifth weight and the sixth weight, compare the third weight change value with the initial weight, and judge whether they are the same; if so, determine that the user takes the tool correctly, generate a correct taking information, and send it to the tool management platform so that the tool management platform can update the usage information of the target tool; if not, determine that the rail user takes the tool incorrectly, generate an incorrect taking information, send it to the tool management platform, and display the incorrect taking information in the indicator of the corresponding modified storage location;

[0023] If the task objective is tool return, after obtaining the return image, compare the return image with the lent image to judge whether the returned tool is correct; if so, determine that the target tool is correctly stored, generate a correct storage information, and send it to the tool management platform so that the tool management platform can update the usage information of the target tool; if not, determine that the target tool is incorrectly stored, generate an incorrect storage information, send it to the tool management platform and display the incorrect storage information in the indicator of the corresponding modified storage location;

[0024] Among them, the usage information at least includes the usage status and usage records.

[0025] In the above solution, before executing the method, it further includes:

[0026] When the tool is initially stored, the tool is photographed to generate an initial image, and the tool is measured using an electronic scale set in the intelligent tool basket or the robotic arm to obtain the initial weight;

[0027] The initial image and the initial weight corresponding to the current tool are combined with other basic information of the tool to generate the tool basic information corresponding to the current tool;

[0028] The tool basic information is bound to the corresponding tool and sent to the tool management platform for storage.

[0029] In the above solution, the method further includes:

[0030] Establish a user recognition model based on deep learning and train the user recognition model through a training set to obtain a trained user recognition model;

[0031] Based on the camera set in the intelligent tool basket, obtain the current user's face image;

[0032] Input the current user's face image into the trained user recognition model, and perform user identity determination based on the pre-stored department face image set in the tool management platform to obtain the user identity recognition result;

[0033] Based on the user identity recognition result, when it is determined that there is an image corresponding to the current user in the department face image set, the lid of the intelligent tool basket is allowed to be opened; when it is determined that there is no image corresponding to the current user in the department face image set, the lid of the intelligent tool basket is not allowed to be opened.

[0034] In the above solution, the method further includes:

[0035] During the training and application of the user recognition model, after the face image is input into the user recognition model, in the convolutional layer with an even number of layers, the face image is combined according to odd and even columns, and after the combination, the Tanh function is used as the activation function to extract the feature vector of the recombined image; The Tanh function is

[0036]

[0037] where e is the natural constant; x is the input value;

[0038] After obtaining the image features, determine the gradient features of the face region based on the corresponding histogram

[0039]

[0040] A + B + C = 1

[0041] where, is the HOG feature of the i-th face image region; is the HOG feature of the i-th face image region in the H channel; is the HOG feature of the i-th face image region in the S channel; is the HOG feature of the i-th face image region in the V channel; A, B, and C are the corresponding feature weight values of the H channel, S channel, and V channel, respectively.

[0042] According to another aspect of the present invention, an intelligent tool container management system is provided, including: a task determination module, an execution module, a detection module, and a feedback module; wherein,

[0043] The task determination module is configured to receive a task instruction sent by a tool management platform and extract task information from the task instruction;

[0044] The execution module is configured to, according to the task information, the rail manipulator, the indicator, and the intelligent tool basket respectively execute the corresponding operations in the task information;

[0045] The detection module is configured to, based on the task information, perform a final detection on the target tool, determine whether the final detection result meets the preset requirements, and obtain a judgment result;

[0046] The feedback module is configured to generate task feedback information according to the judgment result and send it to the tool management platform.

[0047] According to yet another aspect of the present invention, a computing device is provided, including: a processor, a memory, a communication interface, and a communication bus, and the processor, the memory, and the communication interface complete communication with each other through the communication bus;

[0048] The memory is used to store at least one executable instruction, and the executable instruction causes the processor to execute the operations corresponding to an intelligent tool container management method as described above.

[0049] According to still another aspect of the present invention, a computer storage medium is provided, and at least one executable instruction is stored in the storage medium, and the executable instruction causes the processor to execute the operations corresponding to an intelligent tool container management method as described above.

[0050] According to the technical solution provided by the present invention, a task instruction sent by a tool management platform is received, and task information is extracted from the task instruction; according to the task information, the orbital robotic arm, the indicator, and the intelligent tool basket respectively execute the corresponding operations in the task information; based on the task information, a final inspection is performed on the target tool, and it is judged whether the final inspection result meets the preset requirements to obtain a judgment result; a task feedback information is generated according to the judgment result and sent to the tool management platform. By obtaining the task instruction sent by the tool management platform, the orbital robotic arm, the indicator, and the intelligent tool basket are controlled to automatically execute the operations of tool grasping, displaying the tool status, and opening the box lid for the user to pick up and return the tool; by confirming the task objective, the subsequent execution process is intelligently determined. When the tool is picked up and returned, based on the pre-acquired basic information, the robotic arm is controlled to automatically grasp and automatically place it into the intelligent tool basket after the box lid is opened, or grasp it from the intelligent tool basket with the box lid opened and place it in the corresponding storage position of the tool, without manual searching, extraction, or return, greatly reducing the labor cost and accelerating the taking-out and returning speed. At the same time, by taking pictures during grasping and placing, and measuring the weight change of the shelf and the intelligent tool basket, it is judged whether the tool picking is correct, and an error is reported when it is incorrect, greatly improving the correctness and safety of tool picking, and reducing the risk of incorrect tool picking; at the same time, by sending a QR code to the user, it is ensured that only the target user can open the intelligent tool basket, further improving the security of the management process and reducing the possibility of tool loss. In addition, by setting a camera in the intelligent tool basket to obtain the user's face information, based on the constructed user recognition model, it is judged whether the current user belongs to the preset set and has relevant permissions, greatly improving the security of the tool management process; and by recombining the image features and extracting the HOG gradient features, the influence of light change on the texture features is reduced, the robustness of face recognition is improved, the discrimination ability between live faces and non-live faces is enhanced, the accuracy of face recognition is effectively improved, and the possibility of misappropriation by using photos and other means is avoided, further improving the security of tool management.

[0051] Other features and advantages of the present invention will be described in the following specification, and, in part, will be obvious from the specification, or will be understood by implementing the present invention. The objectives and other advantages of the present invention can be achieved and obtained by the structure specifically pointed out in the written specification and the drawings.

[0052] The technical solution of the present invention will be further described in detail below through the drawings and embodiments. Description of the Drawings

[0053] The drawings are used to provide a further understanding of the present invention, and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation to the present invention. In the drawings:

[0054] Figure 1 Shows a schematic flow chart of an intelligent tool container management method according to an embodiment of the present invention;

[0055] Figure 2 Shows a schematic flow chart of an intelligent tool requisition method according to an embodiment of the present invention;

[0056] Figure 3 Shows a schematic flow chart of an intelligent tool return method according to an embodiment of the present invention;

[0057] Figure 4 Shows a schematic flow chart of a tool requisition determination method according to an embodiment of the present invention;

[0058] Figure 5 Shows a schematic flow chart of a tool storage determination method according to an embodiment of the present invention;

[0059] Figure 6 Shows a schematic flow chart of a turbine maintenance access or return personnel identification method according to an embodiment of the present invention;

[0060] Figure 7 Shows a structural block diagram of an intelligent tool container management system according to an embodiment of the present invention;

[0061] Figure 8 Shows a schematic structural diagram of a computing device according to an embodiment of the present invention. Detailed implementation manners

[0062] The following describes the preferred embodiments of the present invention with reference to the accompanying drawings. It should be understood that the preferred embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not used to limit the present invention.

[0063] Figure 1 Shows a schematic flow chart of an intelligent tool container management method according to an embodiment of the present invention. The method includes the following steps:

[0064] Step S101, receive a task instruction sent by the tool management platform, and extract task information from the task instruction.

[0065] Specifically, based on the task information, extract the task objective included therein; wherein, the task objective is tool requisition or tool return.

[0066] Preferably, the task information further includes tool basic information, that is, the name, storage location, initial image, and initial weight of the target tool.

[0067] Step S102, according to the task information, the track robot arm, the indicator and the smart tool basket respectively execute the operations corresponding to the task information.

[0068] Step S103, based on the task information, a final inspection is performed on the target tool to determine whether the final inspection result meets the preset requirements and obtain a determination result.

[0069] Step S104: Generate task feedback information according to the judgment result and send it to the tool management platform.

[0070] Specifically, before executing the above steps, the method further includes:

[0071] When the tool is first stored, the tool is photographed to generate an initial image, and the tool is measured using an electronic scale set in the smart tool basket or the robotic arm to obtain an initial weight;

[0072] The initial image and initial weight corresponding to the current tool and other basic information of the tool are used together to generate tool basic information corresponding to the current tool;

[0073] The basic information of tools and instruments is bound to the corresponding tools and instruments, and sent to the tool management platform for storage.

[0074] According to the intelligent tool container management method provided by this embodiment, the task instruction issued by the tool management platform is received, and the task information is extracted from the task instruction; according to the task information, the rail manipulator, the indicator and the smart tool basket respectively perform the corresponding operations in the task information; based on the task information, the target tool is finally inspected, and the final inspection result is judged whether it meets the preset requirements to obtain the judgment result; the task feedback information is generated according to the judgment result and sent to the tool management platform. Through the intelligent tool container management method provided by this embodiment, by obtaining the task instruction issued by the tool management platform, the rail manipulator, the indicator and the smart tool basket are controlled to automatically perform the operations of tool grabbing, displaying the tool status and opening the box cover for users to take and return; by confirming the task target, the subsequent execution process is intelligently determined, and there is no need for human search, extraction or return, which greatly reduces the labor cost and speeds up the retrieval and return speed. At the same time, by performing the final inspection when the processing is completed, the correctness of the operation on the tool is judged, and an error is reported when it is incorrect, which greatly improves the correctness and safety of the tool collection, reduces the risk of tool collection errors, and improves the safety and overall efficiency of the management process.

[0075] Different task objectives are processed based on the following method flow.

[0076] Figure 2 A schematic diagram of a process of intelligently using tools according to an embodiment of the present invention is shown;

[0077] like Figure 2 As shown, the method comprises the following steps:

[0078] Step S201, measuring the first weight of the tool placed on the shelf.

[0079] Preferably, before the track robot arm, the indicator and the smart tool basket perform their respective operations, a first weight measurement of the tool shelf is performed.

[0080] Step S202, based on the position information of the tool, control the track robot arm to grab and take pictures.

[0081] Specifically, the position information corresponding to the target tool is obtained, and based on the position information, the track robot arm is controlled to move to the storage position of the target tool in the tool placement shelf. After reaching the storage position, the target tool is grasped and photographed to obtain a loan image.

[0082] Step S203, modifying the usage status of the indicator corresponding to the target tool.

[0083] Specifically, tool status modification information is sent to the tool management platform, so that the tool management platform modifies the usage status corresponding to the target tool to borrowed, and modifies the usage status in the indicator of the storage location accordingly.

[0084] Step S204, opening the cover of the smart tool basket.

[0085] Step S205 , controlling the track robot arm to maintain the grasping state and place the target tool into the opened smart tool basket.

[0086] Step S206, measuring the current second weight of the tool shelf, and calculating the first weight change value of the tool shelf based on the first weight and the second weight.

[0087] Step S207, comparing the first weight change value with the initial weight to determine whether the two are the same.

[0088] Specifically, if yes, it is determined that the track robot arm grasps correctly, and step S208 is executed; if no, it is determined that the track robot arm grasps incorrectly, and step S210 is executed.

[0089] Step S208, generate correct crawling information and send it to the tool management platform.

[0090] Specifically, when the track robot arm grasps correctly, close the box cover of the smart tool basket.

[0091] Step S209, in response to the unpacking instruction generated by the user scanning the QR code of the smart tool box, the cover of the smart tool basket is opened.

[0092] Specifically, the smart tool basket opens the box lid according to the unpacking instructions so that the user can take out the target tools.

[0093] Step S210, generating crawling error information, sending it to the tool management platform, and displaying the crawling error information in the indicator corresponding to the modified storage location.

[0094] According to the above method, by confirming the task objectives, the subsequent execution process can be intelligently determined. When tools are collected, based on the basic information obtained in advance, the robot arm is controlled to automatically grab and automatically put the tools into the smart tool basket after the lid is opened. There is no need for manual search, extraction or return, which greatly reduces labor costs and speeds up the retrieval and return. At the same time, by measuring the weight change of the shelf, it is determined whether the tools are used correctly, and an error is reported when it is incorrect, which greatly improves the correctness and safety of tool collection and reduces the risk of incorrect tool collection.

[0095] Figure 3 A schematic diagram of a process of an intelligent tool return method according to an embodiment of the present invention is shown;

[0096] like Figure 3 As shown, the method comprises the following steps:

[0097] Step S301, measuring the third weight of the smart tool basket.

[0098] Specifically, before the track robot arm, the indicator and the smart tool basket perform their respective operations, the third weight of the smart tool basket is measured; wherein the third weight is usually the empty weight of the smart tool basket without putting any items in it.

[0099] Step S302, in response to the unpacking instruction generated by the user scanning the QR code of the smart tool box, the cover of the smart tool basket is opened.

[0100] Specifically, the smart tool basket opens the lid according to the unpacking instructions so that the user can put in the target tools.

[0101] Step S303: modify the tool status displayed in the tool management platform and in the indicator.

[0102] Preferably, after the user places the target tool into the smart tool basket, tool status modification information is sent to the tool management platform so that the tool management platform modifies the usage status corresponding to the target tool to returned, and modifies the usage status in the indicator of the storage location accordingly.

[0103] Step S304, closing the cover of the smart tool basket.

[0104] Step S305: Measure the fourth weight of the intelligent tool basket, and calculate the second weight change value of the intelligent tool basket based on the third weight and the fourth weight.

[0105] Specifically, after the user places the target tool in the intelligent tool basket, measure the weight of the intelligent tool basket to obtain the fourth weight.

[0106] Step S306: Compare the second weight change value with the initial weight to determine whether they are the same.

[0107] Specifically, if they are the same, it is determined that the return of the target tool is correct, and step S307 is executed; if not, it is determined that the return of the target tool is incorrect, and step S309 is executed.

[0108] Step S307: Generate a correct return message and send it to the tool management platform.

[0109] Step S308: Control the rail manipulator to take out the target tool from the intelligent tool basket, place it in the corresponding storage location, and take a photo for record.

[0110] Specifically, open the control to grab the target tool from the intelligent tool basket by the rail manipulator. After taking it out, move it to the corresponding storage location of the target tool on the tool placement shelf. After reaching the storage location, place the target tool and take a photo for record to obtain the return image.

[0111] Step S309: Generate an incorrect return message, send it to the tool management platform, and display the incorrect return message in the indicator corresponding to the modified storage location.

[0112] According to the above method, when returning tools, based on the pre-acquired basic information, grab from the intelligent tool basket with the lid open and place it in the corresponding storage location of the tool, eliminating the need for manual search and return, greatly reducing the labor cost and accelerating the return speed. At the same time, by measuring the weight change of the intelligent tool basket during the replacement process to determine whether the tool is taken correctly, and reporting an error when it is incorrect, explaining the tool return situation to the staff, greatly improving the correctness and safety of tool return and reducing the risk of incorrect tool collection; at the same time, by sending a QR code to the user, ensuring that only the target user can open the intelligent tool basket, further enhancing the security of the management process and reducing the possibility of tool loss.

[0113] Furthermore, Figure 4 shows a schematic flowchart of a method for determining tool requisition according to an embodiment of the present invention;

[0114] As Figure 4 shown, the method includes the following steps:

[0115] Step S401, measure the fifth weight of the intelligent tool basket.

[0116] Preferably, before the user takes out the target tool from the intelligent tool basket, measure the weight of the intelligent tool basket to obtain the fifth weight.

[0117] Step S402, measure the sixth weight of the intelligent tool basket, and calculate the third weight change value of the intelligent tool basket based on the fifth weight and the sixth weight.

[0118] Preferably, before the user takes out the target tool from the intelligent tool basket, measure the weight of the intelligent tool basket to obtain the sixth weight.

[0119] Step S403, compare the third weight change value with the initial weight to determine whether they are the same.

[0120] Specifically, if so, it is determined that the user has taken the tool correctly, and step S404 is executed; if not, it is determined that the user has taken the tool incorrectly, and step S405 is executed.

[0121] Step S404, generate the information that the tool is taken correctly and send it to the tool management platform.

[0122] Specifically, by sending the information that the tool is taken correctly to the tool management platform, the tool management platform can update the usage information of the target tool.

[0123] Step S405, generate the information that the tool is taken incorrectly, send it to the tool management platform, and display the information that the tool is taken incorrectly in the indicator corresponding to the modified storage location.

[0124] Specifically, based on the information that the tool is taken incorrectly, update the usage information of the target tool on the tool management platform and display the information that the tool is taken incorrectly in the indicator.

[0125] Preferably, the usage information at least includes the usage status and usage records;

[0126] Furthermore, the usage records may include the time of each lending and returning of the tool, the lending object, and the task execution information; among which the task execution information may at least include the information of correct grasping and / or incorrect grasping, the information of correct taking and / or incorrect taking, the information of correct returning and / or incorrect returning, and the information of correct storing and / or incorrect storing.

[0127] Figure 5 Fig. shows a schematic flowchart of a method for determining the storage of tools according to an embodiment of the present invention;

[0128] As Figure 5 shown, the method includes the following steps:

[0129] Step S501, obtain the return image and the lending image.

[0130] Step S502, compare the return image with the lending image to determine whether the returned tool is correct.

[0131] Specifically, if so, it is determined that the target tool is stored correctly, and step S502 is executed; if not, it is determined that the target tool is stored incorrectly, and step S503 is executed.

[0132] Step S503, generate storage correct information and send it to the tool management platform.

[0133] Specifically, by sending the storage correct information to the tool management platform, the tool management platform can update the usage information of the target tool.

[0134] Step S504, generate storage error information, send it to the tool management platform and display the storage error information in the indicator corresponding to the modified storage location.

[0135] Specifically, based on the storage error information, the usage information of the target tool on the tool management platform is updated, and the storage error information is displayed in the indicator.

[0136] According to the above two methods, it is possible to determine whether the tool is correctly grasped, retrieved, returned or stored by taking pictures during grasping and placing, and measuring the weight changes of the shelf and the intelligent tool basket, and report an error when it is incorrect, greatly improving the correctness and safety of tool requisition and reducing the risk of incorrect tool collection.

[0137] Figure 6 Shows a schematic flow chart of a method for identifying turbine maintenance retrieval or return personnel according to an embodiment of the present invention;

[0138] As Figure 6 shown, the method includes the following steps:

[0139] Step S601, establish a user recognition model based on deep learning and train the user recognition model through a training set to obtain a trained user recognition model.

[0140] Step S602, based on the camera set in the intelligent tool basket, obtain the current user's face image.

[0141] Step S603, input the current user's face image into the trained user recognition model, and perform user identity determination based on the pre-stored department face image set in the tool management platform to obtain the user identity recognition result.

[0142] Preferably, during the training and application of the user recognition model, after the face image is input into the user recognition model, in the convolutional layer with an even number of layers, the face image is combined according to odd and even columns, and after the combination, the Tanh function is used as the activation function to extract feature vectors from the recombined image; the Tanh function is

[0143]

[0144] where e is the natural constant; x is the input value;

[0145] After obtaining the image features, the gradient features of the face region are determined based on the corresponding histogram

[0146]

[0147] A + B + C = 1

[0148] where is the HOG feature of the i-th face image region; is the HOG feature of the i-th face image region in the H channel; is the HOG feature of the i-th face image region in the S channel; is the HOG feature of the i-th face image region in the V channel; A, B, and C are the feature weight values corresponding to the H channel, S channel, and V channel, respectively.

[0149] Step S604, based on the user identity recognition result, control the intelligent tool basket to open.

[0150] Specifically, based on the user identity recognition result, when it is determined that the corresponding image of the current user exists in the department face image set, the lid of the intelligent tool basket is allowed to be opened; when it is determined that the corresponding image of the current user does not exist in the department face image set, the lid of the intelligent tool basket is not allowed to be opened.

[0151] According to the above method, by setting a camera in the intelligent tool basket, obtaining the user's face information, and based on the constructed user recognition model, it can be determined whether the current user belongs to a preset set and has relevant permissions, greatly improving the security of the tool management process; moreover, by recombining the image features and extracting HOG gradient features, the influence of light changes on texture features is reduced, the robustness of face recognition is improved, the ability to distinguish between live and non-live faces is enhanced, the accuracy of face recognition is effectively improved, the possibility of misappropriation using photos and other methods is avoided, and the security of tool management is further improved.

[0152] Figure 7 shows a structural block diagram of an intelligent tool container management system according to an embodiment of the present invention;

[0153] As Figure 7 shown, the system includes: a task determination module 701, an execution module 702, a detection module 703, and a feedback module 704; among them,

[0154] The task determination module 701 is used to receive a task instruction issued by the tool management platform and extract task information from the task instruction.

[0155] Specifically, the task determination module 701 is further used to,

[0156] Based on the task information, extract the task objectives included therein; where the task objectives are tool requisition or tool return;

[0157] The task information also includes tool basic information, that is, the name, storage location, initial image, and initial weight of the target tool.

[0158] Preferably, the task determination module 701 is further used to,

[0159] When the tool is initially stored, take a photo of the tool to generate an initial image, and use an electronic scale set in the intelligent tool basket or robotic arm to measure the tool to obtain the initial weight;

[0160] Generate the tool basic information corresponding to the current tool by combining the initial image and initial weight corresponding to the current tool with other basic information of the tool;

[0161] Bind the tool basic information to the corresponding tool and send it to the tool management platform for storage.

[0162] The execution module 702 is used to execute the respective corresponding operations in the task information by the rail robotic arm, indicator, and intelligent tool basket according to the task information.

[0163] Specifically, the execution module 702 is further used to,

[0164] If the task objective is tool requisition, measure the first weight of the tool placement shelf before the rail robotic arm, indicator, and intelligent tool basket execute their respective operations;

[0165] Obtain the position information corresponding to the target tool, control the rail robotic arm to move to the storage location of the target tool in the tool placement shelf based on the position information, grab and take a photo record of the target tool after reaching the storage location to obtain a lent-out image; send tool status modification information to the tool management platform so that the tool management platform modifies the usage status corresponding to the target tool to lent out and correspondingly modifies the usage status in the indicator at the storage location; open the lid of the intelligent tool basket;

[0166] After the grasping is completed, control the rail manipulator to keep the grasped state and put the target tool into the opened intelligent tool basket; measure the current second weight of the tool placement shelf, calculate the first weight change value of the tool placement shelf based on the first weight and the second weight, compare the first weight change value with the initial weight, and determine whether they are the same; if so, determine that the rail manipulator grasps correctly, generate a correct grasping message, and send it to the tool management platform; if not, determine that the rail manipulator grasps incorrectly, generate an incorrect grasping message, and send it to the tool management platform, and display the incorrect grasping message in the indicator corresponding to the modified storage location;

[0167] Among them, when the rail manipulator grasps correctly, close the lid of the intelligent tool basket; in response to the tool-taking and opening instruction generated by the user scanning the QR code of the intelligent toolbox, open the lid of the intelligent tool basket so that the user can take out the target tool.

[0168] Specifically, the execution module 702 is further configured to,

[0169] If the task target is tool return, measure the third weight of the intelligent tool basket before the rail manipulator, the indicator, and the intelligent tool basket perform their respective operations;

[0170] In response to the tool-taking and opening instruction generated by the user scanning the QR code of the intelligent toolbox, open the lid of the intelligent tool basket so that the user can return the target tool; after the user puts the target tool into the intelligent tool basket, send a tool status modification message to the tool management platform so that the tool management platform modifies the usage status corresponding to the target tool to returned, and correspondingly modifies the usage status in the indicator of the storage location; close the lid of the intelligent tool basket;

[0171] After the user puts the target tool into the intelligent tool basket, measure the fourth weight of the intelligent tool basket; calculate the second weight change value of the intelligent tool basket based on the third weight and the fourth weight, compare the second weight change value with the initial weight, and determine whether they are the same; if so, determine that the target tool is returned correctly, generate a correct return message, and send it to the tool management platform, control the rail manipulator to grasp the target tool from the intelligent tool basket, move it to the corresponding storage location of the target tool on the tool placement shelf after taking it out, and put and photograph the target tool after reaching the storage location to obtain a return image.

[0172] Specifically, the execution module 702 is further configured to,

[0173] Establish a user recognition model based on deep learning and train the user recognition model through a training set to obtain a trained user recognition model;

[0174] Obtain the current user's face image based on the camera set in the intelligent tool basket;

[0175] Input the current user's face image into the trained user recognition model, and perform user identity determination based on the pre-stored department face image set in the tool management platform to obtain the user identity recognition result;

[0176] Based on the user identity recognition result, when it is determined that there is an image corresponding to the current user in the department face image set, allow the lid of the intelligent tool basket to be opened; when it is determined that there is no image corresponding to the current user in the department face image set, do not allow the lid of the intelligent tool basket to be opened.

[0177] Preferably, the execution module 702 is further configured to,

[0178] During the training and application of the user recognition model, after the face image is input into the user recognition model, in the convolutional layer with an even number of layers, the face image is combined according to odd and even columns, and after combination, the Tanh function is used as the activation function to extract feature vectors from the recombined image; The Tanh function is

[0179]

[0180] where e is the natural constant; x is the input value;

[0181] After obtaining the image features, determine the gradient features of the face area based on the corresponding histogram

[0182]

[0183] A + B + C = 1

[0184] where, is the HOG feature of the i-th face image region; is the HOG feature of the i-th face image region in the H channel; is the HOG feature of the i-th face image region in the S channel; is the HOG feature of the i-th face image region in the V channel; A, B, and C are the feature weight values corresponding to the H channel, S channel, and V channel respectively.

[0185] The detection module 703 is configured to perform a final detection on the target tool based on the task information, and determine whether the final detection result meets the preset requirements to obtain a judgment result.

[0186] Specifically, the detection module 703 is further configured to,

[0187] If the task objective is to pick up tools, measure the fifth weight and the sixth weight of the intelligent tool basket before and after the user takes out the target tool from the intelligent tool basket respectively; calculate the third weight change value of the intelligent tool basket based on the fifth weight and the sixth weight, compare the third weight change value with the initial weight, and determine whether they are the same; if so, it is determined that the user picks up correctly, generate a correct pick-up message, and send it to the tool management platform for the tool management platform to update the usage information of the target tool; if not, it is determined that the user picks up incorrectly, generate an incorrect pick-up message, send it to the tool management platform, and display the incorrect pick-up message in the indicator corresponding to the modified storage location.

[0188] If the task objective is to return tools, after obtaining the return image, compare the return image with the lent image to determine whether the returned tool is correct; if so, it is determined that the target tool is stored correctly, generate a correct storage message, and send it to the tool management platform for the tool management platform to update the usage information of the target tool; if not, it is determined that the target tool is stored incorrectly, generate an incorrect storage message, send it to the tool management platform and display the incorrect storage message in the indicator corresponding to the modified storage location.

[0189] Among them, the usage information at least includes the usage status and usage records.

[0190] The feedback module 704 is used to generate a task feedback message according to the judgment result and send it to the tool management platform.

[0191] The intelligent tool container management system provided according to the present embodiment includes: a task determination module, an execution module, a detection module and a feedback module; wherein the task determination module is used to receive a task instruction issued by a tool management platform and extract task information from the task instruction; the execution module is used to respectively execute the corresponding operations in the task information by the track robot arm, the indicator and the smart tool basket according to the task information; the detection module is used to perform a final inspection of the target tool based on the task information, determine whether the final inspection result meets the preset requirements, and obtain a judgment result; the feedback module is used to generate task feedback information according to the judgment result, and send it to the tool management platform. Through the intelligent tool container management system provided by the present embodiment, by obtaining the task instructions issued by the tool management platform, the rail robot arm, the indicator and the smart tool basket are controlled to automatically perform the operations of grabbing tools, displaying tool status and opening the box cover for users to take and return tools; by confirming the task target, the subsequent execution process is intelligently determined. When the tools are taken and returned, based on the pre-acquired basic information, the robot arm is controlled to automatically grab and automatically put them into the smart tool basket after opening the box cover, or grab them from the smart tool basket with the box cover opened and put them into the storage location corresponding to the tools. There is no need for manual search, extraction or return, which greatly reduces the labor cost and speeds up the retrieval and return speed. At the same time, by taking pictures and measuring the weight changes of the shelves and the smart tool basket when grabbing and putting back, it is judged whether the tools are taken correctly, and an error is reported when it is incorrect, which greatly improves the correctness and safety of tool collection and reduces the risk of incorrect tool collection; at the same time, based on the use of sending a QR code to the user, it is ensured that only the target user can open the smart tool basket, which further improves the security of the management process and reduces the possibility of tool loss. In addition, by setting up a camera in the smart tool basket to obtain user facial information, based on the constructed user recognition model, it is determined whether the current user belongs to the preset collection and has relevant permissions, which greatly improves the security of the tool management process; and, by recombining image features and extracting HOG gradient features, the impact of illumination changes on texture features is reduced, the robustness of face recognition is improved, and the ability to distinguish between living faces and non-living faces is enhanced, which effectively improves the accuracy of face recognition, avoids the possibility of impersonation through photos, etc., and further improves the security of tool management.

[0192] The present invention also provides a non-volatile computer storage medium, which stores at least one executable instruction, and the executable instruction can execute an intelligent tool container management method in any of the above method embodiments.

[0193] Figure 8 A schematic diagram of the structure of a computing device according to an embodiment of the present invention is shown. The specific embodiment of the present invention does not limit the specific implementation of the computing device.

[0194] As shown Figure 8 in the figure, the computing device may include: a processor 802, a communications interface 804, a memory 806, and a communication bus 808.

[0195] Among them:

[0196] The processor 802, the communications interface 804, and the memory 806 communicate with each other through the communication bus 808.

[0197] The communications interface 804 is used to communicate with network elements of other devices such as clients or other servers.

[0198] The processor 802 is used to execute the program 810, and specifically can execute the relevant steps in the above-mentioned embodiments of an intelligent tool container management method.

[0199] Specifically, the program 810 may include program code, and the program code includes computer operation instructions.

[0200] The processor 802 may be a central processing unit CPU, or a specific integrated circuit ASIC (Application Specific Integrated Circuit), or one or more integrated circuits configured to implement the embodiments of the present invention. One or more processors included in the computing device may be of the same type of processor, such as one or more CPUs; or may be of different types of processors, such as one or more CPUs and one or more ASICs.

[0201] The memory 806 is used to store the program 810. The memory 806 may include high-speed RAM memory, and may also include non-volatile memory, such as at least one disk memory.

[0202] The program 810 is specifically used to cause the processor 802 to execute an intelligent tool container management method in any of the above method embodiments. For the specific implementation of each step in the program 810, reference may be made to the corresponding steps and units in the above-mentioned embodiments of an intelligent tool container management method, which will not be elaborated here. Those skilled in the art can clearly understand that for the convenience and brevity of description, the specific working processes of the above-described devices and modules can refer to the corresponding process descriptions in the foregoing method embodiments, which will not be elaborated here.

[0203] The algorithms and displays provided herein are not inherently related to any particular computer, virtual system, or other device. A variety of general-purpose systems may also be used in conjunction with the teachings presented herein. The structure required to construct such systems will be apparent from the above description. Additionally, the present invention is not directed to any particular programming language. It should be understood that the teachings of the present invention described herein can be implemented in a variety of programming languages, and the description of specific languages above is provided to disclose the best mode of the present invention.

[0204] In the specification provided herein, numerous specific details are set forth. However, it can be understood that embodiments of the present invention may be practiced without these specific details. In some instances, well-known methods, structures, and techniques have not been shown in detail so as not to obscure the understanding of this specification.

[0205] Similarly, it should be understood that, in order to streamline this disclosure and assist in understanding one or more of the various inventive aspects, in the foregoing description of exemplary embodiments of the present invention, various features of the present invention are sometimes grouped together in a single embodiment, figure, or description thereof. However, the disclosed method should not be construed as reflecting an intention that the claimed invention requires more features than are expressly recited in each claim. Rather, as the claims reflect, the inventive aspects lie in less than all of the features of the preceding single embodiment. Thus, the claims following the detailed description are hereby expressly incorporated into this detailed description, with each claim standing on its own as a separate embodiment of the present invention.

[0206] Those skilled in the art will appreciate that the modules in the devices in the embodiments can be adaptively changed and disposed in one or more devices different from the embodiments. The modules or units or components in the embodiments can be combined into one module or unit or component, and in addition, they can be divided into multiple sub-modules or sub-units or sub-components. Except that at least some of such features and / or processes or units are mutually exclusive, any combination can be used to combine all the features disclosed in this specification (including the accompanying claims, abstract, and drawings) and all the processes or units of any method or device so disclosed. Unless otherwise expressly stated, each feature disclosed in this specification (including the accompanying claims, abstract, and drawings) can be replaced by an alternative feature providing the same, equivalent, or similar purpose.

[0207] In addition, those skilled in the art can understand that although some embodiments described herein include certain features included in other embodiments rather than other features, the combination of features of different embodiments means that it is within the scope of the present invention and forms different embodiments. For example, in the claims, any one of the claimed embodiments can be used in any combination.

[0208] Each component embodiment of the present invention can be implemented in hardware, or in software modules running on one or more processors, or in a combination thereof. Those skilled in the art should understand that a microprocessor or a digital signal processor (DSP) can be used in practice to implement some or all of the functions of some or all of the components according to the embodiments of the present invention. The present invention can also be implemented as a device or apparatus program (such as a computer program and a computer program product) for executing part or all of the methods described herein. Such a program implementing the present invention can be stored on a computer-readable medium, or can be in the form of one or more signals. Such signals can be downloaded from an Internet website, or provided on a carrier signal, or provided in any other form.

[0209] Obviously, those skilled in the art can make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalent technologies, the present invention is also intended to include these changes and modifications.

Claims

1. A method for managing a container of intelligent tools, comprising: Receive task instructions issued by the tool management platform and extract task information from the task instructions; According to the task information, the track robot arm, the indicator and the smart tool basket respectively perform the operations corresponding to the task information; Based on the task information, the target tool is finally inspected to determine whether the final inspection result meets the preset requirements and obtain the judgment result; Generate task feedback information based on the judgment results and send it to the tool management platform.

2. The method according to claim 1, characterized in that According to the task information, the track robot arm, the indicator and the smart tool basket respectively perform the operations corresponding to the task information, further comprising: Based on the task information, extract the task objectives contained therein; wherein the task objectives are the use of tools or the return of tools; If the task goal is to collect tools, the first weight of the tools on the shelf is measured before the track robot arm, indicator and smart tool basket perform their respective operations; Obtain the position information corresponding to the target tool, and based on the position information, control the rail robot arm to move to the storage position of the target tool in the tool placement shelf, grab and take photos of the target tool after arriving at the storage position, and obtain a loan image; send tool status modification information to the tool management platform, so that the tool management platform changes the use status corresponding to the target tool to loaned, and correspondingly modifies the use status in the indicator of the storage position; open the box cover of the smart tool basket; After the grasping is completed, the rail robot arm is controlled to maintain the grasping state and place the target tool into the opened smart tool basket; the current second weight of the tool shelf is measured, and the first weight change value of the tool shelf is calculated based on the first weight and the second weight, and the first weight change value is compared with the initial weight to determine whether the two are the same; if so, it is determined that the rail robot arm grasps correctly, generates grasping correct information, and sends it to the tool management platform; if not, it is determined that the rail robot arm grasps incorrectly, generates grasping error information, and sends it to the tool management platform, and displays the grasping error information in the indicator corresponding to the modified storage position; When the track robot arm grabs correctly, the cover of the smart tool basket is closed; in response to the unpacking instruction generated by the user scanning the QR code of the smart tool box, the cover of the smart tool basket is opened so that the user can take out the target tool; The task information also includes basic information of tools, namely the name, storage location, initial image, and initial weight of the target tools.

3. The method according to claim 2, characterized in that According to the task information, the track robot arm, the indicator and the smart tool basket respectively perform the operations corresponding to the task information, further comprising: If the task goal is to return the tool, the third weight of the smart tool basket is measured before the track robot arm, the indicator, and the smart tool basket perform their respective operations; In response to the unpacking instruction generated by the user scanning the QR code of the smart tool box, the cover of the smart tool basket is opened so that the user can return the target tool; after the user puts the target tool into the smart tool basket, tool status modification information is sent to the tool management platform so that the tool management platform modifies the use status corresponding to the target tool to return, and modifies the use status in the indicator of the storage position accordingly; the cover of the smart tool basket is closed; After the user puts the target tool into the smart tool basket, the fourth weight of the smart tool basket is measured; the second weight change value of the smart tool basket is calculated based on the third weight and the fourth weight, and the second weight change value is compared with the initial weight to determine whether the two are the same; if so, it is determined that the target tool is returned correctly, and the correct return information is generated and sent to the tool management platform, and the track robot arm is controlled to grab the target tool from the smart tool basket, take it out and move it to the storage position corresponding to the target tool in the tool placement shelf, and after arriving at the storage position, the target tool is placed and photographed to obtain a return image.

4. The method according to claim 1, characterized in that: The method further comprises: performing a final inspection on the target tool based on the task information, judging whether the final inspection result meets the preset requirements, and obtaining the judgment result. If the task goal is to collect tools, before and after the user takes the target tools out of the smart tool basket, measure the fifth weight and the sixth weight of the smart tool basket respectively; calculate the third weight change value of the smart tool basket based on the fifth weight and the sixth weight, and compare the third weight change value with the initial weight to determine whether the two are the same; if so, determine that the user has taken the tools correctly, generate correct information, and send it to the tool management platform so that the tool management platform can update the usage information of the target tools; if not, determine that the user has taken the tools incorrectly, generate and send information about the use error to the tool management platform, and display the information about the use error in the indicator corresponding to the modified storage position; If the task target is to return the tool, after obtaining the return image, the return image is compared with the loaned image to determine whether the returned tool is correct; if so, it is determined that the target tool is stored correctly, and correct storage information is generated and sent to the tool management platform so that the tool management platform can update the usage information of the target tool; if not, it is determined that the target tool is stored incorrectly, and storage error information is generated and sent to the tool management platform and displayed in the indicator corresponding to the modified storage position; The usage information at least includes usage status and usage records.

5. The method according to claim 1, characterized in that Before executing the method, the method further comprises: When the tool is first stored, the tool is photographed to generate an initial image, and the tool is measured using an electronic scale set in the smart tool basket or the robotic arm to obtain an initial weight; The initial image and initial weight corresponding to the current tool and other basic information of the tool are combined to generate the tool basic information corresponding to the current tool; The basic information of tools and instruments is bound to the corresponding tools and instruments, and sent to the tool management platform for storage.

6. The method according to claim 1, characterized in that The method further comprises: Establish a user recognition model based on deep learning and train the user recognition model through a training set to obtain a trained user recognition model; Based on the camera set in the smart tool basket, obtain the current user's face image; Input the current user's face image into the trained user recognition model, perform user identity determination based on the department face image set pre-stored in the tool management platform, and obtain the user identity recognition result; Based on the user identity recognition results, when it is determined that the department face image set contains an image corresponding to the current user, the lid of the smart tool basket is allowed to be opened; when it is determined that the department face image set does not contain an image corresponding to the current user, the lid of the smart tool basket is not allowed to be opened.

7. The method according to claim 6, characterized in that The method further comprises: In the user recognition model training and application process, after the face image is input into the user recognition model, in the convolution layer with an even number of layers, the face image is combined according to the odd columns and the even columns, and after the combination, the feature vector of the recombined image is extracted using the Tanh function as the activation function; the Tanh function is Among them, e is a natural constant; x is an input value; After obtaining the image features, the gradient features of the face area are determined based on the corresponding histogram A+B+C=1 in, is the HOG feature of the i-th face image area; is the HOG feature of the i-th face image region in the H channel; is the HOG feature of the i-th face image region in the S channel; is the HOG feature of the i-th face image region in the V channel; A, B, and C are the feature weight values ​​corresponding to the H channel, S channel, and V channel respectively.

8. An intelligent tool container management system, comprising: Task determination module, execution module, detection module and feedback module; among them, The task determination module is used to receive the task instruction issued by the tool management platform and extract task information from the task instruction; The execution module is used to respectively execute the operations corresponding to the task information by the track robot arm, the indicator and the smart tool basket according to the task information; The detection module is used to perform a final detection on the target tool based on the task information, determine whether the final detection result meets the preset requirements, and obtain a judgment result; The feedback module is used to generate task feedback information according to the judgment result and send it to the tool management platform.

9. A computing device comprising: A processor, a memory, a communication interface and a communication bus, wherein the processor, the memory and the communication interface communicate with each other via the communication bus; The memory is used to store at least one executable instruction, and the executable instruction enables the processor to perform operations corresponding to the intelligent tool container management method according to any one of claims 1-7.

10. A computer storage medium, wherein at least one executable instruction is stored in the storage medium, and the executable instruction enables a processor to execute operations corresponding to the intelligent tool container management method according to any one of claims 1 to 7.

Images