Tool in-out management device based on ai vision and management method thereof

Abstract

The application discloses an AI vision-based tool warehouse-in and warehouse-out management device and method, belongs to the technical field of house management, and aims to solve the problem of house management confusion, wherein the device comprises a sensing module, a data processing module and a system integration module, the modules are in communication connection, and information transmission and processing among the modules are realized; the sensing module is used to acquire the state of tools in the house, and comprises AI camera equipment arranged at the entrances and exits of the house, which is used to monitor the tool warehouse-in and warehouse-out; the data processing module is used to process data from the sensing module, realize analysis and record of the tool state, and comprises a special computer and a backup power supply; the special computer is used to process and record data from the sensing module, and the backup power supply is used to provide backup power for the AI vision-based tool warehouse-in and warehouse-out management device; the system integration module is used for digital management and visual management of tools, and comprises a house management platform, which realizes dynamic real-time update and visibility of tools.

Description

Technical Field

[0001] This invention relates to the field of railway maintenance workshop technology, and in particular to a method for managing the entry and exit of tools. Background Technology

[0002] Railway maintenance depots generally employ a manned management system for their workshops and warehouses, staffed with dedicated warehouse managers. Currently, these warehouses suffer from issues such as mixed storage of tools and materials, low efficiency of manual inventory checks, incomplete return of tools, non-theft losses, and discrepancies between spare parts consumption and records. The failure to follow procedures for issuing medium and large-sized tools and equipment from the warehouse, resulting in their abandonment on railway lines, poses a significant risk to train safety. During railway maintenance operations, maintenance workers obtain a wide variety of tools and equipment from the warehouse, ranging from large hydraulic track jacks and internal combustion tamping picks to small screwdrivers, wrenches, and steel tape measures. Comprehensive digital management covering all categories presents a serious challenge.

[0003] To improve the digitalization of warehouse management, common technical approaches include QR code / barcode labels, RFID, and AI visual recognition. A comparison of related technical solutions is as follows: (1) QR code / barcode technology solution Self-adhesive thermal QR code labels are affixed to the surfaces of tools and equipment, and each item is scanned manually or by a fixed scanning device when entering or leaving the warehouse. While this solution has extremely low label costs, it has significant shortcomings in railway maintenance field applications: Railway tools and equipment surfaces are often covered with oil, mud, and dust, easily obscuring the thermal QR codes and rendering them unreadable; under high-intensity operation, labels suffer severe wear, detachment, and warping, leading to a rapid increase in maintenance costs; and the requirement for visual scanning of each item results in long process interruptions, contradicting the "rapid dispatch" principle required for emergency repairs. Especially for small hand tools (such as screwdrivers, wrenches, and pliers), although the individual value is low, the large quantity means that the workload and cost of label maintenance are relatively high compared to the tool's value. On-site personnel often resist scanning due to the cumbersome operation, rendering the system ineffective. In confined spaces such as station repair points where tools are frequently moved, ordinary self-adhesive labels have poor oil and wear resistance, typically lasting less than a month, making frequent replacements and maintenance work unbearable.

[0004] (2) RFID technology solution Anti-metal passive RFID tags are installed on tools and equipment, enabling batch identification via fixed readers or handheld terminals. Ideally, RFID offers theoretical advantages such as contactless operation, batch reading, and unobstructed access. However, in real-world railway engineering conditions, practical experience has revealed the following insurmountable difficulties: Significant interference from metal: The vast majority of railway tools and equipment are made of metal. When ordinary RFID tags are attached to metal surfaces, the reading distance decreases from the theoretical 3-5 meters to 10-30 centimeters. Even with anti-metal tags, the reading distance still decreases by 60%-80%. If standard base plates are used, the installation space and ease of use cannot be met. When tools and equipment are stacked (such as multiple tools mixed together in a tool cart), the metal blocks each other, preventing the signal from penetrating, and batch identification becomes essentially ineffective.

[0005] Poor environmental adaptability: Oil stains covering the label surface cause signal attenuation, reducing the reading success rate by 30%-50%; on high-frequency vibration equipment such as tamping picks, the label solder joints fatigue and break, resulting in a service life of less than 3 months; in temperature differences of -20℃ to 60℃, the adhesive backing fails, and the label falls off in 3-6 months.

[0006] Cost imbalance: Industrial-grade anti-metal tags cost 15-50 yuan each, installation labor costs 5-15 yuan per tag, and the annual replacement rate is 20%-30%; handheld readers cost 3000-8000 yuan per unit, and fixed readers cost 3000-5000 yuan per unit. For small hand tools such as screwdrivers, wrenches, fitter's hammers, and steel tape measures, with a unit price of less than 100 yuan, the total cost of RFID tags and maintenance will exceed the value of the tools themselves within 6-12 months, and continuous maintenance costs are required every year, resulting in a serious cost inversion.

[0007] Practical challenges in engineering: Drilling holes to fix the handles of small hand tools reduces tool strength, which the track maintenance department explicitly opposes; adhesive methods are unreliable in vibrating or oily environments; replacing damaged labels requires specialized tools, leading to low cooperation from on-site personnel; raised labels are uncomfortable to handle and scratch clothing, resulting in workers frequently removing them voluntarily. Furthermore, labeling precision measuring instruments such as track gauges and track rulers may affect their accuracy and feel, causing significant worker resistance.

[0008] In summary, RFID technology is feasible for large and medium-sized equipment in railway maintenance tool management scenarios. However, it has serious deficiencies in terms of economy and engineering feasibility for small hand tools that are numerous and inexpensive, and is not suitable as a core management method covering all categories.

[0009] (3) Intelligent Tool Cabinet Solution Each cabinet door has a built-in reader antenna. Tools must be placed in designated slots or hooks, and inventory is automatically taken when the door is closed. The cost per cabinet is 30,000-80,000 yuan, and it can only manage the tools inside, not all materials in the entire warehouse and station repair point. The warehouse itself is a "large tool cabinet," making this solution unsuitable for this scenario, and each tool still needs an RFID tag, facing the same maintenance challenges. For scenarios with a large number of small tools, several or even dozens of smart tool cabinets would need to be deployed, which is unacceptable in terms of cost and cannot cover medium to large tools. Furthermore, if a power outage occurs in the warehouse or the smart tool cabinet malfunctions during railway emergency repairs, and the repair tools cannot be retrieved, the repair work will be forced to stop, the line fault cannot be dealt with in a timely manner, causing train traffic obstruction and increased operational risks. Forcibly opening the cabinet without authorization could damage the tool cabinet equipment, and misusing alternative tools could also exacerbate the line fault risk.

[0010] (4) Millimeter-wave radar imaging scheme While it can identify object outlines by penetrating non-metallic materials, the dense metal shelves and tools in the warehouse cause severe multipath reflection and signal interference, making it impossible to reliably identify the type, specifications, and quantity of tools, especially small tools which are completely unidentifiable. The cost of a single device is as high as millions of yuan, and its large size and demanding deployment requirements are seriously inconsistent with the practical needs of low-cost operation and maintenance and large-scale deployment in this scenario.

[0011] (5) Shortcomings of existing AI vision solutions Pure AI visual recognition solutions exhibit significant fluctuations in accuracy under complex scenarios involving occlusion, clutter, oil stains, and wear. For small hand tools (such as wrenches, pliers, and screwdrivers), due to their small size, similar features, and susceptibility to occlusion, the recognition accuracy may be below 70%. Furthermore, the lack of effective human collaboration confirmation mechanisms and fallback measures makes it difficult to meet the rigid management requirements of railway engineering sections for "consistency between accounts and actual conditions, and a closed-loop safety system."

[0012] In summary, existing technical solutions, in the specific scenario of railway maintenance warehouses, are limited in their management of all types of tools and equipment (especially small hand tools) due to environmental adaptability, cost, operational efficiency requirements, or identification reliability. They are all unable to meet the engineering implementation requirements of "not changing front-line work habits, not affecting emergency repair efficiency, low cost and maintainability, and full category coverage".

[0013] The patent search results are as follows: Beijing Jiaotong University, CN202511068533.8, discloses a computer vision-based method for checking the inventory and verification of railway tools and equipment. This method includes acquiring images of tools and equipment at high-speed railway maintenance depots and in warehouses; cropping target regions using a region focusing algorithm to eliminate interference; constructing a diverse training dataset by combining perspective transformation, multi-transformation stitching, and generation methods based on SAM and diffusion models; building a text-visual multimodal rotating target detection model; enhancing feature extraction by fusing textual information and visual features from work plans to guide the model to focus on key categories; adjusting detection parameters using an adaptive threshold optimization algorithm; combining few-shot detection technology; and comparing the results with the work plan after category-aware scale filtering, outlier suppression, and cross-class non-maximum suppression. Finally, a sample cropping and screening mechanism driven by false positives and false negatives is used to extract high-value samples to supplement the training set. This method improves the safety and efficiency of railway tool and equipment management.

[0014] Guangzhou Binju Technology Co., Ltd., CN202010467531.7, discloses a method, system, device, and storage medium for detecting railway tools and equipment. The method includes acquiring multiple images of the tools and equipment; creating a dataset based on all the images; constructing a deep convolutional neural network; building a reflection image extraction network using the dataset and the deep convolutional neural network; obtaining a feature detection network based on the deep convolutional neural network and the reflection image extraction network; obtaining an initial detection network model based on the deep convolutional neural network, the reflection image extraction network, and the feature detection network; training the initial detection network model using the dataset to obtain a target detection network model; and detecting the images of the tools and equipment to be detected based on the target detection network model to obtain the detection results. This invention effectively solves problems such as complex backgrounds, uneven lighting, large differences in target scale, complex shapes, and occlusion, enabling rapid and accurate target detection of railway tools and equipment, and achieving automatic inventory of railway tools and equipment.

[0015] Wuhan University of Technology and Wuhan Yinxing Technology Co., Ltd. have developed a method and apparatus for counting railway tools and equipment. The method includes: reading multiple sample images of tools and equipment from a public database and preprocessing these images to obtain a training image dataset; constructing a target detection network and training it using the training image dataset to obtain a target detection model; capturing images of the work site using a camera to obtain images to be processed; and comparing and analyzing the images to be processed according to the target detection model to obtain the name and quantity of the lost tools and equipment. This invention achieves accurate detection and counting of tools and equipment, meeting practical application needs. It eliminates the need for manual extraction of image features, saving time and improving real-time performance and accuracy, thus laying a technical foundation for intelligent railway operation and maintenance.

[0016] Zhengzhou Sanfang Software Technology Co., Ltd., CN201911407063.8, provides a high-speed railway tooling, equipment, materials, and operation management system and method. The system includes: a cloud server and a warehouse host terminal and a handheld terminal connected to the cloud server; a first RFID reader and its antenna connected to the warehouse host terminal via a 4G wireless router are installed on the warehouse host terminal side; the handheld terminal includes an operation plan creation module, a process module, an inventory module, a 4G mobile communication module, and a core processing module; the operation plan creation module is used to create operation plans on the cloud server; the core processing module is used to acquire inventory information and stage information and send them to the cloud server; the cloud server is used to save the operation plan, send the operation plan to the warehouse host terminal, and receive inventory information and scanning information; the first RFID reader and its antenna are used to scan RFID tags; the warehouse host terminal is used to receive and display the operation plan; acquire scanning information, and send the scanning information to the cloud server. Summary of the Invention

[0017] In view of the above-mentioned problems of inconsistent management of railway engineering tools and equipment, such as disorderly placement, low efficiency of manual inventory, incomplete return of tools and equipment, loss of non-theft items, and discrepancies between spare parts consumption and ledgers, it is necessary to propose a tool entry and exit management device and its management method based on AI vision. Under the premise of not forcibly changing the front-line operation habits, not being demanding on tool placement, and not affecting emergency repair efficiency, it can achieve entry and exit management of all categories of tools, in order to achieve the goal of non-disruptive entry and exit.

[0018] An AI vision-based tool entry and exit management device includes a sensing module, a data processing module, and a system integration module. The modules are interconnected to enable information transmission and processing between them. The sensing module is used to obtain the status of the tools in the work room, including AI camera equipment set at the entrance and exit of the work room; The data processing module is used to process data from the sensing module, and to analyze and record the tool status. It includes a dedicated computer and a backup power supply. The dedicated computer is used to process and record data from the sensing module, and the backup power supply is used to provide backup power to the AI ​​vision-based tool inbound and outbound management device to prevent the risk of the AI ​​vision-based tool inbound and outbound management device failing to operate normally due to a sudden power outage. The system integration module is used for the digital and visual management of tools, including the engineering room management platform, which enables the tools to be dynamically updated and visible at any time.

[0019] Furthermore, the AI ​​vision-based tool entry and exit management device also includes an auxiliary module to supplement the shortcomings of the sensing module. When the sensing module cannot obtain the status of the tools in the workshop, the auxiliary module can collect or label the status of the tools. This includes an OCR terminal, a thermal printer, and a laser marking machine. The thermal printer and laser marking machine can generate identification codes in the form of QR codes or characters to identify tools, equipment, or spare parts. The OCR terminal is used to read the identification codes on the tools to monitor the tools.

[0020] Furthermore, the sensing module also includes infrared camera devices installed at the four corners of the work room to monitor whether tools or equipment in various areas of the work room are in place. By capturing real-time images of the storage areas of various tools and equipment in the work room and comparing them with standard images, and then comparing the status of each tool and equipment with the work room management platform, the status of the tools and equipment is verified, ensuring that the status of the tools and equipment is always up-to-date and correct, which facilitates the correct generation of subsequent work orders.

[0021] Furthermore, the system integration module also includes a work order platform, which is set up on a host computer. The maintenance supervisor assigns tasks to the maintenance department and workers according to maintenance, emergency repair, and other requirements. The work order also includes a material requisition form, which details the tools, equipment, spare parts, etc., needed for the task. The administrator's dedicated computer can receive the work orders, facilitating the issuance of tools and equipment.

[0022] A tool inbound / outbound management method based on AI vision is proposed. It primarily employs AI dual-view visual recognition, supplemented by OCR laser etching of ASCII characters, laser etching of QR codes, or thermal transfer of matte silver QR codes. This method utilizes a combination of warehouse administrator and human-machine collaborative confirmation to achieve the management objectives of non-intrusive tool inbound / outbound operations, dual-view coverage, manned confirmation, and human-machine collaboration. Furthermore, it achieves synchronization of tool logistics and information flow, including... Initialization: Initialize the virtual entry / exit boundary zone; the virtual entry / exit boundary zone is the coordinate position corresponding to the actual location of the engineering room entrance / exit; Image acquisition: AI camera equipment is used to acquire images of tools or equipment entering and exiting the maintenance room; Determining whether a tool is leaving or entering the warehouse: Based on whether the tool cart is getting closer to or farther away from the virtual entry / exit boundary area as it passes through the tool cart, the system determines whether the tool is leaving or entering the warehouse. Tool identification: The data processing module identifies the type and quantity of tools by analyzing images of tools entering and leaving the database collected by the perception module. Administrator Confirmation: During normal maintenance and repair tasks, the administrator confirms the tools issued on a dedicated computer before the tools are issued or stored. During emergency repair tasks, the administrator can skip confirmation and directly issue tools and / or equipment, spare parts, etc. The operator must submit a supplementary requisition form within 24 hours, and the administrator will conduct supplementary confirmation. If the deadline is exceeded, an assessment will be conducted. Update the ledger: Update the tool ledger in the engineering room management platform to achieve dynamic management of tools, and compare the material requisition forms in the work order with the tool ledger to determine the accuracy of the latest tool status.

[0023] Furthermore, the specific process for determining entry and exit from the warehouse is as follows: Track moving targets and acquire images of tools on the vehicle according to the set frame interval; The tool vehicle shall continuously acquire images of crossing the virtual entry and exit boundary zone at least three times; To determine the stability and validity of consecutive frame actions, if the same tool or instrument can be captured in the images of consecutive frames, the consecutive frame actions are considered stable. If the content recorded in the images is inconsistent, it is considered invalid, and the process returns to the previous step to continue capturing images. If it is determined to be stable, the tool cart passes safely, and it is considered a valid outbound or valid inbound operation. For tool entry / exit determination, if the distance between the image of the next frame and the entrance / exit of the maintenance room is greater than that of the previous frame, it is determined as tool entry; if the distance between the image of the next frame and the entrance / exit of the maintenance room is less than that of the previous frame, it is determined as tool exit.

[0024] Furthermore, the AI ​​vision-based tool inventory management method also includes material change verification, which is achieved through infrared cameras installed at the four corners of the maintenance room. The method for material change verification is as follows: According to a preset first time period, the first image and the second image of the target area are collected; the first image is a real-time image of any area in the engineering room at present, and the second image is a standard image of any area in the engineering room where set tools, equipment, etc. are stored; in this embodiment, the first time period is every two hours, and the first image of each area in the engineering room is automatically collected once. Perform difference detection on the first image and the second image to obtain the difference value; When the difference value exceeds a preset change threshold, it is determined that the tool status of the target area has changed abnormally. Then, an abnormal change alarm message corresponding to the target area is generated, and the alarm message is presented through a dedicated computer in the data processing module. Specifically, the alarm message is used to show the administrator the types and quantities of tools missing in a certain area.

[0025] Furthermore, in order to achieve traceability of the engineering room data, the first image and the second image are recorded and saved as comparison images in the data processing module.

[0026] Furthermore, the alarm information can be compared with the tool ledger to confirm that the missing tool in the alarm information is indeed in the out-of-warehouse status in the tool ledger, thereby achieving further verification of the tool out-of-warehouse status and serving as an effective supplement to the AI ​​camera equipment monitoring of tool entry and exit.

[0027] This invention enables AI camera equipment to manage the entry and exit of tools and implements in a coarse manner, large and medium-sized parts to be managed in a refined manner according to type and quantity, and small parts to be managed in a coarse-grained manner according to toolbox format, thus realizing the batch passage of daily tools and implements.

[0028] Furthermore, to prevent the occurrence of borrowing A and returning B, the AI ​​vision-based tool inventory management method also includes return confirmation, the method of which is as follows: When the data processing module detects a critical tool or instrument being put into storage, it sends a secondary confirmation reminder to the administrator. The administrator uses an OCR terminal to scan the identification code of the critical tool or instrument. If the critical tool or instrument being put into storage matches the one being put out, the critical tool or instrument is confirmed to be put into storage. Otherwise, it is marked as borrowing A and returning B, a discrepancy reminder is issued, and the administrator stops the putting into storage.

[0029] Furthermore, the identification code is a laser-etched QR code, a laser-etched ASCII character, or a thermally transferred matte silver QR code that is etched or affixed to a tool or appliance. The identification code can effectively prevent damage from high and low temperatures, oil stains, etc., thus extending the service life of the identification code.

[0030] Furthermore, to improve the management efficiency of tools and equipment in the workshop, the management method for small hand tools with large quantities and low unit prices, such as screwdrivers, wrenches, fitter's hammers, and steel tape measures, is as follows: Small tools are placed in small tool containers such as tool trays, tool frames, or toolboxes, each of which is placed in a fixed position and is labeled with an identification code; The sensing module achieves the statistics of small tools by counting the small tool containers in the workshop; When the small tools are taken out of the warehouse, they are taken out as a whole container. The AI ​​camera device captures and records the small tool container to achieve automatic identification of the small tools when they are taken out of the warehouse. Upon entry into the warehouse, the AI ​​camera device identifies the same small tool container. The administrator then manually confirms the type and quantity of tools in the container, thus enabling the management of small tools entering and leaving the warehouse.

[0031] Furthermore, if the tools placed in the small tool container are too small or severely obscured, causing the sensing module to be unable to identify the number and type of tools and thus unable to compare them with the material requisition form, a low confidence warning will be issued, prompting the administrator to carefully check the small tool containers that have been issued this time.

[0032] The beneficial effects of the technical solution of this invention are as follows: by using AI visual recognition, target tracking and human-machine collaboration technology, it can realize non-disruptive automatic recognition, direction determination, consistency control of accounts and actual use and full-process traceability of tools and equipment entering and leaving the warehouse. It is suitable for tool and equipment management scenarios such as manned operation, use in dust and oil pollution impact conditions and priority for emergency repair. Attached Figure Description

[0033] Figure 1 This is a functional diagram of an AI vision-based tool inventory management device. Figure 2 This is a schematic diagram of the on-site deployment of an AI vision-based tool inventory management device. Figure 3 This is a flowchart illustrating the inbound and outbound management method for tools based on AI vision. Figure 4 This is a diagram illustrating the correction process after an emergency repair task skips administrator confirmation; The area is divided into several sections: 1- medium and large tool storage area; 2- small tool and spare parts shelf area; 3- AI camera equipment; 4- entrance and exit boundary area; 5- infrared camera equipment; 6- dedicated computer; 7- OCR PAD terminal. Detailed Implementation

[0034] To more clearly illustrate the technical solution of the present invention, the technical solution of the invention will be described in detail with reference to the accompanying drawings. Obviously, the following description is some typical embodiments of the present invention. For those skilled in the art, other solutions can be obtained based on these embodiments without creative effort.

[0035] This invention aims to solve the problem of orderly and consistent management of tools, spare parts and components in railway engineering workshops, avoid the risk to train operation caused by tools and equipment possibly falling to the railway site, and also avoid the problem of returning tool B after taking tool A.

[0036] The tool entry and exit management device based on AI vision described in this invention includes a perception module, a data processing module, an auxiliary module, and a system integration module. The modules are interconnected to enable information transmission and processing between them. The sensing module is used to obtain the status of the tools in the work room, including AI camera equipment set at the entrance and exit of the work room; The data processing module is used to process data from the sensing module, and to analyze and record the tool status. It includes a dedicated computer and a backup power supply. The dedicated computer is used to process and record data from the sensing module, and the backup power supply is used to provide backup power to the AI ​​vision-based tool inbound and outbound management device to prevent the risk of the AI ​​vision-based tool inbound and outbound management device failing to operate normally due to a sudden power outage. The system integration module is used for the digital and visual management of tools, including the engineering room management platform, which enables the tools to be dynamically updated and visible at any time.

[0037] In addition, the AI ​​vision-based tool entry and exit management device also includes an auxiliary module to supplement the shortcomings of the sensing module. When the sensing module cannot obtain the status of the tools in the workshop, the auxiliary module can collect or label the status of the tools. This includes an OCR terminal, a thermal printer, and a laser marking machine. The thermal printer and laser marking machine can generate QR codes or character identification codes to identify tools, equipment, or spare parts. The OCR terminal is used to read the identification codes on the tools to monitor the tools.

[0038] In addition, the sensing module also includes infrared camera devices installed at the four corners of the work room to monitor whether tools or equipment in various areas of the work room are in place. By comparing real-time images of the storage areas of various tools and equipment in the work room with standard images, and then comparing the status of each tool and equipment with the work room management platform, the status of the tools and equipment is verified, ensuring that the status of the tools and equipment is always up-to-date and correct, which facilitates the correct generation of subsequent work orders.

[0039] A tool inbound / outbound management method based on AI vision is proposed. It primarily employs AI dual-view visual recognition, supplemented by OCR laser etching of ASCII characters, laser etching of QR codes, or thermal transfer of matte silver QR codes. This method utilizes a combination of warehouse administrator and human-machine collaborative confirmation to achieve the management objectives of non-intrusive tool inbound / outbound operations, dual-view coverage, manned confirmation, and human-machine collaboration. Furthermore, it achieves synchronization of tool logistics and information flow, including... Initialization: Initialize the virtual entry / exit boundary zone; the virtual entry / exit boundary zone is the coordinate position corresponding to the actual location of the engineering room entrance / exit; Image acquisition: AI camera equipment is used to acquire images of tools or equipment entering and exiting the maintenance room; Determining whether a tool is leaving or entering the warehouse: Based on whether the tool cart is getting closer to or farther away from the virtual entry / exit boundary area as it passes through the tool cart, the system determines whether the tool is leaving or entering the warehouse. Tool identification: The data processing module identifies the type and quantity of tools by analyzing images of tools entering and leaving the database collected by the perception module. Administrator Confirmation: During normal maintenance and repair tasks, the administrator confirms the tools issued on a dedicated computer before the tools are issued or stored. During emergency repair tasks, the administrator can skip confirmation and directly issue tools and / or equipment, spare parts, etc. The operator must submit a supplementary requisition form within 24 hours, and the administrator will conduct supplementary confirmation. If the deadline is exceeded, an assessment will be conducted. Update the ledger: Update the tool ledger in the engineering room management platform to achieve dynamic management of tools, and compare the material requisition forms in the work order with the tool ledger to determine the accuracy of the latest tool status.

[0040] This invention achieves automated and efficient management of tool entry and exit without significantly increasing the workload of operators and administrators or management costs, thereby reducing the risks of loss, misplacement, and misuse of tools, equipment, and other items.

[0041] One implementation method, such as Figure 2As shown, the sensing module includes two AI cameras and four infrared cameras. The two AI cameras are positioned inside the workroom entrance with their lenses facing inwards to capture images of tools entering and leaving the workroom. By placing two AI cameras inside the workroom entrance with overlapping coverage, information on all tools entering and leaving the workroom is ensured to be captured. The AI ​​cameras are approximately 3 meters above the workroom floor. They have a 1080P resolution, wide-angle distortion-free design, a frame rate of at least 30fps, and infrared night vision capabilities. They are responsible for identifying the type, quantity, and direction of tools entering and leaving the workroom in real time and automatically capturing images as proof of entry and exit. The four infrared cameras are positioned at the four opposite corners of the workroom. They have ultra-wide-angle lenses and 24-hour recording capabilities. The data collected by the four infrared cameras enables panoramic coverage of the workroom, comparison of material changes, and post-event traceability. Specifically, to ensure that tools and equipment entering and leaving the maintenance room and stored there can be detected and monitored by the sensing module as much as possible, a simple placement standard is followed when handling and storing tools. This means that tools and equipment are not severely stacked, not piled high, and not completely obscured. Tool carts, such as flatbed carts, pallet jacks, and forklifts, are used as much as possible to facilitate compliance with the recommended placement standard. For large and medium-sized tools and equipment, single-layer placement is the main method during handling, with the marked parts facing outwards and features visible. This allows for identification of tools or equipment not only through markings but also through key features, enabling fine-grained management of large and medium-sized tools and / or equipment. For small tools, such as screwdrivers, wrenches, and pliers, they can be placed together in toolboxes, tool trays, or tool frames. It is not necessary to expose the features or markings of each tool individually. Small tools can be roughly managed by identifying the toolboxes, tool trays, and tool frames. Through this hierarchical management approach, efficient management is achieved by focusing on large tools and letting go of smaller ones.

[0042] In this embodiment, the data processing module includes a dedicated computer equipped with GPU acceleration, configured with an i5 or higher processor, at least 16GB of memory, at least 2TB of hard drive space, and a Windows operating system. This dedicated computer is located at the work station of the maintenance manager and is used to process data from the sensing module, including AI inference calculations, ledger management, data storage, and anomaly alerts. Specifically, for example, it performs inference calculations on images of tools entering and leaving the warehouse captured by the AI ​​camera device. If the previous frame of the tool entry / exit image is within the entry / exit boundary zone, and the next frame shows the tool leaving the boundary zone and approaching... If the tool is detected at the entrance / exit of the maintenance room, it is determined to be out of storage. If the tool is in the entry / exit boundary zone in the previous frame and moves away from the boundary zone and away from the entrance / exit of the maintenance room in the next frame, it is determined to be in storage. Alternatively, if the AI ​​camera cannot obtain tool information from the images of tools entering / exiting the maintenance room after AI inference, an anomaly warning is issued, and the administrator intervenes to manually record the relevant information of the tools entering / exiting the maintenance room and supplements the information into the tool ledger for ledger management. The dedicated computer also needs to store all the data acquired from the sensing module to achieve traceable management of tools entering / exiting the maintenance room. The data processing module also includes a backup power supply, which is an industrial UPS backup power supply. This backup power supply can provide at least 2 hours of continuous operation for the AI ​​vision-based tool entry / exit management device, ensuring no data loss even in the event of a power outage. More preferably, to ensure the effectiveness of data storage, the dedicated computer is also equipped with an extended hard drive specifically for storing nearly two months of video from the infrared camera and images collected by the AI ​​camera within the past year, thus providing assurance for tool traceability. Specifically, to improve the accuracy of the determination of whether a tool is entering or leaving the warehouse, at least three frames of images from the sensing module can be continuously analyzed as the tool passes through the entry / exit boundary zone to determine whether the tool is near or far from the maintenance room entrance / exit. If an empty tool cart passes through the entry / exit boundary zone, it is automatically filtered out, and the captured images are neither saved nor analyzed.

[0043] In this embodiment, the auxiliary module includes an OCR terminal, a thermal printer, and a laser marking machine. The thermal printer and laser marking machine can generate QR codes or character codes to identify tools, appliances, or spare parts, thereby enabling tool labeling for easy recording. When the AI ​​camera and infrared camera of the sensing module cannot obtain images, are inconvenient to obtain, or are too costly to obtain, the auxiliary module labels the tools to achieve tool identification and traceability. The OCR terminal is an OCR PAD terminal, which allows operators to easily label tools or spare parts while moving around the workshop. The OCR terminal can perform both OCR character recognition and QR code scanning, facilitating the identification of characters and QR codes on tools or spare parts. The thermal printer is a portable thermal printer that can print matching thermal self-adhesive labels for affixing to tools, shelves, or storage locations, serving as tool labels, shelf labels, storage location labels, and toolbox labels, thus facilitating the identification of tools and shelves. For tools where affixing labels is inconvenient, the laser marking machine can be used. The laser marking machine is used for marking unmarked metal tools, especially suitable for permanent marking of high-value, large and medium-sized equipment and critical tools. The laser marking machine can etch QR codes or ASCII characters onto metal tools or equipment at a depth of 20-50 micrometers. The etched QR codes or ASCII characters have advantages such as wear resistance, oil resistance, high and low temperature resistance, and long service life. In the auxiliary module, by using an OCR terminal to scan or read the etched QR code and ASCII characters or thermal label QR code, information about the corresponding tool can be obtained, thus effectively filling the gap when the perception module cannot obtain it. This enables the AI ​​vision-based tool entry and exit management device of the present invention to comprehensively cover all equipment and tools in the workshop.

[0044] In this embodiment, the maintenance room management platform in the system integration module is set on the dedicated computer to manage the tools, equipment, and spare parts in the maintenance room. The work order platform is set on the host computer, whereby the manager of a certain section assigns tasks to the workers to repair or maintain a certain section of the road, realizing remote task assignment. The dedicated computer is equipped with an external network and an internal network. When the external network is disconnected, it does not affect the normal operation of the AI ​​vision-based tool in / out management device.

[0045] Another implementation method, the AI ​​vision-based tool inbound / outbound management method, primarily employs AI dual-view visual recognition, supplemented by OCR laser etching of ASCII characters, laser etching of QR codes, or thermal transfer of matte silver QR codes. This method, involving warehouse administrators and human-machine collaboration for confirmation, achieves the management objectives of non-intrusive tool inbound / outbound operations, dual-view coverage, manned confirmation, and human-machine collaboration. It also achieves synchronization of tool logistics and information flow, including... Initialization: Initialize the virtual entry / exit boundary zone; the virtual entry / exit boundary zone is the coordinate position corresponding to the actual location of the work room entrance / exit; specifically, in an open area 1m-2m inside the work room entrance / exit, the position on the route that the tool vehicle must take is selected and defined as the entry / exit boundary zone. The coordinate position of the entry / exit boundary zone relative to the entrance / exit is entered into the data processing module as the virtual entry / exit boundary zone for determining the entry and exit of tools. Image acquisition: AI camera equipment is used to acquire images of tools or equipment entering and leaving the maintenance room, and infrared camera equipment is used to periodically acquire images of various areas in the maintenance room; that is, the perception module is used to monitor the dynamic status of tools in the maintenance room and feed back the dynamic status of tools in the form of images to the data processing module in real time as a real-time record of the status of the maintenance room. Determining whether a tool is leaving or entering the warehouse: Based on whether the tool cart is getting closer to or farther away from the entrance / exit of the maintenance room as it passes through the virtual entry / exit boundary zone, the system continuously collects images of the tool cart from the AI ​​camera device to determine whether the tool is leaving or entering the warehouse. Identification tools: The data processing module analyzes images of tools entering and leaving the warehouse collected by the sensing module to identify the type and quantity of tools. When identifying large and medium-sized tools, the identification is based not only on the key features of the tools' appearance but also on the identification codes set on the key tools. Therefore, the identification codes of the tools should not be obscured when handling large tools to avoid invalid identification or the problem of borrowing A and returning B. Administrator Confirmation: During normal maintenance and repair tasks, the administrator confirms the tools issued on a dedicated computer before the tools are issued or stored. During emergency repair tasks, the administrator can skip confirmation and directly issue tools and / or equipment, spare parts, etc. The operator must submit a supplementary requisition form within 24 hours, and the administrator will conduct supplementary confirmation. If the deadline is exceeded, an assessment will be conducted. Update the ledger: Update the tool ledger in the engineering room management platform to achieve dynamic management of tools, and compare the material requisition forms in the work order with the tool ledger to determine the accuracy of the latest tool status.

[0046] As a supplement to this embodiment, the specific process for determining entry and exit from the warehouse is as follows: Track moving targets and acquire images of tools on the vehicle according to the set frame interval; The tool vehicle shall continuously acquire images of crossing the virtual entry and exit boundary zone at least three times; To determine the stability and validity of consecutive frame actions, if the same tool or instrument can be captured in the images of consecutive frames, the consecutive frame actions are considered stable. If the content recorded in the images is inconsistent, it is considered invalid, and the process returns to the previous step to continue capturing images. If it is determined to be stable, the tool cart passes safely, and it is considered a valid outbound or valid inbound operation. For tool entry / exit determination, if the distance between the image of the next frame and the entrance / exit of the maintenance room is greater than that of the previous frame, it is determined as tool entry; if the distance between the image of the next frame and the entrance / exit of the maintenance room is less than that of the previous frame, it is determined as tool exit.

[0047] As another supplement to this embodiment, the specific process of administrator confirmation includes, After the inbound / outbound determination is completed, the work order is matched. If the work order is for routine maintenance tasks, then the normal outbound process will be executed. If the work order is an emergency repair task, an emergency outbound process is triggered. After the sensing module records the outbound information, it skips the administrator's confirmation and modification of control points, directly issues the outbound document, and issues a supplementary order prompt. This achieves a smooth emergency repair outbound process with only warnings and no interception. After the emergency repair tools are issued, the administrator does not fill in a material requisition form based on the work order. Specifically, the supplementary order prompt stipulates that the administrator must complete the supplementary order for material requisition within 24 hours to avoid long delays that cause the tool ledger status to be updated in a timely manner, resulting in the inability to make reasonable selections of tools in new task work orders. If the tools and equipment issued from the warehouse are consistent with the tools and equipment on the material requisition form in the work order, the tool ledger will be cleared on the engineering management platform; otherwise, an exception will be raised and the information will be sent to the material requisitioning personnel.

[0048] As another supplement to this embodiment, the AI ​​vision-based tool inbound and outbound management method further includes material change verification, which is achieved through infrared camera devices installed at the four corners of the workroom. The method for material change verification is as follows: According to a preset first time period, the first image and the second image of the target area are collected; the first image is a real-time image of any area in the engineering room at present, and the second image is a standard image of any area in the engineering room where set tools, equipment, etc. are stored; in this embodiment, the first time period is every two hours, and the first image of each area in the engineering room is automatically collected once. Perform difference detection on the first image and the second image to obtain the difference value; When the difference value exceeds a preset change threshold, it is determined that the tool status of the target area has changed abnormally. Then, an abnormal change alarm message corresponding to the target area is generated, and the alarm message is presented through a dedicated computer in the data processing module. Specifically, the alarm message is used to show the administrator the types and quantities of tools missing in a certain area.

[0049] As a further supplement to this embodiment, in order to achieve traceability of the engineering room data, the first image and the second image are recorded and saved as comparison images in the data processing module.

[0050] As a further supplement to this embodiment, the alarm information can be compared with the tool ledger to confirm that the missing tool in the alarm information is indeed in the out-of-warehouse state in the tool ledger, thereby achieving further verification of the tool out-of-warehouse status, serving as an effective supplement to the AI ​​camera equipment monitoring of tool entry and exit.

[0051] This invention enables AI camera equipment to manage the entry and exit of tools and implements in a coarse manner, large and medium-sized parts to be managed in a refined manner according to type and quantity, and small parts to be managed in a coarse-grained manner according to toolbox format, thus realizing the batch passage of daily tools and implements.

[0052] As a supplement to this embodiment, to prevent the occurrence of borrowing A and returning B, the AI ​​vision-based tool inbound / outbound management method further includes return confirmation, the method of which is as follows: When the data processing module detects a critical tool or instrument being put into storage, it sends a secondary confirmation reminder to the administrator. The administrator uses an OCR terminal to scan the identification code of the critical tool or instrument. If the critical tool or instrument being put into storage matches the one being put out, the critical tool or instrument is confirmed to be put into storage. Otherwise, it is marked as borrowing A and returning B, a discrepancy reminder is issued, and the administrator stops the putting into storage.

[0053] As a further supplement to this embodiment, the identification code is a laser-etched QR code, a laser-etched ASCII character, a thermally transferred matte silver QR code, etc., which can be etched or pasted on tools or appliances. The identification code can effectively prevent high and low temperatures, oil stains, etc., and improve the service life of the identification code.

[0054] As a supplement to this implementation method, in order to improve the management efficiency of tools and equipment in the workshop, the management method for small hand tools such as screwdrivers, wrenches, fitter's hammers, and steel tape measures, which are numerous and have low unit prices, is as follows: Small tools are placed in small tool containers such as tool trays, tool frames, or toolboxes, each of which is placed in a fixed position and is labeled with an identification code; The sensing module achieves the statistics of small tools by counting the small tool containers in the workshop; When the small tools are taken out of the warehouse, they are taken out as a whole container. The AI ​​camera device captures and records the small tool container to achieve automatic identification of the small tools when they are taken out of the warehouse. Upon entry into the warehouse, the AI ​​camera device identifies the same small tool container. The administrator then manually confirms the type and quantity of tools in the container, thus enabling the management of small tools entering and leaving the warehouse.

[0055] As a further supplement to this embodiment, if the tools placed in the small tool container are too small or severely obscured, causing the sensing module to be unable to identify the number and type of tools and thus unable to compare them with the material requisition form, a low confidence warning will be issued, prompting the administrator to carefully check the small tool containers that have been issued this time.

[0056] The implementation of this invention in railway maintenance workshops has significantly improved the consistency of tool management, mainly in the following ways: 1) Achieve seamless and convenient operation and improve work efficiency: Adopting a non-intrusive warehouse design, workers do not need to perform additional operations such as swiping cards or scanning codes. They only need to stay in the designated area for a short time (2-3 seconds), and the system can automatically complete the identification of tools and determine the direction of entry and exit, which greatly simplifies the process of picking up and returning tools.

[0057] 2) Constructing a dual-view, three-dimensional monitoring system to enhance material traceability: By combining an AI camera at the warehouse door with a high-definition infrared night vision camera at the warehouse's diagonal angles, dual-view full coverage is achieved. This design effectively reduces blind spots and obstructions, enabling precise capture and tracking of dynamic changes in materials within the warehouse.

[0058] 3) Implement closed-loop management of inbound and outbound inventory to ensure consistency between records and actual inventory: Establish a strict closed-loop management mechanism for inbound and outbound inventory. For large and medium-sized key tools and equipment, focus on controlling their issuance, return, and verification processes; for small tools, implement quantity-based control. This effectively eliminates management loopholes such as unregistered outbound shipments and incomplete returns, and solves the problem of discrepancies between records and actual inventory caused by loss or consumption not due to theft. The system supports automatic ledger creation and automatic comparison, and provides real-time alerts when anomalies are detected.

[0059] 4) Balancing AI and human verification to ensure business continuity: A rapid verification process by warehouse administrators is introduced on top of AI-powered automatic recognition. This "AI recognition + human verification" model ensures the accuracy of the recognition results while also prioritizing emergency scenarios such as on-site repairs, preventing business disruptions caused by system misjudgments.

[0060] 5) Adaptable to harsh industrial environments and highly scalable: The system does not rely on easily interfered technologies such as RFID, and uses industrial-grade hardware, enabling it to stably adapt to complex industrial environments such as dust, oil, and vibration. In terms of management strategy, it primarily employs "category-level management" to achieve broad coverage of all types of tools and equipment; at the same time, it reserves upgrade space, allowing high-value small tools and equipment to be upgraded to individual-level refined management as needed in the future, balancing current practicality with future scalability.

[0061] The above embodiments are merely descriptions of a typical application of the technical solution of the present invention. Reasonable extensions can be made without requiring creative effort.

Claims

1. A tool inbound / outbound management device based on AI vision, characterized in that, It includes a sensing module, a data processing module, and a system integration module. The modules are interconnected to enable information transmission and processing. The sensing module is used to obtain the status of the tools in the work room, including AI camera equipment set at the entrance and exit of the work room to monitor the tools entering and leaving the work room; The data processing module is used to process data from the sensing module to analyze and record the tool status. It includes a dedicated computer and a backup power supply. The dedicated computer is used to process and record data from the sensing module, and the backup power supply is used to provide backup power for the AI ​​vision-based tool in-and-out management device. The system integration module is used for the digital and visual management of tools, including the engineering room management platform, which enables the tools to be dynamically updated and visible at any time.

2. The tool in / out management device based on AI vision as described in claim 1, characterized in that, The sensing module also includes infrared camera devices installed at the four corners of the work room to monitor whether tools or equipment are in place in various areas of the work room.

3. The tool in / out management device based on AI vision as described in claim 1, characterized in that, It also includes a virtual entry / exit boundary zone, which is used as a reference location for determining whether a tool is leaving or entering the warehouse.

4. The tool in / out management device based on AI vision as described in claim 3, characterized in that, It also includes auxiliary modules, The auxiliary module is used to supplement the shortcomings of the sensing module. When the sensing module cannot obtain the status of the tools in the workshop, the auxiliary module can collect or mark the status of the tools. It includes an OCR terminal, a thermal printer and a laser marking machine. The thermal printer and laser marking machine can generate identification codes in the form of QR codes or characters to identify tools, utensils or spare parts. The OCR terminal is used to read the identification codes on the tools to realize the monitoring of tools and utensils.

5. A tool inventory management method based on AI vision, based on the tool inventory management device based on AI vision as described in claim 4, characterized in that, include, Initialization: Initialize the virtual entry / exit boundary zone; the virtual entry / exit boundary zone is the coordinate position corresponding to the actual location of the engineering room entrance / exit; Image acquisition: AI camera equipment is used to acquire images of tools or equipment entering and exiting the maintenance room, and infrared camera equipment is used to periodically acquire images of various areas within the maintenance room; Determining whether a tool is leaving or entering the warehouse: Based on whether the tool cart is getting closer to or farther away from the entrance / exit of the maintenance room as it passes through the virtual entry / exit boundary zone, the system continuously collects images of the tool cart from the AI ​​camera device to determine whether the tool is leaving or entering the warehouse. Tool identification: The data processing module identifies the type and quantity of tools by analyzing images of tools entering and leaving the database collected by the perception module. Administrator confirmation: During normal maintenance and repair tasks, the administrator confirms the requisitioned tools on a dedicated computer before the tools are issued or stored; during emergency repair tasks, the administrator confirmation is skipped and the tools and / or equipment and / or spare parts are issued directly. Update the ledger: Update the tool ledger in the engineering room management platform to achieve dynamic management of tools.

6. The vision-based tool inventory management method as described in claim 5, characterized in that, The specific process for determining entry and exit from the warehouse is as follows: Track moving targets and acquire images of tools on the vehicle according to the set frame interval; The tool vehicle shall continuously acquire images of crossing the virtual entry and exit boundary zone at least three times; To determine the stability and validity of consecutive frame actions, if the same tool or instrument can be captured in the images of consecutive frames, the consecutive frame actions are considered stable. If the content recorded in the images is inconsistent, it is considered invalid, and the process returns to the previous step to continue capturing images. If it is determined to be stable, the tool cart passes safely, and it is considered a valid outbound or valid inbound operation. For tool entry / exit determination, if the distance between the image of the next frame and the entrance / exit of the maintenance room is greater than that of the previous frame, it is determined as tool entry; if the distance between the image of the next frame and the entrance / exit of the maintenance room is less than that of the previous frame, it is determined as tool exit.

7. The vision-based tool inventory management method as described in claim 5, characterized in that, The specific process for administrator confirmation includes: After the inbound / outbound determination is completed, the work order is matched. If the work order is for routine maintenance tasks, then the normal outbound process will be executed. If the work order is an emergency repair task, the emergency outbound process is triggered. After the sensing module records the outbound information, it skips the administrator's confirmation and modification of the control point, directly issues the outbound order, and issues a supplementary order prompt. Achieve a smooth emergency repair outbound process with only early warning and no interception; after emergency repair tools are issued, the administrator does not fill out a material requisition form based on the work order; If the tools and equipment issued from the warehouse are consistent with the tools and equipment on the material requisition form in the work order, the tool ledger will be cleared on the engineering management platform; otherwise, an exception will be raised and the information will be sent to the material requisitioning personnel.

8. The vision-based tool inventory management method as described in claim 5, characterized in that, It also includes material change verification, which is achieved through infrared camera equipment installed at the four corners of the maintenance room. The method for material change verification is as follows: According to a preset first time period, a first image and a second image of the target area are collected; the first image is a real-time image of any area of ​​the engineering room at present, and the second image is a standard image of any area of ​​the engineering room where the set tools and / or equipment are stored. Perform difference detection on the first image and the second image to obtain the difference value; When the difference value exceeds a preset change threshold, it is determined that the tool status of the target area has changed abnormally. Then, an abnormal change alarm message corresponding to the target area is generated, and the alarm message is presented through a dedicated computer in the data processing module.

9. The vision-based tool inventory management method as described in claim 5, characterized in that, It also includes a return confirmation, the method of which is as follows: When the data processing module detects a critical tool or instrument being put into storage, it sends a secondary confirmation reminder to the administrator. The administrator uses an OCR terminal to scan the identification code of the critical tool or instrument. If the critical tool or instrument being put into storage matches the one being put out, the critical tool or instrument is confirmed to be put into storage. Otherwise, it is marked as borrowing A and returning B, a discrepancy reminder is issued, and the administrator stops the putting into storage.

10. The vision-based tool inventory management method as described in claim 5, characterized in that, Small tool inbound and outbound management methods Small tools are placed in small tool containers such as tool trays, tool frames, or toolboxes, each of which is placed in a fixed position and is labeled with an identification code; The sensing module achieves the statistics of small tools by counting the small tool containers in the workshop; When the small tools are taken out of the warehouse, they are taken out as a whole container. The AI ​​camera device captures and records the small tool container to achieve automatic identification of the small tools when they are taken out of the warehouse. Upon entry into the warehouse, the AI ​​camera device identifies the same small tool container. The administrator then manually confirms the type and quantity of tools in the container, thus enabling the management of small tools entering and leaving the warehouse.

Images