A control system for an automated warehouse combined with augmented reality technology

Abstract

This invention discloses a control system for an automated warehouse combining augmented reality (AR) technology, comprising a front-end display layer, an application layer, a data layer, a network layer, and an equipment layer. The front-end display layer includes an MR client and a PC, used to present a virtual reality industrial scene through 3D visualization technology, and providing human-computer interaction for operations such as goods picking, inventory management, inspection, intelligent operation and maintenance, and remote maintenance / guidance. The application layer establishes a static 3D model of the virtual industrial scene, collects and monitors real-time data feedback, and performs 3D dynamic simulation of the scene / equipment. The data layer uses a database to store basic equipment data, documents, personnel information, and audio / video data. The network layer establishes network connections using multiple communication networks. The equipment layer consists of dedicated equipment actually deployed in the industrial site. This invention utilizes MR technology to achieve innovative applications of real-time, interactive remote support and training, overlaying virtual objects and information onto the real world to help users obtain remote guidance from experts during actual operations, improving the efficiency and accuracy of industrial operations.

Description

Technical Field

[0001] This invention belongs to the field of virtual reality, specifically a control system for a three-dimensional warehouse combined with augmented reality technology. Background Technology

[0002] With the development of AI technology, intelligent logistics MR systems are becoming increasingly integrated with it. By combining machine learning and image recognition functions, the system can optimize algorithms based on data accumulation to improve decision-making efficiency, such as analyzing transportation data to predict logistics demand and allocate resources in advance. However, there are several drawbacks in hardware: 1) Poor wearing comfort: Wearing an MR headset for extended periods may cause user fatigue, discomfort, or even dizziness. Factors such as the weight, size, and heat dissipation of the device can affect the wearing experience, thereby impacting the work efficiency and willingness of staff to use it in logistics operations. 2) Performance needs improvement: Some complex logistics scenarios place high demands on the computing power and graphics processing capabilities of MR devices. Some existing devices may not be able to meet the requirements of real-time, high-precision virtual scene rendering and data processing, leading to screen stuttering, delays, and affecting the smoothness and accuracy of operations. There are also drawbacks in software and technology: 1) Accurately matching virtual information with real-world scenes in the logistics environment requires high-precision positioning and tracking technology. However, logistics scenarios are complex and varied, with conditions such as metal shelves and signal obstructions, which can affect the accuracy of positioning and tracking, causing deviations in the display position of virtual information and affecting the effectiveness of functions such as operation guidance. 2) High Data Processing and Transmission Pressure: Intelligent logistics MR systems generate and process massive amounts of data, such as cargo information and real-time video streams. Data transmission may experience delays and packet loss, affecting the system's real-time performance and stability. Simultaneously, data analysis and processing require powerful algorithms and computing resources; current technology may have limitations in handling complex data and enabling rapid decision-making. 3) System Compatibility and Integration Challenges: Logistics companies typically use various information systems and equipment, requiring MR systems to integrate with warehouse management systems, transportation scheduling systems, etc. However, incompatibility in interfaces and inconsistent data formats between different systems increases the difficulty and cost of system integration.

[0003] With current warehouse management technologies, intelligent picking, where employees wearing smart glasses receive real-time digital information and follow system guidance to find goods and optimal routes, can improve picking speed and accuracy. It also enables real-time inventory tracking, allowing managers to promptly grasp inventory information and avoid technical problems such as overstocking or stockouts. Regarding customer experience enhancement, if businesses can provide customized logistics solutions through MR technology, customers can track order progress and obtain delivery information in real time, improving interactive experiences and satisfaction. This ultimately leads to highly efficient and interactive intelligent logistics. Summary of the Invention

[0004] The purpose of this invention is to leverage MR technology to achieve innovative applications of real-time, interactive remote support and training, based on the characteristics of the enterprise's industry and its own production processes. Virtual objects and information are overlaid on the real world, helping users obtain remote guidance from experts during actual operations. This improves remote equipment guidance and maintenance, enhancing the efficiency and accuracy of industrial operations.

[0005] The technical solution adopted by the present invention to achieve the above objectives is as follows:

[0006] A control system for an automated warehouse that combines augmented reality technology includes a front-end display layer, an application layer, a data layer, a network layer, and a device layer.

[0007] The front-end presentation layer, including the MR client and PC, is used to present virtual reality industrial scene scenarios through 3D visualization technology, and provide human-computer interaction for virtual and real combined operations such as goods picking, goods inventory, inspection, intelligent operation and maintenance, and remote maintenance / guidance. Among them, the MR client is for operators in the industrial field, and the PC is for technical experts in remote control rooms or even further away.

[0008] The application layer is used to build static 3D models of virtual industrial sites, collect and monitor feedback data in real time, update data in real time according to human-computer interaction instructions from the front-end display layer, and perform 3D dynamic simulation of scenes / equipment.

[0009] The data layer uses a database to store basic data, documents, personnel information, and audio / video data.

[0010] The network layer uses multiple communication networks to establish network connections between the MR client, PC, and server;

[0011] The equipment layer consists of proprietary equipment actually deployed in the industrial field.

[0012] The MR client displays a virtual reality interface to the on-site operator, receives operator commands and sends them to the application layer for parsing, and displays order attribute data parsed by the application layer; it projects wiring diagrams, displays exploded views of the stacker crane, and displays drawing information; in the remote maintenance / guidance function, the MR client is also used by the operator to mark on the virtual interface using a drawing and annotation tool, and to record actual on-site video and audio and send them to the application layer or PC.

[0013] The operator instructions include at least one of the following: orders and equipment QR codes in goods picking, goods inventory, inspection, intelligent operation and maintenance, and remote maintenance / guidance; and specific operation point selection instructions related to source location and box number.

[0014] The order attribute data includes at least one of the following: map, actual physical location of the device, item name, quantity, appearance image, and inventory surplus or shortage information.

[0015] The PC terminal is intended for technical experts in remote control rooms or even further away, enabling video communication with MR devices and achieving synchronous video communication; it also receives wiring diagrams or design documents sent by the MR client, as well as annotations, screenshots, and projected video and audio.

[0016] The application layer includes the following functional modules: goods picking module, goods inventory module, inspection module, intelligent operation and maintenance module, and remote maintenance / guidance module;

[0017] The goods picking module, goods inventory module, and inspection module implement specific functions through the document management unit, QR code scanning unit, and server upload unit.

[0018] The intelligent operation and maintenance module includes an equipment information unit, an equipment QR code unit, and an equipment drawing unit. The equipment information unit stores descriptions of all operating equipment in the industrial site. The equipment drawing unit stores wiring diagrams for the equipment, and exploded views of stacker cranes. The equipment QR code unit combines the descriptions of the operating equipment with the exploded views of the wiring diagrams to create a QR code image, which is then provided to the MR client. When the MR client scans the intelligent operation and maintenance QR code in the industrial site, it displays the equipment descriptions and drawings.

[0019] The remote maintenance / guidance module includes a communication unit, a video and audio unit, and a remote annotation unit. The communication unit uses WebRTC to establish a communication connection between the MR client, the PC client, and the server. The video and audio unit uses audio and video encoding and decoding technology to analyze the audio and video recorded by the MR client on-site. The remote annotation unit provides editing tools to the MR client for annotation.

[0020] The document management unit is used by developers to pre-create documents with different functions according to the process flow of the industrial site, and fill in the specific attribute data of different functional documents row by row and column by column in the documents;

[0021] The QR code scanning unit is used to bind QR codes to the goods picking orders, goods inventory orders, and inspection orders generated by the document management unit, and provide them to the MR terminal of the operator in the industrial field in the form of QR code images; and to receive QR code scanning instructions from the MR client and parse them into goods picking order, goods inventory order, and inspection order information and provide them to the MR client.

[0022] Furthermore, the different functional documents mentioned above include goods picking orders, goods inventory orders, and inspection orders;

[0023] Furthermore, the specific attribute data includes customized attribute elements for operators in pre-made goods picking orders, goods inventory orders, and inspection orders. Operators use these customized attribute elements to add, delete, modify, and query goods picking orders, goods inventory orders, and inspection orders in order to generate new picking orders, goods inventory orders, and inspection orders.

[0024] The upload server unit is used to upload various functional documents generated by the document management unit, as well as QR code images generated by the QR code scanning unit, to the server database for storage.

[0025] The various communication networks mentioned are one or more of the following: public network, private network, wired network, wireless network, 5G network, and Internet.

[0026] The proprietary equipment in the equipment layer includes at least one of stacker cranes, conveyor lines, racks, and extended equipment.

[0027] The present invention has the following beneficial effects and advantages:

[0028] By simulating material picking and inventory management scenarios, operators can experience these processes firsthand, improving their skill mastery. Currently, actual collaboration requires engineers to be in the same space and time. MR technology allows designers and engineers in different locations to collaborate and communicate in real time, improving work efficiency. This technology breaks down geographical limitations, making team collaboration more flexible and efficient, enabling rapid location tracking, which reduces maintenance costs and increases efficiency. It also reduces communication costs and time delays. This makes project management and decision-making processes faster and more accurate. Attached Figure Description

[0029] Figure 1 This is a diagram illustrating the software architecture and relationships of the present invention.

[0030] Figure 2 This is a schematic diagram of the hardware installation and connection logic of the present invention;

[0031] Figure 3 System architecture design diagram;

[0032] Figure 4 A schematic diagram illustrating the establishment of WebRTC communication;

[0033] Figure 5 This is the order picking interface;

[0034] Figure 6 This is a QR code scanning interface;

[0035] Figure 7 Map for picking goods;

[0036] Figure 8 This is the interface for selecting goods information;

[0037] Figure 9 This is the order check interface;

[0038] Figure 10 Map for inventorying goods;

[0039] Figure 11 This is the interface for inventory information;

[0040] Figure 12 This is the inspection order interface;

[0041] Figure 13 Map for inspection equipment;

[0042] Figure 14 The interface for taking photos of the inspection equipment;

[0043] Figure 15 This is the exploded view after scanning;

[0044] Figure 16 This is a demonstration diagram of the PC remote annotation function. Detailed Implementation

[0045] To make the above-mentioned objects, features, and advantages of the present invention more apparent and understandable, the specific embodiments of the present invention will be described in detail below with reference to the accompanying drawings. Many specific details are set forth in the following description to provide a thorough understanding of the present invention. However, the present invention can be implemented in many other ways different from those described herein, and those skilled in the art can make similar modifications without departing from the spirit of the invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.

[0046] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. The terminology used in the description of the invention herein is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The invention will now be described in further detail with reference to the accompanying drawings and embodiments.

[0047] This invention, based on the characteristics of the enterprise's industry and its own production processes, focuses on researching key technologies and systems related to Mixed Reality (MR). It aims to overcome challenges in cargo picking, inventory management, inspection, intelligent operation and maintenance, and remote maintenance / guidance, enabling demonstration applications in scenarios such as remote assistance and guidance, picking, inventory management, and inspection in automated warehouses. Utilizing MR technology, it achieves innovative applications of real-time, interactive remote support and training, overlaying virtual objects and information onto the real world to help users receive remote expert guidance during actual operations. This improves remote equipment guidance and maintenance, enhancing the efficiency and accuracy of industrial operations.

[0048] The software system of the present invention includes: a front-end presentation layer, an application layer, a data layer, a network layer, and a device layer.

[0049] 1) The front-end presentation layer, including the MR (Mixed Reality) and PC terminals, is used to present virtual reality industrial scene scenarios through 3D visualization technology. It provides human-computer interaction functions that combine virtual and real-world elements, such as goods picking, inventory management, inspection, intelligent operation and maintenance, and remote maintenance / guidance. The MR terminal is for operators in the industrial field, while the PC terminal is for technical experts in remote control rooms or even further afield. The 3D visualization process includes: creating or using open-source databases to select on-site components, rendering them, and obtaining the virtual industrial scene.

[0050] Specifically, the MR client displays a virtual reality interface to on-site operators, receives operator instructions (including orders and equipment QR codes for goods picking, inventory counting, inspection, intelligent operation and maintenance, and remote maintenance / guidance, as well as specific operation selection instructions related to source location, box number, etc.), sends them to the application layer for parsing, and displays the order attribute data parsed by the application layer, such as: maps, actual physical locations of equipment, item names, quantities, appearance images, inventory surpluses, and inventory shortages; it also projects wiring diagrams, exploded views of stacker cranes, and other drawing information. In the remote maintenance / guidance function, the MR client's virtual interface also provides on-site operators with annotation tools for their use, and can record actual on-site video and audio and send them to the application layer or PC.

[0051] Specifically, the PC client is for technical experts in remote control rooms or even further away to conduct video communication with MR devices and achieve synchronous video communication; it can also receive wiring diagrams or design documents, as well as annotations, screenshots, and projected video and audio from MR clients.

[0052] 2) Application layer: Used to build static 3D models of virtual industrial sites, collect and monitor real-time data, update data in real-time according to human-machine interaction commands, and perform 3D dynamic simulation of scenes / equipment. It mainly includes the following functional modules: goods picking module, goods inventory module, inspection module, intelligent operation and maintenance module, and remote maintenance / guidance module.

[0053] Specifically, the goods picking module, goods inventory module, and inspection module implement their functions through the document management unit, QR code scanning unit, and server upload unit.

[0054] The document management unit is used by developers to pre-create documents with different functions based on the industrial process flow, and to fill in the specific attribute data of each functional document row by row and column by column. These functional documents include: goods picking orders, goods inventory orders, and inspection orders. The specific attribute data includes customized attribute elements for operators in the pre-made goods picking orders, goods inventory orders, and inspection orders. Operators can use these customized attribute elements to add, delete, modify, and query goods picking orders, goods inventory orders, and inspection orders to generate new orders.

[0055] For example, the document management unit generates the following order:

[0056] A goods picking order is generated by combining specific attribute data (shelf map, actual physical location of the goods to be picked, source location or box number; item name, quantity, appearance image, etc.).

[0057] Inspection orders are generated by combining specific attribute data (shelf map, actual physical location of inventory goods, source location or box number; item name, quantity, appearance image; inventory surplus, inventory shortage, etc.).

[0058] Inspection orders are generated by combining specific attribute data (equipment map, actual physical location of the inspected equipment; location and appearance of the inspected equipment).

[0059] The QR code scanning unit is used to bind QR codes to the goods picking orders, goods inventory orders, and inspection orders generated by the document management unit, and provide them to the MR terminal of the operator in the industrial field in the form of QR code images; and to receive QR code scanning instructions from the MR client and parse them into goods picking order, goods inventory order, and inspection order information and provide them to the MR client.

[0060] The upload server unit is used to upload various functional documents generated by the document management unit, as well as QR code images generated by the QR code scanning unit, to the server database for storage.

[0061] Specifically, the intelligent operation and maintenance module includes an equipment information unit, an equipment QR code unit, and an equipment drawing unit. The equipment information unit stores descriptive information for all operating equipment in the industrial site; the equipment drawing unit stores wiring diagrams and exploded views of equipment such as stacker cranes; the equipment QR code unit combines the equipment's descriptive information and the exploded wiring diagrams to create a QR code image, which is then provided to the MR client. When the MR client scans the intelligent operation and maintenance QR code in the industrial site, it displays the equipment description information and drawings.

[0062] Specifically, the remote maintenance / guidance module includes a communication unit, a video and audio unit, and a remote annotation unit. The communication unit uses WebRTC to establish a communication connection between the MR client, the PC client, and the server; the video and audio unit uses audio and video encoding and decoding technology to analyze the audio and video recorded on-site by the MR client; and the remote annotation unit provides the MR client with editing tools for annotation.

[0063] 3) Data layer: Mainstream databases, WMS databases, 3D databases, and interface databases are used to store basic data, documents, personnel information, audio and video, etc.

[0064] 4) Network layer: Employs various communication networks such as public / private networks, wired / wireless / 5G, and the Internet to establish network connections between the MR client, PC client, and server.

[0065] 5) Equipment layer, which consists of stacker cranes, conveyor lines, racks, and other proprietary equipment that can be expanded and deployed on the industrial site.

[0066] The above describes the intelligent logistics MR (Mixed Reality) system of this invention. Based on the .NET platform and developed using the C# language, it employs a mature three-tier architecture, supports centralized / distributed architecture and multi-warehouse presentation, and allows for close integration with systems such as WMS and WCS through interface technologies such as WebService, intermediate tables, TCP / IP, and file exchange. It also supports multiple mainstream database platforms such as Oracle and SQL Server. The system is characterized by its advanced technology, ease of use, ease of maintenance, modularity, scalability, good compatibility, practicality, and cost-effectiveness.

[0067] Overall software architecture and relationships, such as Figure 1 The hardware and software installation and logic of the system of this invention, such as... Figure 2 First, install the server program, then install the server itself, and then install the SQL Server database; add the MRDB table (User table) to the database; configure the DB string on the server and test whether the data communicates successfully with the database; install the PC client program and configure the server's IP address and port number, open the server and PC client, and test whether the connection is successful; install the MR glasses client program and configure the server's IP address and port number, open the server and MR glasses, and test whether the connection is successful; prepare the shelving: affix pallet barcodes to the pallets; affix equipment barcodes to the conveyor lines and stacker cranes;

[0068] The system is structurally divided into client and server sides. The client is responsible for providing the interface display and video / voice interaction, while the server side is responsible for interfacing with the upper-layer system, collecting lower-layer data, and performing logical processing on the data. The overall framework design, such as... Figure 3 .

[0069] Server-side: The system connects to business servers via a proxy server, enabling communication between these systems. Decoupling techniques are employed to reduce coupling between the data model, business logic, and view display layers, minimizing dependencies and allowing for pluggable development. The proxy server uses the DoNetty framework to implement an NIO server.

[0070] Since MR glasses transmit video via WebRTC, the server-side, in addition to sending messages and connecting to the database, also includes a signaling server function. The signaling server is a crucial component of WebRTC communication, responsible for establishing and maintaining the communication channel and negotiating media between the communicating parties. The signaling server does not directly participate in the transmission of media data; instead, it is responsible for exchanging necessary metadata, such as SDP (Session Description Protocol) and ICE (Candidate Network Traversal) information. This signaling information is used to establish and manage PeerConnection connections and negotiate the transmission method and parameters of media data.

[0071] Divided into PC and MR ends, the PC end requires video communication with the MR device. WebRTC is used here to implement video communication functionality for remote maintenance and guidance. WebRTC is a real-time communication protocol widely used in audio and video calls, online conferencing, and other scenarios. This article will analyze in detail the three main server architectures of WebRTC: Mesh, MCU, and SFU, as well as their advantages, disadvantages, and applicable scenarios. We will also delve into the four key components of the WebRTC architecture: media capture device, signaling server, PeerConnection, and STUN / TURN server, explaining these abstract technical concepts through examples and vivid language, such as... Figure 4 Furthermore, the workflow operation steps of the software system of the present invention are as follows:

[0072] Goods picking function: The MR client system binds a picking list after scanning the QR code of the order, such as... Figure 5 After clicking the confirmation button on the QR code scanning interface in the virtual reality interface, you can bind the picking order, such as... Figure 6 After clicking the linked picking order button, a shelf map will appear, indicating the location information of the items to be picked, such as... Figure 7 Users can move to the actual location of the item to be picked in the picking order, scan the source location or box number, and then the virtual reality interface will display information such as the name, quantity, and appearance image of the item to be picked. After clicking confirm, the selection process ends. Figure 8 .

[0073] Inventory counting function: The MR client system binds an inventory list after scanning the QR code on the order, such as... Figure 9After clicking the confirmation button on the QR code scanning interface, you can bind the inventory order, such as... Figure 6 Clicking the linked inventory order button will display a shelf map indicating the location information of the inventory items, such as... Figure 10 Users can move to the location of the inventory items in the inventory order, scan the source location or box number, and then be displayed with the name, quantity, and appearance image of the items being inventoried, as well as information such as inventory surplus or shortage. Clicking "Confirm" ends the selection process. Figure 11 .

[0074] Inspection function: The MR client system binds the inspection order after scanning the QR code of the order, such as... Figure 12 After clicking the confirmation button on the QR code scanning interface, you can bind the inspection order, such as... Figure 6 After clicking the "Bound Inspection Order" button, a shelf map will appear, indicating the location of the equipment to be inspected, such as... Figure 13 Users can move to the location of the equipment to be inspected in the inspection order, scan the equipment number, and then see the location and appearance of the equipment to be inspected. Clicking the virtual button allows them to take a photo for later use. Clicking "Confirm" prompts for the next equipment to be inspected, continuing until the entire order is completed. Figure 14 .

[0075] Intelligent operation and maintenance functions: QR codes are affixed to equipment (electrical cabinets), and wiring diagrams are projected after scanning with MR glasses. Stacker cranes and other equipment display exploded views. These diagrams and exploded views can be zoomed in and their angles changed. Figure 15 Clicking the image below the model on the virtual interface will display the drawing information for that device.

[0076] Remote maintenance / guidance: This is divided into PC and MR terminals. The PC terminal needs to communicate with the MR device via video to achieve video communication functionality. The operator, wearing MR glasses, stands near the touchscreen, projecting wiring diagrams (or design documents), which can be circled and marked. The engineer's video and audio are projected, and the PC terminal displays the MR glasses' camera content and the operator's audio. Figure 16 After clicking the send button, you can send the image to the MR client.

[0077] This invention simulates material picking and inventory management scenarios, allowing operators to experience these processes firsthand and improving their skill mastery. It addresses the current technical limitations where actual collaboration requires engineers to work in the same space and at the same time, resulting in low efficiency and constraints.

[0078] This invention, based on the characteristics of the enterprise's industry and its own production processes, utilizes MR technology to achieve innovative applications of real-time, interactive remote support and training. It overlays virtual objects and information onto the real world, helping users obtain remote guidance from experts during actual operations. This improves remote equipment guidance and maintenance, enhancing the efficiency and accuracy of industrial operations.

[0079] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A control system for an automated warehouse combining augmented reality technology, characterized in that, It includes the front-end presentation layer, application layer, data layer, network layer, and device layer; The front-end presentation layer, including the MR client and PC, is used to present virtual reality industrial scene scenarios through 3D visualization technology, and provide human-computer interaction for virtual and real combined operations such as goods picking, goods inventory, inspection, intelligent operation and maintenance, and remote maintenance / guidance. Among them, the MR client is for operators in the industrial field, and the PC is for technical experts in remote control rooms or even further away. The application layer is used to build static 3D models of virtual industrial sites, collect and monitor feedback data in real time, update data in real time according to human-computer interaction instructions from the front-end display layer, and perform 3D dynamic simulation of scenes / equipment. The data layer uses a database to store basic data, documents, personnel information, and audio / video data. The network layer uses multiple communication networks to establish network connections between the MR client, PC, and server; The equipment layer consists of proprietary equipment actually deployed in the industrial field.

2. The control system for an automated warehouse combined with augmented reality technology according to claim 1, characterized in that, The MR client displays a virtual reality interface to the on-site operator, receives operator instructions and sends them to the application layer for parsing, and displays the order attribute data parsed by the application layer. The system projects wiring diagrams, displays exploded views of the stacker crane, and provides drawing information. In the remote maintenance / guidance function, the MR client is also used by operators to mark areas on the virtual interface using drawing and annotation tools, as well as record actual on-site video and audio and send them to the application layer or PC.

3. The control system for an automated warehouse combined with augmented reality technology according to claim 2, characterized in that, The operator instructions include at least one of the following: orders and equipment QR codes in goods picking, goods inventory, inspection, intelligent operation and maintenance, and remote maintenance / guidance; and specific operation point selection instructions related to source location and box number. The order attribute data includes at least one of the following: map, actual physical location of the device, item name, quantity, appearance image, and inventory surplus or shortage information.

4. The control system for an automated warehouse combined with augmented reality technology according to claim 1, characterized in that, The PC terminal is intended for technical experts in remote control rooms or even further away, enabling video communication with MR devices and achieving synchronous video communication; it also receives wiring diagrams or design documents sent by the MR client, as well as annotations, screenshots, and projected video and audio.

5. The control system for an automated warehouse combined with augmented reality technology according to claim 1, characterized in that, The application layer includes the following functional modules: goods picking module, goods inventory module, inspection module, intelligent operation and maintenance module, and remote maintenance / guidance module; The goods picking module, goods inventory module, and inspection module implement specific functions through the document management unit, QR code scanning unit, and server upload unit. The intelligent operation and maintenance module includes an equipment information unit, an equipment QR code unit, and an equipment drawing unit. The equipment information unit stores descriptions of all operating equipment in the industrial site. The equipment drawing unit stores wiring diagrams for the equipment, and exploded views of stacker cranes. The equipment QR code unit combines the descriptions of the operating equipment with the exploded views of the wiring diagrams to create a QR code image, which is then provided to the MR client. When the MR client scans the intelligent operation and maintenance QR code in the industrial site, it displays the equipment descriptions and drawings. The remote maintenance / guidance module includes a communication unit, a video and audio unit, and a remote annotation unit. The communication unit uses WebRTC to establish a communication connection between the MR client, the PC client, and the server. The video and audio unit uses audio and video encoding and decoding technology to analyze the audio and video recorded by the MR client on-site. The remote annotation unit provides editing tools to the MR client for annotation.

6. The control system for an automated warehouse combined with augmented reality technology according to claim 5, characterized in that, The document management unit is used by developers to pre-create documents with different functions according to the process flow of the industrial site, and fill in the specific attribute data of different functional documents row by row and column by column in the documents; The QR code scanning unit is used to bind QR codes to the goods picking orders, goods inventory orders, and inspection orders generated by the document management unit, and provide them to the MR terminal of the operator in the industrial field in the form of QR code images; and to receive QR code scanning instructions from the MR client and parse them into goods picking order, goods inventory order, and inspection order information and provide them to the MR client. The upload server unit is used to upload various functional documents generated by the document management unit, as well as QR code images generated by the QR code scanning unit, to the server database for storage.

7. The control system for an automated warehouse combined with augmented reality technology according to claim 1, characterized in that, The various communication networks mentioned are one or more of the following: public network, private network, wired network, wireless network, 5G network, and Internet.

8. The control system for an automated warehouse combined with augmented reality technology according to claim 5, characterized in that, The proprietary equipment in the equipment layer includes at least one of stacker cranes, conveyor lines, racks, and extended equipment.

Images