Intelligent storage system for automobile parts based on digital twinning

Abstract

The invention relates to an intelligent storage system for automobile parts based on digital twinning, and belongs to the field of logistics automation.

Description

Technical Field

[0001] The present invention belongs to the field of logistics automation and relates to an intelligent warehousing system for automotive parts based on digital twin. Background Art

[0002] The currently adopted warehousing management system usually presents the information value of data resources to users in the form of two-dimensional graphs and tables. The information represented is limited and rather abstract, and the degree of interaction and integration between the physical world and the information world is relatively low. There are problems such as scattered pages, unintuitive information, small amount of information, and inconvenient operation. This leads to inaccuracies in inventory management, causing a large amount of funds to be tied up in unnecessary inventory, or resulting in shortages of parts and delayed order deliveries, affecting customer satisfaction. In addition, traditional logistics processes may also have problems such as unnecessary waiting times and non-optimized transportation routes, increasing logistics costs and operation cycles.

[0003] To address the above problems, constructing a digital twin model can present information such as the warehouse environment, inventory situation, and equipment status in a visual form, enabling warehousing managers to intuitively understand and monitor warehouse operations. The digital twin system can connect to real-time data sources, synchronize and update data with the actual warehouse, and achieve real-time monitoring of the warehouse status, inventory changes, and equipment operation conditions. It can apply intelligent algorithms and optimization models to automatically schedule and optimize the logistics and operation processes of the warehouse, providing refined warehousing management, including aspects such as storage location management, parts traceability, and quality management.

[0004] By means of an intelligent warehousing management system for automotive parts based on digital twin in the automotive industry chain supporting, the warehousing management efficiency can be improved, the supply chain management can be optimized, customer satisfaction can be enhanced, data-driven decision-making support can be provided, the digital transformation of enterprises can be promoted, and higher operational efficiency and competitive advantages can be brought to enterprises.

[0005] (1) Research Trends of Digital Twin

[0006] In 2003, Professor Grieves of the University of Michigan proposed the concept of digital twin, that is, the mirror space model of product life cycle management. Subsequently, NASA officially put forward the concept of digital twin and widely applied digital twin technology in the aviation field. With the continuous development of digital twin technology, it has gradually been introduced into various industries such as healthcare, manufacturing, and education. So far, digital twin technology is still evolving. Scholar Zhang Lin summarized the definitions of digital twin by different scholars in recent years and pointed out through comparative analysis that digital twin is a digital model of a physical object, which evolves in real time by receiving data from the physical object, so as to be consistent with the physical object throughout the life cycle (Zhang Lin, 2020). This scholar and scholars such as Schroeder G N also gave the correct digital twin framework from the perspective of model engineering (Zhang L, 2021; Schroeder G N et al, 2020). Scholars such as VanDerHorn E put forward a generalized definition of digital twin for different industries: digital twin is a virtual representation of a physical system (and its related environment and processes), which updates data through the information exchange between the physical system and the virtual system (VanDerHorn E et al, 2021). From the definition of digital twin, we can see that digital twin and modeling and simulation are in the same line of inheritance. On this basis, Zhang Lin and Lu Han revealed the relationship between digital twin and modeling and simulation, and explored how to draw on the theoretical methods of modeling and simulation for the construction and evaluation of digital twin (Zhang Lin, Lu Han, 2021). Scholars such as Wright L emphasized the essential difference between the model and the digital twin. The digital twin is mainly to create a digital model equivalent to the physical entity (Wright L et al, 2020). Scholars such as Singh M integrated different types of digital twins and different definitions of digital twins in the literature and gave the differences between digital twin and concepts such as "product avatar", "digital thread", "digital model" and "digital shadow" (Singh M et al, 2021).

[0007] Digital twins can achieve optimization functions such as physical state detection, equipment fault diagnosis, and behavior trend prediction. The realization of the above functions is inseparable from basic support technologies, and many scholars have conducted research on this. Scholars such as Minerva R reviewed the latest technologies used in digital twins in the context of the Internet of Things (Minerva R, 2020). Qi Q and Fuller A studied and summarized the common enabling technologies and tools for digital twins, providing technical support for the development of digital twins (Qi Q et al, 2021; Fuller A et al, 2020). Regarding the connection and mapping issues between the physical world and the virtual space in digital twins, scholars such as Wang Junfeng proposed a data mapping method for digital twin simulation for the production performance of the workshop (Wang Junfeng et al, 2021). Jiang H focused on discussing the connection implementation mechanism between the production system in the physical world at the workshop level and its mirrored virtual model (Jiang H et al, 2021).

[0008] In recent years, the rapid development of digital twins has prompted researchers and practitioners to actively design application cases of digital twins in their respective industries. The intelligent manufacturing industry has received increasing attention due to its advantages in production efficiency, cost, flexibility, sustainability, etc., and digital twin technology has demonstrated obvious advantages in the sustainable development of the intelligent manufacturing industry. When conducting a visual analysis of the keywords in the literature on digital twins in the past three years, we can also find that the frequency of appearance of intelligent manufacturing is the highest, indicating that scholars have been committed to researching the application of digital twin scenarios in the manufacturing field in recent years. He B analyzed the relevant content of intelligent manufacturing and proposed the development trend of intelligent manufacturing based on digital twin technology (He B et al, 2021). Scholars such as Shao G synthesized different views on digital twins and proposed a digital twin framework for the manufacturing industry (Shao G et al, 2020). Based on this, scholars such as Fan Y proposed an overall structure for the visualization of the digital twin of a flexible manufacturing system (FMS) (Fan Y et al, 2021). Redelinghuys A J H proposed an architecture for digital twins through a case of a manufacturing system, which can achieve data and information exchange between remote simulation or emulation and the physical twin (Redelinghuys A J H et al, 2020). Lu Y summarized issues such as the meaning and application scenarios of digital twin-driven intelligent manufacturing under the background of Industry 4.0 based on previous research (Lu Y et al, 2020). The rapid development of intelligent manufacturing has brought some challenges to the management of production workshops, such as the application of information technology and the processing of massive amounts of information. Digital twin technology provides a novel, feasible, and clear implementation path for the digital development of workshops. Kong T proposed a data construction method for the workshop digital twin system and verified the feasibility and effectiveness of the method he proposed with the application of tool wear prediction as an example (Kong T et al, 2021). Zhuang C proposed an implementation framework for the construction and application of the workshop digital twin system (Zhuang C et al, 2021). Ma J established a digital twin production management system, which can dynamically simulate and optimize the manufacturing process and achieve real-time synchronization, high-fidelity, and real-virtual fusion in cyber-physical production (Ma J et al, 2020).

[0009] In the era of Industry 4.0, product production features multi-variety and small-batch, which requires the logistics system within the factory to be highly flexible and adaptable. In response to the problems of immature 3D visualization, unsmooth information interaction, and slow data flow in the intelligent logistics warehousing system, some scholars have proposed solutions based on digital twins. Scholars such as Kosacka-Olejnik M et al. comprehensively reviewed the application research of digital twins in the logistics warehousing system and presented the research trends in the application of digital twins in the logistics warehousing system (Kosacka-Olejnik M et al, 2021). Scholars such as Leng J et al. proposed a new digital twin joint optimization method for the warehousing problems in large automated stereoscopic warehouses (Leng J et al, 2021). Glatt M established a material handling digital twin system for the problems of high complexity and high cost in the material handling system of small-batch customized products (Glatt M et al, 2021). As the application scenarios of digital twins continue to expand, the controversies and discussions around the concepts, paradigms, frameworks, applications, and technologies of digital twin brothers are increasing in both academia and industry. Many scholars have systematically reviewed digital twins from different aspects. Scholars such as Liu M and Wu J et al. elaborated in detail on the concept, technology, and system construction of DT (Liu M et al, 2021; Wu J et al, 2020). Semeraro C analyzed the ongoing research and technical challenges in constructing DTs according to different application fields and related technologies (Semeraro C et al, 2021). Scholars such as Boje C, Jones D, Rasheed A et al. provided issues such as the characteristics (Boje C et al, 2020), challenges faced (Jones D et al, 2020), and future research directions of digital twins (Rasheed A et al, 2020).

[0010] (2) Research Trends of Intelligent Warehousing

[0011] With the rapid development of modern industrial production and the logistics industry, modern warehousing is no longer the traditional warehouse management but the intelligent warehousing integrating more scientific and technological achievements. Due to the continuous expansion of enterprise scale, the sharp increase in business volume, and the rising land and labor costs, more enterprises begin to pay attention to and choose intelligent warehousing, relying on intelligent equipment and advanced information technology to gradually replace manual operations, enhance enterprise management capabilities, and promote the realization of the business goal of "cost reduction and efficiency improvement". K.L. Choy (2010) mentioned in the article "Improving Order Picking Efficiency in an RFID-Based Storage Allocation System" that due to the complexity of the system and the high procurement cost, many warehouse operators are difficult to adapt to the management mode of intelligent warehousing and lack the management experience of intelligent warehousing, and rely more on previous experience when allocating storage locations for goods. BRIDGET M CCREA (2017) believes that in warehousing management activities, manual labor is the most expensive cost in logistics warehousing, and some countries have implemented LMS (Labor Management System), which is a software for collecting workers' labor data and also an engineering standard that can effectively help workers perform specific operation tasks. Trebilcock, Bob (2015) stated in the analysis of Cisco's creation that future warehousing will focus on intelligence, automation, and integration, using sensors, intelligent connected devices, combined with technologies such as data collection and data analysis, and multiple devices will transfer more information from multiple nodes to build a dynamic supply chain that moves up and down, including dynamically responding to various events in the warehouse.

[0012] Liu Jun, Zhao Dongjie, and Xu Yan (2018) analyzed in the article "Research on the Implementation Path of the Integrated Warehouse Management and Control System Based on the Internet of Things" that most of the operating equipment configured in current warehouses does not have strict production standards, with a low degree of standardization and poor reusability. Due to the large number of domestic equipment manufacturers and the lack of strict production standards, the equipment is diverse. In the specific application of warehouses, even for equipment models with the same configuration and the same purpose, there will be significant differences in performance. This also leads to problems such as weak equipment linkage and inability to coordinate consistently when integrating multiple devices in warehousing because the technical standards used by equipment manufacturers are different. Liu Jun, Zhao Dongjie, and Xu Yan (2018) further mentioned that although intelligent warehousing can significantly improve operation efficiency and effectively reduce the error rate of manual operations in traditional warehousing, relying solely on intelligent devices or advanced information technology cannot completely avoid the problems that occur in actual operations, and corresponding management systems are needed for restraint. Chen Yi (2018) believed that by adopting intelligent devices and advanced information technology when studying the positive role of intelligent warehousing in modern logistics, the entire process of goods can be automatically managed, bringing about a subversive change to traditional warehousing. Although intelligent warehousing has changed greatly, it is still not perfect and cannot be fully automated, and still requires a small amount of manual assistance. Therefore, there is still great room for improvement in intelligent warehousing. Lai Hui (2018) studied the development status of warehousing by comparing aspects such as the overall planning of the warehousing industry, regulatory policies, and the innovation of enterprise self-management technology.

[0013] Through the research of the above scholars, it is found that although current intelligent warehousing has achieved good development results, there are still many problems, such as non-standard industry standards, inconsistent equipment standards, lack of supporting management technology, low popularity of intelligent warehousing, and pain points in the intelligent transformation and upgrading of traditional warehousing. As a crucial link in the processing and manufacturing of the automotive industry, the informatization construction and digital transformation of automotive parts warehousing have taken initial shape in recent years. However, according to on-site inspections and research, there are still problems in the enterprise's warehouse system construction, such as opaque processing information, fragmented production information, serious delays in information interaction, and a decline in manual efficiency instead of an increase. There is a large gap between the current digital transformation work and the enterprise's high-quality development goals. The key technical problems to be solved are as follows:

[0014] (1) Lack of intelligent algorithm optimization in operation scheduling and distribution routes. This project will introduce relevant optimization algorithms to effectively manage the warehouse system and plan distribution routes, improve warehouse efficiency, and reduce transportation costs.

[0015] (2) The information data of the warehouse system is too messy and fragmented. This project will select hardware and software suitable for the warehouse system in data acquisition, transmission, and storage, and accurately build a digital twin warehouse data model.

[0016] (3) The twin geometric model is not realistic enough. To solve this problem, in this project, the real-time lighting, terrain editor, and physical collision box functions of Unity3D are used, and directional lights are added to simulate the real sun light source. By adjusting the light direction, color, and object shadows, etc., the realism of the virtual warehouse is improved. Summary of the Invention

[0017] In view of this, the purpose of the present invention is to provide an intelligent warehousing system for automotive parts based on digital twin.

[0018] To achieve the above object, the present invention provides the following technical solutions:

[0019] An intelligent warehousing system for automotive parts based on digital twin, comprising:

[0020] A warehouse geometric model construction module, which is used to train a neural network through multi-view image data to construct a high-precision three-dimensional geometric model of the warehouse;

[0021] A data acquisition module, including an RFID reader / writer, a UWB positioning device, a weight sensor, a temperature and humidity sensor, and a video monitoring device, which is used to collect the operation status data, inventory data, and environmental data of the equipment in the warehouse in real time;

[0022] A data integration module, which uses a data balancing algorithm to process the collected data and fuse it with the three-dimensional geometric model to generate a dynamically updated digital twin model;

[0023] An intelligent algorithm module, including:

[0024] An inventory optimization algorithm unit, which generates an inventory adjustment plan based on historical data and demand prediction;

[0025] A path planning algorithm unit, which uses the grid method to model the warehouse environment, plans the optimal handling path based on the A* algorithm, and optimizes the number of path inflection points by adding a straight-line priority heuristic function;

[0026] An equipment scheduling algorithm unit, which schedules forklifts to perform handling tasks according to the path planning result and the forklift minimum quantity deployment model;

[0027] A visualization module, which three-dimensionally visualizes the digital twin model, the path planning result, and the equipment status based on the Unity3D engine, and supports the input and feedback of user interaction instructions;

[0028] Among them, the output end of the data acquisition module is connected to the data integration module, the output end of the data integration module is connected to the intelligent algorithm module, the output end of the intelligent algorithm module is connected to the visualization module, and the user instructions of the visualization module are fed back to the intelligent algorithm module to dynamically adjust the scheduling strategy.

[0029] Further, the warehouse geometric model construction module adopts a voxel rendering algorithm based on Neural Radiance Field (NRF), realizes three-dimensional reconstruction with millimeter-level accuracy through training a neural network with multi-view image data, and generates a dynamically updatable geometric model by fusing laser scanning point cloud data.

[0030] Further, the path planning algorithm unit specifically includes:

[0031] A grid division unit that divides the warehouse area into uniform grids and labels obstacles and walkable areas;

[0032] The core unit of the A* algorithm calculates the node priority using a mixed heuristic function of Manhattan distance and Euclidean distance;

[0033] A path optimization unit that extends the search direction to 12 or 16 neighborhoods through interpolation and introduces a straight-line priority weight to reduce path inflection points.

[0034] Further, the minimum forklift quantity deployment model is implemented through the following steps:

[0035] Abstract the warehouse shelves as a graph G=(V, E), calculate the Euclidean distance and cargo carrying capacity between each node;

[0036] Use an approximation algorithm to divide into K subsets to ensure that the total carrying capacity of each subset does not exceed the maximum capacity B of the forklift;

[0037] Based on closed-loop path planning, generate the minimum forklift quantity deployment plan and the corresponding transportation path.

[0038] Further, the data integration module further includes a fault warning unit that identifies abnormal fluctuations and triggers a fault alarm signal by analyzing the temporal characteristics of equipment operation data.

[0039] Further, the UWB positioning device includes tags deployed on forklifts and four base stations in the warehouse, realizes a three-dimensional space positioning error ≤ 0.1 meter, and synchronizes the real-time position data to the digital twin model.

[0040] Further, the visualization module supports users to adjust the inventory threshold in real time through gestures or command interfaces, and dynamically renders the inventory heat map and equipment operation trajectory.

[0041] Further, the inventory optimization algorithm unit uses an LSTM neural network to predict demand, combines it with a safety inventory model to generate a replenishment plan, and automatically triggers a purchase order.

[0042] An intelligent warehousing method for auto parts based on digital twin includes the following steps:

[0043] S1: Construct a 3D geometric model of the warehouse by fusing the NeRF algorithm with laser scanning;

[0044] S2: Collect real-time device operation data, inventory data, and environmental data, and integrate them with the geometric model after being processed by the data balancing algorithm to generate a digital twin model;

[0045] S3: Train a demand prediction model based on historical data, generate an inventory optimization plan, and trigger a replenishment order;

[0046] S4: Divide the warehouse area using the grid method, plan the optimal handling path through an improved A* algorithm, and allocate transportation tasks in combination with the forklift minimum quantity deployment model;

[0047] S5: Visualize the digital twin model, path planning results, and device status in 3D through the Unity3D engine, and receive user interaction instructions to dynamically adjust the scheduling strategy.

[0048] Furthermore, the improved A* algorithm in S4 specifically includes:

[0049] Calculate the heuristic function by weighting the Manhattan distance and the Euclidean distance;

[0050] Interpolate the grid nodes by 2×2 or 3×3 to expand the search direction;

[0051] Introduce a path smoothing factor and prefer paths with the number of turning points ≤ 3.

[0052] The beneficial effects of the present invention are as follows:

[0053] 1. Economic benefits

[0054] (1) The intelligent warehousing management system based on digital twin can reduce inventory costs, transportation costs, and labor costs by optimizing inventory management, logistics paths, and distribution plans.

[0055] (2) The system can accurately predict spare parts demand, avoid overstocking or shortages, improve the utilization efficiency of warehousing resources, and reduce the capital occupancy cost.

[0056] (3) By optimizing logistics paths, reducing transportation time, and improving logistics efficiency, the system can reduce the operating costs of the supply chain and improve the responsiveness and flexibility of the supply chain.

[0057] (4) Precise inventory management and distribution plans can improve the supply speed and accuracy of spare parts, improve after-sales service quality, and increase customer satisfaction.

[0058] 2. Social benefits

[0059] (1) The intelligent warehousing management system can track the storage, transportation, and delivery processes of spare parts in real time, improve the transparency and traceability of the supply chain, and help consumers and regulatory agencies supervise and ensure product quality and safety.

[0060] (2) Optimized logistics routes and delivery plans can reduce the driving mileage and delivery frequency of trucks, reduce traffic congestion and environmental pollution, and improve the urban traffic conditions.

[0061] (3) The application of this system has promoted the digital transformation of the automotive industry chain, improved the intelligent level and competitiveness of enterprises, and driven the development and upgrading of the entire industry.

[0062] Other advantages, objectives, and features of the present invention will, to some extent, be described in the subsequent specification, and to some extent, will be obvious to those skilled in the art based on the study of the following text, or can be taught from the practice of the present invention. The objectives and other advantages of the present invention can be realized and obtained through the following specification. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to make the objectives, technical solutions, and advantages of the present invention clearer, the present invention will be described in detail with reference to the accompanying drawings, where:

[0064] Figure 1 is the overall research idea of the project;

[0065] Figure 2 is the structural diagram of the research content;

[0066] Figure 3 is the schematic diagram of the TSDF map;

[0067] Figure 4 is the distance position matrix of production;

[0068] Figure 5 is the schematic diagram of the projection coordinates;

[0069] Figure 6 is the schematic diagram of the neural network. DETAILED DESCRIPTION OF THE INVENTION

[0070] The following illustrates the implementation manners of the present invention through specific specific examples. Those skilled in the art can easily understand other advantages and effects of the present invention from the content disclosed in this specification. The present invention can also be implemented or applied through other different specific implementation manners, and various details in this specification can also be modified or changed based on different viewpoints and applications without departing from the spirit of the present invention. It should be noted that the diagrams provided in the following embodiments only illustrate the basic concept of the present invention schematically, and the following embodiments and the features in the embodiments can be combined with each other without conflict.

[0071] Among them, the accompanying drawings are only for illustrative purposes, showing only schematic diagrams rather than actual object diagrams, and should not be construed as limiting the present invention; in order to better illustrate the embodiments of the present invention, some components in the accompanying drawings will be omitted, enlarged or reduced, which does not represent the size of the actual product; for those skilled in the art, it is understandable that some well-known structures and their descriptions in the accompanying drawings may be omitted.

[0072] In the accompanying drawings of the embodiments of the present invention, the same or similar reference numerals correspond to the same or similar components; in the description of the present invention, it should be understood that if there are terms such as "upper", "lower", "left", "right", "front", "rear", etc. indicating the orientation or positional relationship, they are based on the orientation or positional relationship shown in the accompanying drawings. This is only for the convenience of describing the present invention and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, the terms describing the positional relationship in the accompanying drawings are only for illustrative purposes and should not be construed as limiting the present invention. For those of ordinary skill in the art, the specific meanings of the above terms can be understood according to specific circumstances.

[0073] As Figure 1 shown is the overall research idea of this project. Through the investigation of automotive parts warehousing, based on the analysis of real-world requirements, this project proposes three technical problems to be solved in the current automotive parts warehousing system, namely the lack of a supporting intelligent application system, scattered basic data, and the lack of intelligent technology. For this reason, this project formulates five specific system functions of real-time connection, historical state reproduction, intelligent monitoring and beating, dynamic scheduling, and in-depth virtual-real interaction according to the problems to be solved. To achieve all the functions of the system, three main research contents are proposed in combination with the actual problems to be solved, namely digital twin model construction, data collection and integration, and intelligent algorithm design. Through three major research directions, combined with the literature found and in-depth investigation since the project was launched, the research content is refined. The specific refined research content structure is as Figure 2 shown, so the research content is summarized as follows.

[0074] The present invention provides an intelligent warehousing system for automotive parts based on digital twin, as well as a corresponding intelligent warehousing method, aiming to achieve digitalization, intelligentization, and visualization of warehousing management, improve warehousing efficiency, and reduce operating costs.

[0075] This system is mainly composed of the following modules:

[0076] Warehouse geometric model construction module: This module uses 3D modeling software or 3D scanning equipment to construct a 3D geometric model of the automotive parts warehouse, including the size of the warehouse, the layout of the shelves, the placement of equipment, etc. For example, 3D modeling software such as Revit and SketchUp can be used for manual modeling, or devices such as laser scanners and 3D cameras can be used for automatic modeling.

[0077] Data acquisition module: This module collects real-time data on equipment operation, inventory, and environment in the warehouse through various sensors and Internet of Things devices, including:

[0078] RFID reader / writer: Used to read RFID tags on goods to obtain information on the location and quantity of goods.

[0079] Barcode scanner: Used to scan barcodes on goods to obtain information on the goods.

[0080] Weight sensor: Used to measure the weight of goods.

[0081] Temperature sensor: Used to measure the temperature in the warehouse.

[0082] Humidity sensor: Used to measure the humidity in the warehouse.

[0083] Video surveillance equipment: Used to monitor the security status in the warehouse.

[0084] Data integration module: This module integrates the collected data with the warehouse geometric model to form a complete digital twin model, which truly reflects the operating state of the warehouse. For example, a TSDF map can be used to fuse point cloud data with a 3D geometric model to construct a high-precision digital twin model.

[0085] Intelligent algorithm module: This module uses the digital twin model for inventory optimization, path planning, and equipment scheduling to provide intelligent decision-making support for warehouse management. It includes:

[0086] Inventory optimization algorithm: Based on historical data and demand forecasting, analyze the inventory level and generate an inventory optimization plan, such as adjusting the inventory level, formulating a replenishment plan, etc.

[0087] Path planning algorithm: Based on the warehouse layout and inventory information, plan the optimal goods handling path, such as the path from shelf to shelf, the path from outbound to the distribution area, etc.

[0088] Equipment scheduling algorithm: Based on the equipment operating state and task requirements, schedule the equipment for operations, such as controlling the handling robot to carry goods along the planned path, controlling the stacker to store and retrieve goods, etc.

[0089] Visualization module: This module presents the results of the digital twin model and intelligent algorithms to users in a 3D visualization manner and supports users to perform interactive operations, such as querying inventory information, adjusting the inventory level, viewing the equipment status, etc. For example, the Unity3D engine can be used to build a virtual warehouse environment and present the results of the digital twin model and intelligent algorithms to users in a 3D visualization manner.

[0090] The intelligent warehousing method of the present invention mainly includes the following steps:

[0091] Construct a warehouse geometric model: Use 3D modeling software or 3D scanning equipment to construct a 3D geometric model of the auto parts warehouse.

[0092] Collect warehouse data: Through sensors and Internet of Things devices, collect the equipment operation data, inventory data, and environmental data in the warehouse in real time.

[0093] Integrate the digital twin model: Integrate the collected data with the warehouse geometric model to form a complete digital twin model.

[0094] Optimize inventory: According to historical data and demand forecasts, use inventory optimization algorithms to analyze the inventory level and generate an inventory optimization plan.

[0095] Perform path planning: According to the inventory optimization plan and warehouse layout, use path planning algorithms to plan the optimal goods handling path.

[0096] Perform equipment scheduling: According to the path planning plan and equipment operation status, use equipment scheduling algorithms to schedule equipment for operations.

[0097] Visualization display: Present the results of the digital twin model and intelligent algorithms to users in a 3D visualization manner and support users to perform interactive operations.

[0098] I. Research on Two-dimensional to Three-dimensional Model Conversion of Images Based on KinectFusion

[0099] (I) Algorithm Introduction

[0100] This algorithm uses an RGBD camera for a real-time dense three-dimensional reconstruction system. The algorithm requires a GPU, or even multiple GPUs or a high-performance GPU for real-time image operations. The map it uses is a Truncated Signed Distance Function (TSDF) map, which also has great significance for the development of subsequent dense maps. Currently, many real-time three-dimensional reconstruction systems in dynamic environments are extended based on KinectFusion or ElasticFusion. The real-time three-dimensional reconstruction technology has a great correlation with SLAM. The difference is that three-dimensional reconstruction pays more attention to the integrity and accuracy of mapping, while SLAM technology pays more attention to positioning accuracy. Mapping is to assist positioning and does not pay much attention to the accuracy and density of the map.

[0101] (II) Algorithm Principle

[0102] A 3D image is a special form of information representation, characterized by data in three dimensions in the expressed space. Its forms of expression include depth maps, geometric models, and point cloud models. Compared with 2D images, 3D images can achieve natural object-background decoupling by virtue of the information in the third dimension. Point cloud data is the most common and fundamental 3D model.

[0103] A point cloud is a set of a large number of spatial points that represent the spatial distribution of an object and the characteristics of its surface in the same spatial reference system. After obtaining the spatial coordinates of each sampling point on the object surface, a set of points is obtained, which is called a point cloud (PointCloud, PC). The normal vector of a point cloud is an important feature and can be used to match feature points and perform ICP to match the pose of point clouds. Currently, the PCL library has implemented a large number of operations on point clouds, and its status is similar to that of OpenCV for images.

[0104] Since the map of the algorithm is TSDF, it is necessary to understand TSDF for the construction of the algorithm. TSDF is translated as Truncated Signed Distance Function. It is a grid-like map. First, a 3D space to be modeled is selected, such as 3×3×3m 3 in size. According to a certain resolution, this space is divided into many small blocks, and the information inside each small block is stored. Its schematic diagram is as Figure 3 shown. The entire TSDF map is stored in the video memory rather than in the memory. Since the calculations of each voxel do not interfere with each other, the parallel characteristics of the GPU can be utilized to calculate and update each voxel in parallel.

[0105] Inside each TSDF voxel, the distance between this small block and the nearest object surface is stored. If the small block is in front of the nearest object surface, it has a positive value; conversely, if the small block is behind the surface, then this value is negative. Since the object surface is usually a very thin layer, values that are too large and too small are both taken as 1 and -1, and this gives the truncated distance, as Figure 4 shown, which is the so-called TSDF.

[0106] Based on each frame of depth image, the point cloud of this frame can be obtained. First, noise point cloud data is removed through a bilateral filter, and then the normal vector of each spatial point is calculated. The specific method for calculating the normal vector is shown in the following formula.

[0107] N k (u) = v[(V k (u + 1, v) - V k (u, v)) × (V k (u, v + 1) - V k (u, v))]

[0108] where

[0109] v[x] = x / ∥x∥ 2

[0110] Essentially, it is to find the vector between the current point and the adjacent point, and then calculate the vector product (also known as the outer product or cross product) of the two vectors. The vector product is perpendicular to the two vectors, and the normal vector can be obtained by normalizing this vector. KinectFusion does not directly calculate the pose through the ICP algorithm using the current frame and the previous frame of images. Instead, it projects the depth image predicted by the previous frame (Frame-to-Model) through the maintained TSDF map and the pose of the previous frame of image. This method has much better effect than directly calculating the pose between two frames, and the error of the latter will gradually accumulate, seriously affecting the reconstruction accuracy. KinectFusion uses the ICP algorithm to estimate the camera pose, and there are many explanations of the principle on the Internet. First, the point cloud of the current frame is transformed into the global coordinate system (according to the currently assumed pose T), and then transformed into the camera coordinate system of the previous frame, and projected onto the image coordinate system to calculate the corresponding point in the previous frame. The schematic diagram of the projection coordinates is as Figure 5 shown. By comparing the distance and normal vector of the above-mentioned matching points, it can be seen whether they are matching points. After obtaining the matching points, the pose is calculated by minimizing the distance from the point to the plane, and the objective function is shown as follows:

[0111]

[0112] where q i is a point in the current point cloud, p i is the spatial point after the pose transformation of its matching point, is the vector passing through the two points, and in Figure 6 it is the vector d i s i . Multiply the vector by the normalized normal vector n i of d i to obtain the point s i ; the distance to the tangent plane of the point d i , which is the error term we want to minimize. When the two points are completely coincident after projection, this term is 0.

[0113] (2) Model Construction and Conclusion

[0114] The dataset for model construction uses a general dataset for 3D model construction. The dataset is imported into the algorithm for CUDA-based GPU modeling. This model is constructed using an RTX 2080 graphics card and iteratively performs 3D fitting using languages and computing architectures such as Python 3.7, Pycharm 2018, and Pytorch to construct a 3D object model. The following conclusions are obtained through observation. It does not take into account the environmental transformation and is not robust enough in complex scenarios. The generated point cloud does not have loop detection and other links, and there is a large gap between the model and the actual object.

[0115] Therefore, research on high-precision model construction was carried out, and a voxel rendering algorithm based on Neural Radiance Field was proposed.

[0116] II. Research on Converting 2D Images to 3D Models Based on the Voxel Rendering Algorithm of Neural Radiance Field (1) Algorithm Introduction

[0117] NeRF can be briefly summarized as using an MLP (fully connected layer instead of convolution, plus activation layer) neural network to implicitly learn a static 3D scene and achieve the synthesis (rendering) of arbitrary new perspectives of complex scenes.

[0118] To train the network, for a static scene, a training set containing a large number of images with known camera parameters, as well as the 3D coordinates of the cameras corresponding to the images and the camera orientations (2D, but actually represented by 3D unit vectors) need to be provided.

[0119] Using multi-view data for training, the target positions in space have higher density and more accurate colors, which prompts the neural network to predict a more continuous scene model.

[0120] Taking an arbitrary camera position + orientation as input and performing volume rendering through the trained neural network, the image result can be rendered.

[0121] (2) Algorithm Principle

[0122] The NeRF function represents a continuous scene as a function with a 5D vector as input, including the 3D coordinate position x = (x, y, z) of a spatial point and the direction (θ, φ). The output is the color c = (r, g, b) of the 3D point related to the perspective and the density σ at the corresponding position. In practice, the direction is represented by a 3D Cartesian unit vector d, so this neural network can be written as: F(θ): (x, d) - (c, σ). The schematic diagram of the neural network is as Figure 6 shown.

[0123] In the specific implementation, x is first input into the MLP network, and σ and a 256-dimensional intermediate feature are output. The intermediate feature and d are then input into an additional fully connected layer (128-dimensional) to predict the color.

[0124] In this embodiment, the parts warehouse of an automobile manufacturing enterprise is used as the application scenario. The warehouse area is about 10,000 square meters, storing key components such as engines and gearboxes, and is equipped with equipment such as handling robots, stackers, and UWB positioning forklifts.

[0125] 1. Construction of Warehouse Geometric Model

[0126] Implementation method:

[0127] Data collection: Use a Kinect Azure depth camera to capture panoramic views of the warehouse from 12 different perspectives to obtain multiple groups of RGB-D images; simultaneously use a Leica BLK360 laser scanner to generate high-density point cloud data (accuracy ±1mm).

[0128] Model reconstruction: Input the above data into a voxel rendering algorithm (PyTorch framework) based on NeRF (Neural Radiance Field), and learn the radiance field and density field of the scene through an MLP network to generate a three-dimensional geometric model with millimeter-level accuracy.

[0129] Dynamic update: The model is fused with the laser point cloud data, and the displacement of the shelves or the change of the equipment position (such as the moving trajectory of the forklift) is updated in real time through a TSDF (Truncated Signed Distance Function) map.

[0130] 2. Data collection and integration

[0131] Implementation method:

[0132] Sensor deployment:

[0133] RFID reader / writer: Install an Impinj R420 reader / writer on each layer of the shelves to read the goods labels (EPC encoding) in real time, with a positioning accuracy of ±5cm.

[0134] UWB positioning system: The forklift is equipped with a Decawave DW1000 tag, and AnkerWork base stations are deployed at the four corners of the warehouse to achieve three-dimensional positioning (error ≤0.1m).

[0135] Environmental monitoring: Install Sensirion SHT45 temperature and humidity sensors in the cargo area, with a data sampling frequency of 1Hz.

[0136] Data integration:

[0137] Data balancing algorithm: Use a sliding window method (window size 10s) to smooth the sensor data and eliminate instantaneous noise.

[0138] Fault warning: Analyze the time series data of the equipment current through an LSTM network. If the fluctuation exceeds the threshold (±15%) for 3 consecutive cycles, trigger an alarm (such as the motor overload of the stacker).

[0139] Digital twin synchronization: The integrated data is mapped to the three-dimensional model through the OPC UA protocol to update the inventory location, forklift status, and environmental parameters in real time.

[0140] 3. Operation of intelligent algorithm module

[0141] 3.1 Inventory optimization

[0142] Implementation method:

[0143] Demand forecasting: Use an LSTM network (64 nodes in the hidden layer) to analyze the sales data of the past 12 months, and predict that the demand for engines next month is 950 ± 50 units.

[0144] Replenishment strategy: If the current inventory is 800 units and the safety stock threshold is 200 units, the system automatically generates a replenishment order (quantity = predicted value + safety stock - current inventory) and triggers the procurement process through the ERP interface.

[0145] 3.2 Path planning

[0146] Implementation method:

[0147] Environmental modeling: Divide the warehouse into 0.5m × 0.5m grids, and mark the obstacles (red) and the walkable areas (green).

[0148] A* algorithm optimization:

[0149] Hybrid heuristic function: F(n) = 0.7 × Manhattan distance + 0.3 × Euclidean distance, balancing the path length and the number of inflection points.

[0150] Neighborhood expansion: Interpolate the grids with 3 × 3, and expand the search direction from 8 directions to 16 directions to reduce the risk of local optimality.

[0151] Path smoothing: Introduce a straight line weight factor (λ = 0.2), and prefer paths with the number of inflection points ≤ 3.

[0152] Output example: Plan the optimal path from shelf A3 to the outbound area (total length 35.2m, 2 inflection points), and the time consumption is <0.5s.

[0153] 3.3 Forklift scheduling

[0154] Implementation method:

[0155] Minimum quantity model:

[0156] Problem modeling: Abstract the warehouse shelves as a graph G = (V, E), the node carrying capacity Δ_i ∈ [50, 200] kg, and the maximum capacity of the forklift B = 1000 kg.

[0157] Approximation algorithm: Use a greedy strategy to divide into K = 3 subsets (V1 - V3), ensure that ∑Δ_i ≤ B, and generate a closed-loop path.

[0158] Real-time scheduling: According to the path planning results, dynamically allocate 3 forklifts to perform transportation tasks, and the equipment utilization rate is increased to 92%.

[0159] 4. Visualization and interaction

[0160] Embodiment:

[0161] 3D Rendering: Build a virtual warehouse based on the Unity3D engine, and render the heat map of the shelves in real time (the higher the inventory, the redder the color), the forklift trajectory (blue line), and the equipment status (green for normal, red for failure).

[0162] Interactive Function:

[0163] Gesture Operation: Support gesture recognition of the Hololens 2 AR glasses, and users can adjust the position of the shelves through the grasping action.

[0164] Command Interface: Drag the slider on the PC interface to set the inventory threshold, and the system automatically optimizes the replenishment strategy and updates the visualization results.

[0165] 5. Implementation Effect

[0166] The inventory turnover rate is increased by 30%, the transportation cost is reduced by 18%, and the failure downtime is reduced by 45%.

[0167] Technical Advantages:

[0168] Through the fusion modeling of NeRF + laser point cloud, the model accuracy reaches the millimeter level (the error of the traditional method > 5 cm).

[0169] The path planning efficiency of the improved A* algorithm is increased by 40%, and the inflection points are reduced by 50%.

[0170] The forklift scheduling model realizes the minimization of the number of vehicles (K = 3), reducing 1 - 2 vehicles compared with the empirical scheduling.

[0171] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention and are not intended to limit them. Although the present invention has been described in detail with reference to the preferred embodiments, those of ordinary skill in the art should understand that the technical solutions of the present invention can be modified or equivalently replaced without departing from the spirit and scope of the present technical solution, and they should all be covered within the scope of the claims of the present invention.

Claims

1. The intelligent storage system for automobile parts based on digital twins is characterized by: include: Warehouse geometry model building module, which is used to train neural networks through multi-view image data to build a high-precision 3D geometry model of the warehouse; Data collection module, including RFID reader / writer, UWB positioning device, weight sensor, temperature and humidity sensor and video monitoring equipment, used to collect real-time operation status data, inventory data and environmental data of equipment in the warehouse; A data integration module, which processes the collected data using a data balancing algorithm and fuses the data with the three-dimensional geometric model to generate a dynamically updated digital twin model; Intelligent algorithm modules, including: Inventory optimization algorithm unit, which generates inventory adjustment plans based on historical data and demand forecasts; The path planning algorithm unit uses the grid method to model the warehouse environment, plans the optimal transportation path based on the A-star algorithm, and optimizes the number of path turning points by adding a straight line priority heuristic function; The equipment scheduling algorithm unit schedules forklifts to perform handling tasks based on the path planning results and the minimum number of forklift deployment model; The visualization module uses the Unity3D engine to visualize the digital twin model, path planning results, and equipment status in three dimensions, and supports the input and feedback of user interaction commands; Among them, the output end of the data acquisition module is connected to the data integration module, the output end of the data integration module is connected to the intelligent algorithm module, the output end of the intelligent algorithm module is connected to the visualization module, and the user instructions of the visualization module are fed back to the intelligent algorithm module to dynamically adjust the scheduling strategy.

2. The digital twin-based intelligent storage system for automobile parts according to claim 1 is characterized in that: The warehouse geometry model building module adopts a voxel rendering algorithm based on Neural Radiance Field (NRF), trains a neural network with multi-view image data to achieve millimeter-level precision three-dimensional reconstruction, and integrates laser scanning point cloud data to generate a dynamically updateable geometry model.

3. The digital twin-based intelligent storage system for automobile parts according to claim 1 is characterized in that: The path planning algorithm unit specifically includes: Grid division unit, which divides the warehouse area into uniform grids and marks obstacles and walkable areas; The core unit of the A-star algorithm uses a hybrid heuristic function of Manhattan distance and Euclidean distance to calculate node priority; The path optimization unit expands the search direction to 12 or 16 neighborhoods through interpolation and introduces straight line priority weights to reduce path turning points.

4. The digital twin-based intelligent storage system for automobile parts according to claim 3 is characterized in that: The minimum number of forklift deployment model is implemented by the following steps: Abstract the warehouse shelves into a graph G = (V, E), and calculate the Euclidean distance between each node and the cargo carrying capacity; Use an approximate algorithm to divide the K subsets to ensure that the total load of each subset does not exceed the maximum capacity B of the forklift; Based on closed-loop path planning, generate the minimum number of forklift deployment plans and corresponding transportation routes.

5. The digital twin-based intelligent storage system for automobile parts according to claim 1 is characterized in that: The data integration module further includes a fault warning unit, which identifies abnormal fluctuations and triggers a fault alarm signal by analyzing the time series characteristics of the equipment operation data.

6. The automotive parts intelligent warehousing system based on digital twins according to claim 1 is characterized by: The UWB positioning device includes a tag deployed on a forklift and four base stations in the warehouse, which can achieve a three-dimensional spatial positioning error of ≤0.1 meter and synchronize real-time location data to the digital twin model.

7. The automotive parts intelligent warehousing system based on digital twins according to claim 1 is characterized by: The visualization module supports users to adjust inventory thresholds in real time through gestures or command interfaces, and dynamically renders inventory heat maps and equipment operation trajectories.

8. The digital twin-based intelligent storage system for automobile parts according to claim 1 is characterized in that: The inventory optimization algorithm unit uses an LSTM neural network to predict demand, and combines the safety inventory model to generate a replenishment plan and automatically trigger a purchase order.

9. An intelligent warehousing method for automobile parts based on digital twins, characterized by: The following steps are involved: S1: Constructing a 3D geometric model of the warehouse by fusing the NeRF algorithm with laser scanning; S2: Real-time collection of equipment operation data, inventory data and environmental data, which are processed by the data balance algorithm and integrated with the geometric model to generate a digital twin model; S3: Train the demand forecasting model based on historical data, generate inventory optimization solutions and trigger replenishment instructions; S4: The grid method is used to divide the warehouse area, the optimal transportation path is planned by improving the A-star algorithm, and the transportation tasks are allocated in combination with the minimum number of forklift deployment model; S5: The digital twin model, path planning results and equipment status are visualized in three dimensions through the Unity3D engine, and user interaction instructions are received to dynamically adjust the scheduling strategy.

10. The intelligent storage method for automobile parts based on digital twins according to claim 9 is characterized in that: The improved A-star algorithm in S4 specifically includes: Manhattan distance and Euclidean distance are used to calculate the heuristic function. Perform 2×2 or 3×3 interpolation on the grid nodes to expand the search direction; A path smoothing factor is introduced to give priority to paths with inflection points ≤ 3.

Images