A packaging recycling box management system and management method

Abstract

The present invention relates to the technical field of packaging circulation box management, and in particular to a packaging circulation box management system and management method, in which a colored light strip is arranged on the outer edge of the packaging circulation box, and a storage unit, a Bluetooth unit and a box control unit are embedded in the box body, so as to realize real-time data collection and storage of the circulation box. The circulation box positioning module includes positioning communication devices on the site and the transport vehicle, which interact with the packaging circulation box data and transmit it to the intelligent cloud management module, so as to realize real-time tracking and monitoring of the circulation box and ensure the accuracy and timely update of the data. The intelligent cloud management module determines the use status of the circulation box according to the real-time data and makes dynamic adjustments and reasonable allocations. The circulation box is scheduled by a reinforcement learning algorithm, and a scheduling plan is generated based on the site demand, the current position of the circulation box and the real-time traffic conditions to improve the rationality of the site distribution of the circulation box. The abnormal alarm module detects the loss of the circulation box and issues an alarm to reduce losses and improve safety.

Description

Technical Field

[0001] The present invention relates to the technical field of packaging circulation box management, and in particular to a packaging circulation box management system and a management method. Background Art

[0002] In recent years, with the rapid development of logistics and e-commerce, packaging recycling boxes have been increasingly used in transportation and warehousing. Packaging recycling boxes are reusable containers that can effectively reduce logistics costs and environmental pollution. However, with the increase in the number of recycling boxes and the complexity of logistics networks, how to efficiently manage and dispatch these recycling boxes has become an urgent problem to be solved.

[0003] In traditional systems, the scheduling of recycling boxes is mainly done manually. Staff need to manually record and allocate recycling boxes to handle a large number of orders and demands. This method is not only time-consuming and labor-intensive, but also prone to human errors. The inventory management of traditional systems is relatively simple, usually only recording the number and location of recycling boxes, and lacks detailed tracking of the real-time status and usage of recycling boxes. The inaccuracy and lag of inventory information lead to a lack of data support for scheduling decisions, making it difficult to respond to changes in market demand in a timely manner. Traditional systems are difficult to dynamically adjust according to the real-time needs and usage of each site, often resulting in a shortage of recycling boxes at some sites and a surplus at other sites. Due to the lack of intelligent scheduling methods, the system cannot effectively balance the distribution of recycling boxes between sites. When planning transportation routes, traditional systems usually only consider the shortest path or fixed route, and fail to fully consider real-time traffic conditions and the demand priorities of each site, resulting in low transportation efficiency.

[0004] At the same time, due to the large number of circulating boxes and their similar appearance, it is usually difficult for staff to clearly classify the sealed boxes, which makes the inspection process cumbersome. It is also impossible to monitor the goods loaded in the missed boxes in real time, which increases the difficulty of management. Summary of the invention

[0005] To solve the above problems, the present invention provides a packaging circulation box management system and management method. By setting an interactive Bluetooth device in the circulation box and setting a communication device that interacts with the Bluetooth device in the key transportation node, data for effective reinforcement learning is provided, and a scheduling strategy for the circulation box is output based on targeted reinforcement learning.

[0006] To achieve the above object, the technical solution adopted by the present invention is:

[0007] A packaging circulation box management system, comprising a packaging circulation box, a circulation box positioning module, an intelligent cloud management module and an abnormal alarm module, wherein the packaging circulation box, the circulation box positioning module and the intelligent cloud management module are communicatively connected in sequence, and the abnormal alarm module is communicatively connected with the circulation box positioning module;

[0008] A colored light strip is arranged on the outer edge of the packaging circulation box, and a storage unit, a Bluetooth unit and a box control unit are embedded in the packaging circulation box. The storage unit is used to store real-time data of the circulation box, and the box control unit is used to control the opening and closing of the packaging circulation box, control the color change of the colored light strip and control the Bluetooth unit to send and receive usage data of the circulation box;

[0009] The circulation box positioning module includes a positioning communication device arranged at the site and a positioning communication device arranged at the transport vehicle, and the positioning communication device is used to perform data interaction with the packaging circulation box and communicate with the intelligent cloud management module;

[0010] The intelligent cloud management module is used to determine the use status of the circulation box according to the real-time data of the storage circulation box, and the use status includes idle, in use at the starting point, on the way and at the destination, and based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box;

[0011] The abnormal alarm module is used to determine whether the circulation box is lost according to the real-time data of the circulation box and issue an abnormal alarm;

[0012] Determining the use status of the circulation box according to the real-time data of the storage circulation box comprises the following steps:

[0013] Receive and store real-time data uploaded from each circulation box;

[0014] Compare the current position of the circulation box with the cargo starting point and cargo destination to determine whether the circulation box is located at the starting point, destination or transportation path;

[0015] If the circulation box is at the cargo starting point and is not loaded, the marked use status is idle; if the circulation box is at the cargo starting point and is loaded, the marked use status is in use at the starting point; when the circulation box is on the transportation path, the marked use status is on the way; if the circulation box arrives at the cargo destination and is not unloaded, the marked use status is arrived at the destination; if the circulation box arrives at the cargo destination and is unloaded, the marked use status is idle;

[0016] The method of scheduling the circulation box based on the reinforcement learning algorithm according to the real-time traffic conditions, station requirements and the use status of the circulation box includes the following steps:

[0017] Collect historical traffic data for each station, including daily recycling box usage and returns;

[0018] Predict the demand for recycling boxes at each station within a preset time in the future based on historical traffic data, and calculate the actual demand at each station based on the future demand and the current inventory of recycling boxes at each station;

[0019] The actual demand of each station, the current position of each cycle box and the real-time traffic conditions between stations are taken as the states, and the number of cycle boxes transported between stations is taken as the action. The reward function is defined based on the degree of satisfying the actual demand of each station and the degree of satisfying the actual demand of each station, and the DQN network is trained.

[0020] Based on the trained DQN network, the number of transport loop boxes between sites is generated in real time, and the number of idle loop boxes required for a single delivery is obtained to schedule the loop boxes.

[0021] The abnormality alarm module detects abnormalities by monitoring the position signal and usage data of the circulation box and utilizing the associated information in the Bluetooth network.

[0022] Furthermore, the real-time data of the circulation box includes cargo loading status, single storage cargo type, cargo serial number, cargo origin and cargo destination, and also includes adjacent cargo association set and light color strip color data.

[0023] Furthermore, controlling the color change of the color light strip comprises the following steps:

[0024] Each circulation box continuously scans the Bluetooth signals of other circulation boxes within a preset range through its Bluetooth unit, and collects the cargo type, cargo serial number and cargo batch information of other circulation boxes through encryption logic;

[0025] The box control unit compares the information of other circulation boxes collected within a preset range with the information of the current circulation box to determine whether they belong to the same batch of goods;

[0026] If they belong to the same batch of goods, the box control unit generates a color signal corresponding to the same batch of goods according to a preset color display rule, associates the circulation boxes of the same batch of goods within a preset range, and updates the adjacent goods association set.

[0027] Furthermore, the formula of the DQN network is as follows:

[0028] ;

[0029] Among them, s is the current state parameter set, including the actual demand of each station, the current position of each circulation box and the real-time traffic conditions between stations; is the current action parameter set, including the number of transport cycle boxes between stations; is the action parameter set taken under the state parameter set s Q value; is the learning rate; r is the immediate reward of the current action; is the discount factor; is the next state parameter set; is the action parameter set in the next state.

[0030] Furthermore, the formula of the reward function is as follows:

[0031] ;

[0032] in, is the reward value; and is the weight coefficient; The actual site demand that is met; is the total demand of the site; The actual transportation time; The maximum transport time.

[0033] Furthermore, the real-time traffic conditions between the stations include a plurality of real-time traffic condition values, and the real-time traffic condition values ​​are obtained by the following steps:

[0034] Access the real-time navigation API to obtain estimated transportation time between each station;

[0035] The estimated transport time is graded according to the preset time gradient, and each grade is assigned a value according to the gradient to obtain the real-time traffic status value.

[0036] A packaging circulation box management method is applied to any one of the packaging circulation box management systems described above, and is characterized by comprising:

[0037] A positioning communication device is arranged at the site, and a positioning communication device is arranged in the transport vehicle, wherein the positioning communication device is used to perform data exchange with the packaging circulation box;

[0038] The use status of the circulation box is determined based on the real-time data of the storage circulation box obtained from the communication device, and the use status includes idle, in use at the starting point, on the way and at the destination. Based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box.

[0039] The beneficial effects of the present invention are as follows: the present invention arranges a color light strip on the outer edge of the circulation box through packaging, and a storage unit, a Bluetooth unit and a box control unit are embedded in the box. Through the Bluetooth unit and the storage unit, the system can automatically collect and store the real-time data of the circulation box, including the current position of the circulation box, the loading status of the goods, the type of goods, the serial number of the goods, the starting point of the goods and the destination of the goods. The circulation box positioning module includes a positioning communication device arranged at the site and a positioning communication device arranged on the transport vehicle. These devices interact with the packaging circulation box and transmit the data to the intelligent cloud management module to realize real-time tracking and monitoring of the circulation box, ensuring the accuracy and timely update of the information. The intelligent cloud management module is used to determine the use status of the circulation box according to the stored real-time data of the circulation box, including idle, in use at the starting point, on the way and at the destination. In this way, the system can dynamically adjust the use status of the circulation box and reasonably allocate it according to demand. Based on the reinforcement learning algorithm, the deep Q network (DQN) is used to schedule the circulation box. The system takes the actual demand of each station, the current position of each recycling box and the real-time traffic conditions between stations as the state, and the number of recycling boxes transported between stations as the action, and defines the reward function based on the degree of meeting the actual demand of each station and the realization of meeting the actual demand of each station, and trains the DQN network. Based on the trained DQN network, the system can generate the number of recycling boxes transported between stations in real time, and obtain the number of idle recycling boxes required for a single delivery of goods, and perform recycling box scheduling. In this way, the system can dynamically adjust according to the real-time traffic conditions and station requirements, improve transportation efficiency, and balance the distribution of recycling boxes between stations. The abnormal alarm module is used to determine whether the recycling box is lost based on the real-time data of the recycling box and issue an abnormal alarm. Through real-time data monitoring and intelligent analysis, the system can promptly detect abnormal situations of the recycling box and take quick measures to reduce losses and improve safety. Through the comprehensive application of Internet of Things technology, intelligent cloud management and reinforcement learning algorithms, this solution realizes automatic data collection, intelligent inventory management, intelligent scheduling and optimization, automatic abnormal detection and alarm, color identification and batch management of packaging recycling boxes, and realizes reasonable scheduling of recycling boxes and optimizes management efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] Figure 1 It is a structural schematic diagram of a packaging circulation box management system in the present invention.

[0041] Figure 2 It is a flow chart of the steps of scheduling the circulation box based on the reinforcement learning algorithm according to the real-time traffic conditions, station requirements and the use status of the circulation box in the present invention. DETAILED DESCRIPTION

[0042] See also Figure 1-2As shown, the present invention relates to a packaging circulation box management system, comprising a packaging circulation box, a circulation box positioning module, an intelligent cloud management module and an abnormal alarm module, wherein the packaging circulation box, the circulation box positioning module and the intelligent cloud management module are communicatively connected in sequence, and the abnormal alarm module is communicatively connected with the circulation box positioning module;

[0043] A colored light strip is arranged on the outer edge of the packaging circulation box, and a storage unit, a Bluetooth unit and a box control unit are embedded in the packaging circulation box. The storage unit is used to store real-time data of the circulation box, and the box control unit is used to control the opening and closing of the packaging circulation box, control the color change of the colored light strip and control the Bluetooth unit to send and receive usage data of the circulation box;

[0044] The circulation box positioning module includes a positioning communication device arranged at the site and a positioning communication device arranged at the transport vehicle, and the positioning communication device is used to perform data interaction with the packaging circulation box and communicate with the intelligent cloud management module;

[0045] The intelligent cloud management module is used to determine the use status of the circulation box according to the real-time data of the storage circulation box, and the use status includes idle, in use at the starting point, on the way and at the destination, and based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box;

[0046] The abnormal alarm module is used to determine whether the circulation box is lost according to the real-time data of the circulation box and issue an abnormal alarm.

[0047] In some embodiments, a color light strip is provided on the outer edge of the packaging circulation box, and the color light strip is composed of a high-brightness LED light, which can display multiple colors and is used to identify different states and batch information. The packaging circulation box is embedded with a storage unit, a Bluetooth unit and a box control unit. The storage unit adopts a high-capacity non-volatile memory to store real-time data of the circulation box, including information such as cargo loading status, cargo type, cargo serial number, cargo starting point and cargo destination. The Bluetooth unit adopts a low-power Bluetooth module to realize data transmission and reception between the circulation box and other devices. The box control unit has a built-in microcontroller, which is responsible for processing data from the storage unit and the Bluetooth unit, and controlling the opening and closing of the circulation box and the color change of the color light strip. The circulation box positioning module includes a positioning communication device set at the site and a positioning communication device set at the transport vehicle. The site positioning communication device adopts a fixed installation method, integrates GPS and RFID readers, is used to locate the position of the circulation box in real time, and communicates data with the intelligent cloud management module. The transportation vehicle positioning communication device adopts a mobile installation method, integrates GPS and Bluetooth modules, and realizes real-time positioning and data interaction of the circulation box during transportation. The intelligent cloud management module is built through a remote server and is equipped with a large-capacity storage device and a high-speed processor. The intelligent cloud management module communicates data with the circulation box positioning module through the Internet, and uses the reinforcement learning algorithm to process and analyze the real-time data uploaded from each circulation box to determine the use status of the circulation box, including idle, in use at the starting point, on the way, and at the destination. Based on the real-time traffic conditions and the needs of each station, the intelligent cloud management module generates an optimized circulation box scheduling plan and sends the scheduling instructions to the corresponding circulation boxes and transportation vehicles. The abnormal alarm module monitors the location signal and usage data of the circulation box and uses the associated information in the Bluetooth network for abnormal detection. When a circulation box is detected to be lost during transportation, other associated circulation boxes cannot detect the signal of the circulation box in the Bluetooth network. This information will be sent to the intelligent cloud management module through the circulation box positioning module. The intelligent cloud management module determines whether the circulation box is lost based on the received abnormal data, and triggers the abnormal alarm module to send an alarm signal to notify the management personnel to check and handle it.

[0048] Furthermore, the real-time data of the circulation box includes cargo loading status, single storage cargo type, cargo serial number, cargo origin and cargo destination, and also includes adjacent cargo association set and light color strip color data.

[0049] In some embodiments, first, when loading goods, the circulation box can automatically detect and record the loading status of the goods through the set pressure sensor, or record the loading status of the goods through manual input, including whether it has been loaded, the loading amount, and the loading time. These data are transmitted to the storage unit through the Bluetooth unit, and updated to the intelligent cloud management module in real time through the circulation box positioning module, so that the management system can monitor the loading status of each circulation box at any time. The circulation box will record the type and serial number of the goods every time it stores goods. The type of goods can be obtained by barcode scanning or RFID tag reading, and the serial number is the unique identifier of each piece of goods. These data ensure the traceability and uniqueness of the goods to prevent confusion and misdelivery. For the data of the starting point and destination of the goods, the system automatically inputs and records the starting point and destination of the goods through the site positioning communication device during each loading and unloading process (it can also be recorded manually). These data can help the intelligent cloud management module track the transportation path of the goods and ensure that the goods can be delivered to the destination as planned. The adjacent goods association set refers to the association information between adjacent circulation boxes in the same batch of transportation. During transportation, each circulation box continuously scans the Bluetooth signals of other circulation boxes within the preset range through its Bluetooth unit, and collects the cargo type, cargo serial number and batch information of other circulation boxes. The box control unit compares the collected information of the adjacent circulation box with the information of the current circulation box to determine whether they belong to the same batch of goods. If they belong to the same batch of goods, the box control unit will generate and store the adjacent cargo association set, including the associated circulation box number and related cargo information. These association information are uploaded to the intelligent cloud management module for subsequent management and scheduling. The light color bar color data refers to the color information displayed by the color light bar on the outside of the packaging circulation box. The color light bar is composed of high-brightness LED lights and can display a variety of colors. It is used to identify the colors corresponding to different states and batch information for staff to identify. When loading goods, the box control unit interacts with the adjacent circulation box through the Bluetooth unit according to the preset color display rules, determines and generates color signals corresponding to the same batch of goods. These color data are stored in the storage unit and uploaded to the intelligent cloud management module through the circulation box positioning module for real-time monitoring and management.

[0050] Furthermore, controlling the color change of the color light strip comprises the following steps:

[0051] Each circulation box continuously scans the Bluetooth signals of other circulation boxes within a preset range through its Bluetooth unit, and collects the cargo type, cargo serial number and cargo batch information of other circulation boxes through encryption logic;

[0052] The box control unit compares the information of other circulation boxes collected within a preset range with the information of the current circulation box to determine whether they belong to the same batch of goods;

[0053] If the goods belong to the same batch, the box control unit generates a color signal corresponding to the goods of the same batch according to a preset color display rule, associates the circulation boxes of the goods of the same batch within a preset range, and updates the adjacent goods association set.

[0054] In some embodiments, each circulation box continuously scans the Bluetooth signals of other circulation boxes within a preset range through its Bluetooth unit, and collects the cargo type, cargo serial number and cargo batch information of other circulation boxes through encryption logic. The built-in Bluetooth module of each circulation box uses low-power Bluetooth to continuously scan the surrounding environment. During the scanning process, the confidentiality and integrity of data transmission are ensured through secure encryption logic to prevent data from being tampered with or leaked. The box control unit compares the information of other circulation boxes collected within the preset range with the information of the circulation box to determine whether they belong to the same batch of goods. The box control unit has a built-in high-performance microprocessor that can quickly process and analyze the data received from the Bluetooth module. The comparison process includes matching the cargo type, cargo serial number characteristics and batch information. If all the information matches, it can be confirmed that these circulation boxes belong to the same batch of goods. If they belong to the same batch of goods, the box control unit generates a color signal corresponding to the same batch of goods according to the preset color display rules. The preset color display rules are stored in the memory of the box control unit, and a unique color code is specified for each batch according to the category and batch of goods. The box control unit sends this color signal to the color light bar control circuit to light up the corresponding color LED. At the same time, the box control unit associates the circulation boxes of the same batch of goods within the preset range and updates the adjacent goods association set. The updated adjacent goods association set is synchronized to all associated circulation boxes through the Bluetooth module to ensure that the circulation box information of the entire batch is consistent. The box control unit extracts color data from other scanned circulation boxes, and records and analyzes it. Color conflicts are avoided based on the color data of the light color strips that do not belong to the same batch of goods. The box control unit uses the logic of conflict detection to ensure that the color identification of circulation boxes from different batches does not conflict. If a color conflict is detected, the box control unit will automatically adjust the color display of this circulation box and select an unused color to avoid confusion.

[0055] In some specific embodiments, it is assumed that a batch of electronic products is loaded into multiple circulation boxes. Each circulation box continuously scans the circulation box signals around it through the built-in Bluetooth module. Assuming that the A circulation box detects that there is a B circulation box nearby, the system obtains the cargo type "electronic products", the serial number "E12345", and the batch information "Batch001" from the B circulation box through encryption logic. The box control unit of the A circulation box compares this information with the data stored in itself to confirm that the B circulation box and the A circulation box belong to the same batch. After confirmation, the box control unit of the A circulation box generates a color signal indicator light to display green according to the color display rule, and sends this color information to the B circulation box through the Bluetooth module, and updates the adjacent cargo association set at the same time to ensure that the A and B circulation boxes display the same green light bar. If the A circulation box detects the C circulation box, and after comparison, it is found that the cargo type of the C circulation box is "clothing", the serial number is "C67890", and the batch information is "Batch002", it does not belong to the same batch as the A circulation box. At this time, loop box A will record the color display of loop box C (assuming it is blue) and perform color conflict detection. If there is no conflict between green and blue in the current color rule, loop box A will maintain the original green display; if it is detected that other loop boxes also use green and may cause confusion, loop box A will adjust to another unused color (such as yellow) to avoid conflict.

[0056] Furthermore, determining the use status of the circulation box according to the real-time data of the storage circulation box includes the following steps:

[0057] Receive and store real-time data uploaded from each circulation box;

[0058] Compare the current position of the circulation box with the cargo starting point and cargo destination to determine whether the circulation box is located at the starting point, destination or transportation path;

[0059] If the circulation box is at the starting point of the cargo and is not loaded, the marked usage status is idle; if the circulation box is at the starting point of the cargo and is loaded, the marked usage status is in use at the starting point; when the circulation box is on the transportation path, the marked usage status is on the way; if the circulation box arrives at the cargo destination and is not unloaded, the marked usage status is arrived at the destination; if the circulation box arrives at the cargo destination and is unloaded, the marked usage status is idle.

[0060] In some embodiments, first, the real-time data uploaded from each circulation box is received and stored. Each circulation box is equipped with a storage unit and a Bluetooth unit, which can collect and transmit the status data of the circulation box in real time, including the current position, cargo loading status, cargo starting point, cargo destination and other information. The site positioning communication device and the transport vehicle positioning communication device in the circulation box positioning module interact with the Bluetooth unit in the circulation box through wireless communication technology, and transmit these real-time data to the intelligent cloud management module. Then, the intelligent cloud management module compares the current position of the circulation box with the cargo starting point and cargo destination to determine whether the circulation box is located at the starting point, destination or transportation path. The intelligent cloud management module is equipped with a high-performance processor and a large-capacity storage device, which can quickly process and store data from a large number of circulation boxes. By comparing the current position of each circulation box with the preset cargo starting point and destination, the system can determine the specific location of each circulation box at present. If the circulation box is located at the cargo starting point and is not loaded, the intelligent cloud management module marks the use status of the circulation box as idle. In this case, the system will update the status information of the circulation box to the database, indicating that the circulation box is currently in a state of waiting for loading and can be assigned to a new transportation task. For example, a recycling box has just completed a transportation task. After returning to the starting point, the system detects that it is not loaded with any goods, and marks its status as idle. If the recycling box is at the cargo starting point and is loaded, the intelligent cloud management module marks the use status of the recycling box as in use at the starting point. This indicates that the recycling box has been loaded with goods but has not yet started transportation. For example, a recycling box is loaded with new goods at the starting point and is ready to be sent to the destination. The system will update its status to in use at the starting point. When the recycling box is on the transportation path, the intelligent cloud management module marks the use status of the recycling box as on the way. Through GPS and RFID technology, the system can track the transportation location of the recycling box in real time to ensure that the goods are transported according to the planned route. For example, a recycling box is loaded with goods and departs from the starting point. The system detects that it is moving on the transportation path, so it updates its status to on the way. If the recycling box arrives at the cargo destination and is not unloaded, the intelligent cloud management module marks the use status of the recycling box as arrived at the destination. This means that the recycling box has arrived at the scheduled destination site, but the goods have not yet been unloaded. For example, after a recycling box arrives at the destination station, the system confirms its location through positioning data and detects that the goods are still in the box, so it updates the status to arrival at the destination. If the recycling box arrives at the destination and has been unloaded, the intelligent cloud management module marks the use status of the recycling box as idle. At this time, the recycling box has completed the delivery of the goods and can be reassigned to a new transportation task. For example, after a recycling box has unloaded the goods at the destination station, the system detects that the goods have been taken out and updates its status to idle.

[0061] Furthermore, the scheduling of the circulation box based on the reinforcement learning algorithm according to the real-time traffic conditions, station requirements and the use status of the circulation box includes the following steps:

[0062] Collect historical traffic data for each station, including daily recycling box usage and returns;

[0063] Predict the demand for recycling boxes at each station within a preset time in the future based on historical traffic data, and calculate the actual demand at each station based on the future demand and the current inventory of recycling boxes at each station;

[0064] The actual demand of each station, the current position of each cycle box and the real-time traffic conditions between stations are taken as the states, and the number of cycle boxes transported between stations is taken as the action. The reward function is defined based on the degree of satisfying the actual demand of each station and the degree of satisfying the actual demand of each station, and the DQN network is trained.

[0065] Based on the trained DQN network, the number of transport cycle boxes between sites is generated in real time, and the number of idle cycle boxes required for several single deliveries is obtained to perform cycle box scheduling.

[0066] In some embodiments, historical traffic data of each site is collected, including the daily usage and return of recycling boxes. The system integrates the data uploaded by each site through the intelligent cloud management module to form a detailed historical database. These data include the usage of recycling boxes at each site in different time periods, such as daily loading, return, transshipment and other specific operation records. The system cleans and preprocesses the data to ensure the accuracy and completeness of the data. Then, the demand for recycling boxes at each site in the future preset time is predicted based on the historical traffic data. The intelligent cloud management module uses time series analysis and prediction models, such as ARIMA (autoregressive integrated moving average model) or LSTM (long short-term memory network) to model and analyze historical data and predict the demand for recycling boxes at each site in the future. The prediction results can reflect the demand fluctuations of each site in different time periods, helping the system to make scheduling arrangements in advance. According to the future demand and the current inventory of recycling boxes at each site, the actual demand of each site is calculated. The intelligent cloud management module compares the predicted demand with the real-time inventory data to calculate the actual demand of each site. If the predicted demand of a station is higher than the current inventory, the system will mark the station as "high demand"; otherwise, it will be marked as "sufficient inventory" or "low demand". These marking information will serve as an important basis for subsequent scheduling decisions. Taking the actual demand of each station, the current position of each recycling box and the real-time traffic conditions between stations as the state, and the number of recycling boxes transported between stations as the action, the reward function is defined to meet the degree of actual demand of each station and the degree of meeting the actual demand of each station, and train the DQN (deep Q network) model. Specifically, the state includes the actual demand of each station, the real-time position of each recycling box, the loading status and the traffic conditions between stations. The action refers to the number of recycling boxes allocated between different stations. The system defines the reward function, comprehensively considers factors such as time and station demand satisfaction, and ensures that the overall benefit of the system is maximized under the optimal scheduling strategy. Based on the trained DQN network, the system can generate the number of recycling boxes transported between stations in real time, and obtain the number of idle recycling boxes required to be added for several single deliveries, and perform recycling box scheduling. For example, if the system detects that the actual demand at site A is higher than the current inventory, while site B has sufficient inventory, it will generate a scheduling plan through the DQN network to arrange for a certain number of idle recycling boxes to be deployed from site B to site A. At the same time, the system will select the optimal transportation route and time based on real-time traffic conditions to ensure efficient execution of the scheduling plan. In some specific embodiments, it is assumed that in a certain actual logistics network, sites A, B, C, and D need to handle a large number of recycling box transportation tasks every day. The intelligent cloud management module first collects and analyzes the historical traffic data of these sites, and finds that the demand for recycling boxes at site A increases significantly on Mondays and Wednesdays, while site C has a higher demand on Fridays.Based on these historical data, the system predicts the demand for each site in the next week, and calculates the actual demand based on the current inventory situation of each site.

[0067] For example, the system predicts that Station A will need to add 50 recycling boxes on the next Monday, while Stations B and D have sufficient inventory to support Station A. The system simulates different scheduling plans through the DQN network and evaluates the pros and cons of each plan based on the reward function. In the end, the system chose to allocate 30 recycling boxes from Station B and 20 recycling boxes from Station D to Station A. During the scheduling process, the system also takes into account the current traffic conditions and selects the optimal transportation route to ensure that the recycling boxes can arrive at Station A in time.

[0068] Furthermore, the formula of the DQN network is as follows:

[0069] ;

[0070] Among them, s is the current state parameter set, including the actual demand of each station, the current position of each circulation box and the real-time traffic conditions between stations; is the current action parameter set, including the number of transport cycle boxes between stations; is the action parameter set taken under the state parameter set s Q value; is the learning rate; r is the immediate reward of the current action; is the discount factor; is the next state parameter set; is the action parameter set in the next state.

[0071] It should be noted that the action parameter set Includes the number of transported recycling boxes between sites. For example, it is decided to transport 20 idle recycling boxes from site B to site A, and 10 idle recycling boxes from site C to site B. The action parameter set taken under the state parameter set S The Q value indicates taking action in the current state S The expected return that can be obtained. The Q value is calculated and updated through the DQN network. Learning rate The step size for updating the Q value determines the weight distribution of the new and old Q values ​​at each update. The immediate reward r is the feedback given based on the effect of the current action. For example, if the action of transporting from station B to station A meets the needs of station A and the transportation time is short, then the immediate reward r is high. Discount factor Used to calculate the present value of future rewards. The discount factor ranges from 0 to 1, indicating the importance of future rewards. Indicates the new state of the system after the current action is completed, which may include updated demand for each station and the position of the circulation box. Action parameter set in the next state It indicates other possible actions that may be taken in the new state. Through continuous iterative training, the system's DQN network can gradually optimize the scheduling strategy and improve the efficiency and accuracy of the cycle box scheduling. Ultimately, in practical applications, the system can generate the optimal scheduling plan based on real-time data, effectively meet the needs of each site, and ensure the efficient operation of the logistics network.

[0072] Furthermore, the formula of the reward function is as follows:

[0073] ;

[0074] in, is the reward value; and is the weight coefficient; The actual site demand that is met; is the total demand of the site; The actual transportation time; The maximum transport time.

[0075] It should be noted that Specifically, it is the maximum time required for all stations to reach the target quantity. It is a time limit preset by the system to measure transportation efficiency. Its function is to set a benchmark to help the system evaluate the overall pros and cons of scheduling actions by measuring the proportion of actual transportation time relative to the preset maximum time. The smaller the ratio, the shorter the transportation time and the higher the efficiency; conversely, the lower the efficiency. The system evaluates the effect of the current scheduling action based on the reward value R. If the reward value is high, it means that the scheduling plan is better and can continue to be used or referenced; if the reward value is low, it is necessary to adjust the scheduling strategy and select other better plans.

[0076] Furthermore, the real-time traffic conditions between the stations include a plurality of real-time traffic condition values, and the real-time traffic condition values ​​are obtained by the following steps:

[0077] Access the real-time navigation API to obtain estimated transportation time between each station;

[0078] The estimated transport time is graded according to the preset time gradient, and each grade is assigned a value according to the gradient to obtain the real-time traffic status value.

[0079] In some embodiments, the system accesses a real-time navigation API to obtain the estimated transportation time between each site. The navigation API can come from multiple service providers. Through these APIs, the system can obtain the road traffic conditions, estimated transportation time and possible delay information between each site in real time. For example, when the system requests the real-time traffic conditions from site A to site B, the navigation API returns the current estimated transportation time of 45 minutes, including considerations such as traffic congestion. After obtaining the estimated transportation time, the system grades the estimated transportation time according to the preset time gradient. The time gradient can be set according to specific logistics needs and traffic conditions. Common graded gradients include: assigning a value of 1.5 to the first time gradient such as within 1 day, assigning a value of 2 to the second time gradient such as 1 day to 2 days, and so on. These estimated times are graded according to the preset time gradient and assigned corresponding traffic condition values. In the process of circulating box scheduling, these real-time traffic condition values ​​serve as important reference parameters to help the system optimize scheduling decisions, improve logistics efficiency, and ensure the timeliness and reliability of transportation. This embodiment shows how the system uses real-time traffic data to dynamically adjust logistics scheduling, reflecting the professionalism and intelligence of the system in dealing with complex traffic conditions.

[0080] A packaging recycling box management method, applied to any one of the packaging recycling box management systems described above, comprising:

[0081] A positioning communication device is arranged at the site, and a positioning communication device is arranged in the transport vehicle, wherein the positioning communication device is used to perform data exchange with the packaging circulation box;

[0082] The use status of the circulation box is determined based on the real-time data of the storage circulation box obtained from the communication device, and the use status includes idle, in use at the starting point, on the way and at the destination. Based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box.

[0083] The above implementation modes are merely descriptions of the preferred implementation modes of the present invention, and are not intended to limit the scope of the present invention. Without departing from the design spirit of the present invention, various modifications and improvements made to the technical solutions of the present invention by ordinary engineering and technical personnel in the field shall fall within the protection scope determined by the claims of the present invention.

Claims

1. A packaging circulation box management system, characterized in that: It includes a packaging circulation box, a circulation box positioning module, an intelligent cloud management module and an abnormal alarm module, wherein the packaging circulation box, the circulation box positioning module and the intelligent cloud management module are communicatively connected in sequence, and the abnormal alarm module is communicatively connected with the circulation box positioning module; A colored light strip is arranged on the outer edge of the packaging circulation box, and a storage unit, a Bluetooth unit and a box control unit are embedded in the packaging circulation box. The storage unit is used to store real-time data of the circulation box, and the box control unit is used to control the opening and closing of the packaging circulation box, control the color change of the colored light strip and control the Bluetooth unit to send and receive usage data of the circulation box; The circulation box positioning module includes a positioning communication device arranged at the site and a positioning communication device arranged at the transport vehicle, and the positioning communication device is used to perform data interaction with the packaging circulation box and communicate with the intelligent cloud management module; The intelligent cloud management module is used to determine the use status of the circulation box according to the real-time data of the storage circulation box, and the use status includes idle, in use at the starting point, on the way and at the destination, and based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box; The abnormal alarm module is used to determine whether the circulation box is lost according to the real-time data of the circulation box and issue an abnormal alarm; Determining the use status of the circulation box according to the real-time data of the storage circulation box comprises the following steps: Receive and store real-time data uploaded from each circulation box; Compare the current position of the circulation box with the cargo starting point and cargo destination to determine whether the circulation box is located at the starting point, destination or transportation path; If the circulation box is at the cargo starting point and is not loaded, the marked use status is idle; if the circulation box is at the cargo starting point and is loaded, the marked use status is in use at the starting point; when the circulation box is on the transportation path, the marked use status is on the way; if the circulation box arrives at the cargo destination and is not unloaded, the marked use status is arrived at the destination; if the circulation box arrives at the cargo destination and is unloaded, the marked use status is idle; The method of scheduling the circulation box based on the reinforcement learning algorithm according to the real-time traffic conditions, station requirements and the use status of the circulation box includes the following steps: Collect historical traffic data for each station, including daily recycling box usage and returns; Predict the demand for recycling boxes at each station within a preset time in the future based on historical traffic data, and calculate the actual demand at each station based on the future demand and the current inventory of recycling boxes at each station; The actual demand of each station, the current position of each cycle box and the real-time traffic conditions between stations are taken as the states, and the number of cycle boxes transported between stations is taken as the action. The reward function is defined based on the degree of satisfying the actual demand of each station and the degree of satisfying the actual demand of each station, and the DQN network is trained. Based on the trained DQN network, the number of transport loop boxes between sites is generated in real time, and the number of idle loop boxes required for a single delivery is obtained to schedule the loop boxes. The abnormality alarm module detects abnormalities by monitoring the position signal and usage data of the circulation box and utilizing the associated information in the Bluetooth network.

2. A packaging recycling box management system according to claim 1, characterized in that: The real-time data of the circulation box includes cargo loading status, single storage cargo type, cargo serial number, cargo origin and cargo destination, and also includes adjacent cargo association set and light color strip color data.

3. A packaging recycling box management system according to claim 2, characterized in that: The control of the color change of the color light strip comprises the following steps: Each circulation box continuously scans the Bluetooth signals of other circulation boxes within a preset range through its Bluetooth unit, and collects the cargo type, cargo serial number and cargo batch information of other circulation boxes through encryption logic; The box control unit compares the information of other circulation boxes collected within a preset range with the information of the current circulation box to determine whether they belong to the same batch of goods; If they belong to the same batch of goods, the box control unit generates a color signal corresponding to the same batch of goods according to a preset color display rule, associates the circulation boxes of the same batch of goods within a preset range, and updates the adjacent goods association set.

4. A packaging recycling box management system according to claim 1, characterized in that: The formula of the DQN network is as follows: ； Among them, s is the current state parameter set, including the actual demand of each station, the current position of each circulation box and the real-time traffic conditions between stations; is the current action parameter set, including the number of transport cycle boxes between stations; is the action parameter set taken under the state parameter set s Q value; is the learning rate; r is the immediate reward of the current action; is the discount factor; is the next state parameter set; is the action parameter set in the next state.

5. A packaging recycling box management system according to claim 4, characterized in that: The formula of the reward function is as follows: ； in, is the reward value; and is the weight coefficient; The actual site demand that is met; is the total demand of the site; The actual transportation time; The maximum transport time.

6. A packaging recycling box management system according to claim 5, characterized in that: The real-time traffic conditions between the stations include a plurality of real-time traffic condition values, and the real-time traffic condition values ​​are obtained by the following steps: Access the real-time navigation API to obtain estimated transportation time between each station; The estimated transport time is graded according to the preset time gradient, and each grade is assigned a value according to the gradient to obtain the real-time traffic status value.

7. A packaging circulation box management method, applied to a packaging circulation box management system according to any one of claims 1 to 6, characterized in that: include: A positioning communication device is arranged at the site, and a positioning communication device is arranged in the transport vehicle, wherein the positioning communication device is used to perform data exchange with the packaging circulation box; The use status of the circulation box is determined based on the real-time data of the storage circulation box obtained from the communication device, and the use status includes idle, in use at the starting point, on the way and at the destination. Based on the reinforcement learning algorithm, the circulation box is scheduled according to the real-time traffic conditions, site requirements and the use status of the circulation box.

Images