An intelligent parcel warehousing, storage and pick-up error prevention method and management system based on multi-modal information fusion

Abstract

This invention provides an intelligent parcel warehousing, storage, and retrieval error prevention method and management system based on multimodal information fusion, belonging to the field of automated parcel storage and retrieval technology. It solves the technical problems of poor intelligence and lack of error prevention functions in existing systems. The system includes a parcel locker with an exhaust fan at the bottom and two AI cameras at the top. A locker door is hinged to the front of the locker via a spring-loaded hinge, and the door has a double-sided touch panel and an electric lock. A flipping mechanism is located on the inside of the door, with a displacement and placement platform on the flipping mechanism. The double-sided touch panel includes a human-machine interface touchscreen module, a flashing indicator light, and a voice alarm. The mainboard of the double-sided touch panel includes a main control and binding module, a wireless transmission module, an AI processing module, an identity verification module, and a parcel retrieval verification module. This invention can intelligently identify parcels and proactively prevent errors in real-time during retrieval and placement, resulting in efficient, accurate, and secure parcel management.

Description

Technical Field

[0001] This invention belongs to the field of automated parcel storage and retrieval technology, and relates to an intelligent parcel warehousing, storage and retrieval error prevention method based on multimodal information fusion, and in particular an intelligent parcel warehousing, storage and retrieval error prevention management system based on multimodal information fusion. Background Technology

[0002] Currently, the application of express delivery points and smart parcel lockers is widespread. However, traditional parcel management methods have drawbacks. In small delivery points or open-shelf scenarios, parcel entry relies heavily on manual scanning of barcodes using barcode scanners, followed by relying on memory or simple records of storage locations, which is inefficient. When picking up parcels, staff manually search the shelves or customers search for them themselves, which can easily lead to the wrong parcel being picked up due to similar parcel appearances, unclear label information, or human negligence, resulting in customer complaints and property losses.

[0003] Existing improvement solutions mainly fall into two categories: one is based on QR code or RFID / Bluetooth tag technology. However, the former requires attaching an additional tag to each package, increasing costs and operational steps; the latter requires installing dedicated RFID / Bluetooth tags, which are costly and difficult to popularize. The second is simple image recognition technology, which can only achieve waybill OCR recognition or package presence detection, lacking precise dynamic binding between the physical entity of the package and its storage space. It cannot perform real-time verification and proactive error prevention the moment the package is picked up, resulting in insufficient reliability.

[0004] Based on this, we propose an intelligent parcel warehousing, storage, and retrieval error prevention method and management system based on multimodal information fusion. Summary of the Invention

[0005] The purpose of this invention is to address the aforementioned problems in existing technologies by proposing an intelligent parcel warehousing, storage, and retrieval error prevention method and management system based on multimodal information fusion. The technical problem this invention aims to solve is: how to achieve automatic intelligent parcel identification, accurate spatial binding, and real-time proactive error prevention during parcel retrieval and placement, so as to improve the efficiency, accuracy, and security of parcel management.

[0006] The objective of this invention can be achieved through the following technical solutions: A smart parcel receiving, storage, and retrieval error prevention method and management system based on multimodal information fusion includes a parcel locker. The locker has several fixed partitions inside, dividing the interior into several storage compartments. Each partition has an indicator light on its front side. Two AI cameras are located at the top of the locker. A door is hinged to the front of the locker via a spring-loaded hinge. The door has a double-sided touch panel, an electric lock, and several monitoring cameras. The position and number of the monitoring cameras correspond to the storage compartments, and each camera is directly facing its corresponding compartment. The electric lock engages with the locker. The lock is locked. The inside of the cabinet door is equipped with a flipping mechanism, on which a displacement and placement platform is installed. One AI camera faces the top of the flipped and unfolded displacement and placement platform, and the other AI camera faces the front of the express cabinet. The outside of the cabinet door is equipped with a handle. The indicator light, monitoring camera, flipping mechanism, electric lock, displacement and placement platform, and AI camera are all electrically connected to the double-sided touch panel. The double-sided touch panel includes a human-computer interaction touch screen module, a flashing indicator light, and a voice alarm. The main board of the double-sided touch panel contains a main control and binding module, a wireless transmission module, an AI processing module, an identity verification module, and a package retrieval and placement verification module.

[0007] Working principle of the invention: I. Parcel Inbound Process Identity verification and opening: The courier authenticates their identity through the human-computer interaction touch screen module on the double-sided touch panel on the cabinet door, or through facial recognition or QR code scanning using the AI ​​camera facing the front of the cabinet. The verification information is transmitted to the identity verification module. After successful verification, the main control and binding module controls the electric lock to open. The courier pulls the handle to open the cabinet door. After the AI ​​camera above the unfolded displacement platform detects that the cabinet door is open, the main control and binding module controls the flipping mechanism inside the cabinet door to start, moving and flipping the displacement platform to a horizontal position perpendicular to the cabinet door for easy package placement.

[0008] Visual data acquisition: After the courier places the package on the displacement platform, the AI ​​camera on the unfolded platform takes a picture of the package. At this time, the AI ​​camera identifies the package itself, confirms its existence and position, and transmits this information to the AI ​​processing module. The AI ​​processing module controls the flipping mechanism to adjust the height of the displacement platform, and controls the displacement platform to adjust the horizontal position of the package, so that the waybill on the package is facing the AI ​​camera. The AI ​​camera collects, but is not limited to, text information on the waybill, the shape, color, and key features of the package. Package storage: The courier places the package into any empty storage compartment. The courier confirms the storage is complete via the human-machine interface touchscreen module on the double-sided touch panel. Then, the displacement and placement platform and the flipping mechanism reset sequentially. The courier closes the cabinet door, and the electric lock automatically engages. The monitoring camera facing the corresponding storage compartment monitors whether a package is stored in that compartment and transmits the package information to the AI ​​processing module. The AI ​​processing module then transmits the information to the main control and binding module. The main control and binding module binds the identified package information, recipient information, and the location information of the storage compartment and stores them in the database.

[0009] Sending pickup information: The main control and binding module extracts the stored information and transmits it to the wireless transmission module. The wireless transmission module sends the pickup information to the client, the system updates the status, and the warehousing is completed.

[0010] II. User pickup process Identity verification and request: The recipient can verify their identity through the human-computer interaction touch screen module on the double-sided touch panel on the cabinet door or through facial recognition via the AI ​​camera facing the front of the express cabinet. The verification information is transmitted to the identity verification module. After successful verification, the electric lock opens, the courier pulls the handle to open the cabinet door, and at the same time, the flipping mechanism on the inside of the cabinet door is activated, moving and flipping the displacement platform to a horizontal position perpendicular to the cabinet door for easy placement of the package.

[0011] Package location and retrieval: After the identity verification module confirms the identity, the main control and binding modules locate the package corresponding to the recipient and its storage compartment based on the binding relationship, and the indicator light of the storage compartment lights up.

[0012] Error prevention verification: When the package verification module is activated, the package is placed on the displacement platform, and the AI ​​camera facing the unfolded displacement platform takes pictures of the package in real time.

[0013] The real-time image is compared with the package image features bound at the time of entry into the warehouse.

[0014] The verification logic ensures that the package to be retrieved matches the system record, preventing accidental retrieval.

[0015] If the wrong item is selected, the item selection and placement verification module will control the indicator light and flashing indicator light to flash, and the voice alarm will broadcast the incorrect placement information.

[0016] Retrieval and Delivery: After verification, the recipient takes the package and confirms the completion through the human-machine interaction touch screen module of the double-sided touch panel. Then, the displacement and placement platform and the flipping mechanism are reset in sequence, the recipient closes the cabinet door, and the electric lock automatically locks.

[0017] Sending completion information: The main control and binding module extracts and transmits the stored information to the wireless transmission module. The wireless transmission module then sends the information to the sender, courier, and client. The system updates the status to idle, and the package retrieval is complete.

[0018] III. Auxiliary and Error Prevention Mechanisms Human-computer interaction: The entire process is guided by the screen of the dual-sided touch panel, indicator lights and voice, providing step-by-step guidance, status prompts and abnormal alarms.

[0019] Multiple verification: Integrating identity information, visual information, and physical perception information to form the basis for error prevention through multimodal information fusion.

[0020] The lower four corners of the express cabinet are all fixed with support feet, and a lock hole is opened on the inner left side wall of the express cabinet, the position of which corresponds to the locking pin of the electric lock.

[0021] The above structure raises the cabinet with the support legs, which also helps prevent moisture. The position of the keyhole corresponds to the locking pin of the electric lock, which is used to lock the cabinet door.

[0022] The cabinet door has a reinforced cavity plate in the shape of a square on its inner periphery. A double-sided touch panel is located in the middle of the reinforced cavity plate. Magnets are fixed on the inner side of the reinforced cavity plate and are symmetrically distributed vertically. An electric lock is located inside the reinforced cavity plate. Two clearance holes are provided on the inner wall of the reinforced cavity plate.

[0023] Using the above structure, the reinforced cavity plate is a reinforcing frame embedded within the cabinet door, enhancing the overall structural strength and rigidity of the cabinet door. When the user closes the cabinet door to a near-closed position, the symmetrically distributed magnets attract the cabinet body, ensuring a stable closure between the door and the cabinet body. This guarantees that the electric lock's locking pin can accurately align and insert into the lock hole on the cabinet body. The clearance hole is a channel specifically reserved for the moving parts of the flipping mechanism.

[0024] The flipping mechanism includes two fixed bases arranged symmetrically on the left and right. The fixed bases are fixed to the inside of the cabinet door, and both fixed bases extend out of the clearance hole on the same side. Each fixed base is provided with a flipping component, and a flipping mounting rod is provided between the two flipping components. The flip assembly includes a guide plate fixed to a fixed base on the same side. The guide plate has an L-shaped guide hole. Two fixed plates symmetrically arranged vertically are fixed to the inner side of the guide plate. A flip slide rail is fixed between the two fixed plates. A flip slide block is slidably mounted on the flip slide rail. A flip electric push rod is fixed to the upper fixed plate. The telescopic end of the flip electric push rod is fixedly connected to the flip slide block. A flip bearing seat is fixed to the flip slide block. A flip shaft is fixed inside the inner ring of the flip bearing seat. A flip connecting rod is fixed to the end of the flip shaft. The flip connecting rod is perpendicular to the axis of the flip shaft. A guide roller is rotatably mounted at the end of the flip connecting rod. The guide roller is rolled inside the L-shaped guide hole. Limiters are fixed to both fixed plates. The limiters on the two fixed plates face each other and are located on the upper and lower sides of the flip bearing seat. The flip mounting rod is C-shaped, and its two ends are fixed to the ends of the flip shafts of the two flip assemblies, respectively. The limiters and the flip electric push rod are electrically connected to the double-sided touch panel.

[0025] With the above structure, the fixed base provides a stable mounting point. The fixed base extends out of the clearance hole on the same side to ensure the stable operation of the flipping component. When it is necessary to flip and move the platform, the flipping electric push rod receives the command and directly drives the flipping slide fixed to it to make vertical linear movement on the flipping slide rail.

[0026] The linear motion of the flip slide is transmitted through the flip bearing seat and the flip shaft to the flip linkage that is fixed perpendicularly to the flip shaft. The guide roller at the end of the flip linkage is restricted to rolling within the L-shaped guide hole.

[0027] First stage (start / retraction): When the flip slide is at one end of the stroke and the guide roller is at the end of the "short side" of the L-shaped guide hole, the flip mounting rod is parallel to the cabinet door (the displacement support platform retracts).

[0028] The second stage (flipping process): The flipping electric push rod pushes the flipping slide to move downward. Due to the constraint of the L-shaped hole "corner" and "long side" on the guide roller, it forces the flipping linkage not only to follow the downward slide, but also to rotate around the flipping axis.

[0029] This constraint forces the vertical linear motion of the flipping slide to be converted into the rotational motion of the flipping shaft (and the C-shaped flipping mounting rod fixed thereto).

[0030] The third stage (in place): When the flip slide moves to the other end of the stroke (the lowest end), the guide roller reaches the other end of the "long side" of the L-shaped hole. At this time, the flip shaft rotates exactly 90 degrees, driving the flip mounting rod and the displacement support platform to be in a state perpendicular to the cabinet door (the platform unfolds).

[0031] The symmetrically arranged limit switches correspond to the extreme positions of the vertical movement of the rotating slide. When the slide touches the limit switch, the system senses that the rotating action has reached the correct endpoint (degree or degree) and immediately stops the rotating electric push rod to ensure accurate rotating angle and prevent mechanical overtravel, thus protecting the equipment safety.

[0032] The displacement platform includes a platform base plate, a Y-axis plate, and an X-axis plate. The platform base plate, Y-axis plate, and X-axis plate have equal widths, and their lengths decrease sequentially. One end of the platform base plate is fixed to the upper end of a flip-mounting rod. T-shaped guide rails are provided at the upper center of both the platform base plate and the Y-axis plate, and these guide rails are perpendicular to each other. The Y-axis plate slides on the T-shaped guide rails of the platform base plate, and the X-axis plate slides on the T-shaped guide rails of the Y-axis plate. Two symmetrically arranged Y-axis mounting pivots are fixed to the other end of the platform base plate. A Y-axis lead screw rotates between the two Y-axis mounting pivots. One of the Y-axis mounting pivots is fixed with... There is a Y-axis motor, the output shaft of which is fixedly connected to the Y-axis lead screw. A Y-axis shifter is fixed at the middle of the side of the Y-axis plate, and the Y-axis shifter is connected to the Y-axis lead screw in a transmission engagement. Two symmetrically arranged X-axis mounting seats are fixed on the side of the Y-axis plate, and an X-axis lead screw is rotatably mounted between the two X-axis mounting seats. An X-axis motor is fixed on one of the X-axis mounting seats, and the output shaft of the X-axis motor is fixedly connected to the X-axis lead screw. An X-axis shifter is fixed at the middle of the side of the X-axis plate, and the X-axis shifter is connected to the X-axis lead screw in a transmission engagement. When the displacement platform is placed horizontally, one of the AI ​​cameras is positioned directly opposite the middle of the X-axis plate. Both the Y-axis motor and the X-axis motor are electrically connected to the double-sided touch panel.

[0033] The above structure consists of: platform base plate (bottom layer, fixed on the flip mounting rod) → Y-axis plate (middle layer) → X-axis plate (top layer, directly supporting the package).

[0034] Motion transmission path: The platform base plate is fixed; the Y-axis motor drives the Y-axis plate to move back and forth (defined as the Y-axis) along the T-shaped rail on the platform base plate; the X-axis motor drives the X-axis plate to move left and right (defined as the X-axis) along the T-shaped rail on the Y-axis plate. The two motions are orthogonal and do not interfere with each other.

[0035] Y-axis motion: The rotation of the Y-axis motor drives the rotation of the Y-axis lead screw. The Y-axis shifter (fixed on the Y-axis plate) that meshes with the Y-axis lead screw converts the rotational motion of the lead screw into the accurate linear movement of the Y-axis plate.

[0036] X-axis motion: Similarly, the rotation of the X-axis motor drives the rotation of the X-axis lead screw, which in turn drives the X-axis shifter (fixed on the X-axis plate) that meshes with the lead screw, thereby achieving accurate linear movement of the X-axis plate.

[0037] The T-shaped guide rail plays a crucial supporting and guiding role in the movement of each layer, ensuring smooth and stable movement without shaking and strong load-bearing capacity.

[0038] Synergy with the core functions of the system: When the package is placed on the X-axis plate, the AI ​​camera above will take a picture. The AI ​​processing module analyzes the image, and if the label is not in the correct position or is not in the center of the field of view, it will generate control commands.

[0039] The command is sent to the displacement platform, and the Y-axis motor and X-axis motor work together to drive the X-axis plate (along with the package) to move in the horizontal plane until the label area on the package is accurately aligned with the center of the AI ​​camera's optimal field of view. Improving the success rate and accuracy of label recognition is the foundation for ensuring the reliability of key processes such as subsequent information binding and package verification.

[0040] The main control and binding module serves as the central processing and coordination hub of the system. It includes an information binding and storage unit and a system process control unit. The information binding and storage unit receives package information from the AI ​​processing module and physical location identifiers from the spatial perception module, uniquely binds the three, and stores them in the database. The system process control unit acts as the general commander, coordinating and controlling the start, stop, and sequence of all sub-modules such as identity verification, flipping mechanism, displacement platform, AI recognition, and verification according to preset logic, driving the complete warehousing and retrieval process. The wireless transmission module is responsible for data communication between the system and the external network, and remote data exchange: after the package is put into storage, the pickup information is sent to the recipient's client; after the pickup is completed, the status completion information is sent to the sender, the courier, and the client to achieve information synchronization. The AI ​​processing module is responsible for intelligent analysis of image information, including a package detection unit and a waybill recognition unit. The package detection unit runs a first AI model to identify the outline and location of the package from the image and confirm its existence. The waybill recognition unit runs a second AI model to accurately locate the waybill area from the package image and extract key text information such as the recipient's mobile phone number, address, and waybill number.

[0041] The identity verification module is responsible for verifying the legitimacy of the operator's identity. It includes a front-end interaction unit and a back-end verification unit. The front-end interaction unit collects identity information by receiving the operator's employee number, pickup code, and mobile phone number through the touch screen of the double-sided touch panel, keyboard, or front-facing AI camera, or by collecting their facial features and QR code information. The back-end verification unit judges the legitimacy of the identity by comparing the collected information with the reserved information in the database to verify the identity of the courier or recipient and decide whether to authorize the opening of the cabinet door.

[0042] The package retrieval and verification module is responsible for real-time verification during the storage and retrieval process. It includes a real-time image analysis unit and a comparison and logic judgment unit. When storing or retrieving a package, the real-time image analysis unit controls the AI ​​camera to take real-time photos of the package on the platform. The logic judgment unit performs error prevention comparison and alarm, comparing the features of the real-time image with the original records bound in the main control module. If they do not match, an audible and visual alarm is immediately triggered to stop the current process and prevent errors from occurring.

[0043] A method for preventing errors in intelligent parcel warehousing, storage, and retrieval based on multimodal information fusion includes the following steps: Step 1: Package Inbound Binding Process Step 1.1, Operator Authentication and System Startup: The courier enters their employee ID / password through the cabinet door interactive screen, or performs facial recognition / scanning through the front-facing AI camera; the authentication module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, the courier pulls the handle to open the cabinet door; after opening the cabinet door, the flipping mechanism is activated, flipping the displacement receiving platform horizontally to a vertical position outside the cabinet door, ready to receive the package; Step 1.2, Intelligent Collection and Active Positioning of Package Visual Information: The courier places the package on the displacement platform; visual collection: an AI camera located above the platform photographs the package; AI recognition and feedback control: the AI ​​processing module runs in parallel, the package detection unit identifies the package itself, confirms its presence and approximate position, and the waybill recognition unit attempts to read the text on the waybill; active focusing: if the waybill is not in the optimal recognition position, the AI ​​processing module sends instructions to the main control module, controlling the X-axis and Y-axis motors of the displacement platform to coordinate their movements, fine-tuning the horizontal position of the package until the waybill is stably located in the center of the camera's field of view; information extraction: in the optimal position, the camera captures high-definition images, and the waybill recognition unit accurately extracts key text information such as the waybill number and the recipient's mobile phone number; Step 1.3, Package Storage: The courier will pick up the package from the displacement platform and place it into any empty storage compartment. The courier will confirm the storage through the human-machine interface touchscreen module on the double-sided touch panel. Then, the displacement platform and the flipping mechanism will reset in sequence. The courier will close the cabinet door, and the electric lock will automatically lock. The monitoring camera facing the corresponding storage compartment will monitor whether a package is stored in that compartment and transmit the package information to the AI ​​processing module. The AI ​​processing module will then transmit the information to the main control and binding module. The main control and binding module will strongly associate the package / recipient information extracted in Step 1.2 with the storage compartment information generated in Step 1.3 to generate an entry record and store it in the database. Step 1.4: Send pickup information: The main control and binding module extracts the stored information and transmits it to the wireless transmission module. The wireless transmission module sends the pickup information to the client, the system updates the status, and the warehousing is completed. Step 2: Error Prevention Process for User Package Pickup: Step 2.1, Recipient Identity Verification and System Preparation: The recipient enters the pickup code / mobile phone number through the human-computer interaction touch screen module of the double-sided touch panel, or performs facial recognition through the front AI camera; the identity verification module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, and the courier pulls the handle to open the cabinet door; at the same time, the flipping mechanism is activated to flip the displacement and placement platform to a convenient handover position; Step 2.2, Package Location Guidance: The main control module queries the binding record in the database based on the recipient's identity information, finds the corresponding package and its storage compartment, and lights up the indicator light of the storage compartment; Step 2.3: Retrieve the package and place it on the verification platform: The recipient retrieves the package from the corresponding storage compartment according to the indicator light and places it on the extended displacement platform; the package retrieval and placement verification module controls the AI ​​camera above the platform to take real-time photos of the package on the platform; the verification module quickly compares the real-time image features with the original image features / waybill information stored when the package was bound to the warehouse; the error prevention logic judges and executes the verification logic, the core of which is to determine whether the "package on the current platform" is consistent with the information "the package that should be picked up by the current recipient in the system record"; Step 2.4, Verification Result Decision and Execution: Scenario 1, Verification Successful: The human-machine interaction module prompts "Verification successful, please pick up the package"; the recipient picks up the package from the platform and confirms completion on the touch screen; subsequent actions include the main control module resetting the control mechanism, the cabinet door closing and locking, the wireless transmission module sending a pickup completion notification to the shipping end, the express delivery end, and the client, and the database updating the inventory status. Scenario 2, verification fails: Upon system execution, the item pick-up and drop-off verification module immediately triggers an audible and visual alarm and displays an error warning on the human-machine interface screen; the system locks the current process, preventing further normal operation, and requires administrator intervention.

[0044] Compared with existing technologies, this intelligent parcel warehousing, storage, and retrieval error prevention method and management system based on multimodal information fusion has the following advantages: This system integrates a sophisticated flipping mechanism and a displacement platform to automate the entire process of parcel storage and retrieval, creating a deeply integrated system of "hardware execution and software decision-making": using an AI vision module for multimodal information recognition and binding, spatial perception and mechanical mechanisms for accurate positioning and operation, and an environmental management system to ensure equipment reliability, forming an autonomous and error-proof intelligent storage and retrieval terminal.

[0045] This method is based on a closed-loop control logic of multimodal information fusion. It strongly binds visual information at the time of entry into the warehouse with physical space and performs real-time visual reverse verification at key handover points when retrieving the package. This dynamically links personnel identity, package characteristics and storage location, forming a proactive error prevention chain that runs through the entire storage and retrieval process, improving the accuracy and security of operations from the source of process design. Attached Figure Description

[0046] Figure 1 This is a three-dimensional structural diagram of the express delivery locker in the management system of the present invention when the door is opened.

[0047] Figure 2 This is a schematic diagram of the express delivery locker in the management system of the present invention from another perspective.

[0048] Figure 3 This is a three-dimensional structural diagram of the delivery platform of the express cabinet in the management system of the present invention when it is unfolded.

[0049] Figure 4 This is a schematic diagram of the structure of the express delivery locker's receiving platform in the management system of the present invention when it is unfolded from another perspective.

[0050] Figure 5 This is a three-dimensional structural diagram of the flipping mechanism in this invention.

[0051] Figure 6 This is a three-dimensional structural diagram of the flipping component in this invention.

[0052] Figure 7 This is a schematic diagram of the displacement support platform in this invention.

[0053] Figure 8 This is a flowchart of the method of the present invention.

[0054] In the diagram: 1. Cabinet body; 2. Support legs; 3. Monitoring camera; 4. Shelf; 5. Indicator light; 6. Cabinet door; 7. Reinforced cavity plate; 8. Magnet; 9. Flipping mechanism; 10. Electric lock; 11. Displacement platform; 12. Double-sided touch panel; 13. AI camera; 14. Handle; 15. Fixed base; 16. Flipping assembly; 17. Flipping mounting rod; 18. Guide plate; 19. L-shaped guide hole; 20. Guide roller; 21. Flipping connector. 21. Rod; 22. Flip bearing seat; 23. Fixing plate; 24. Flip shaft; 25. Flip slide; 26. Flip slide rail; 27. Limiter; 28. Flip electric push rod; 29. ​​Platform base plate; 30. Y-axis plate; 31. Y-axis lead screw; 32. Y-axis moving seat; 33. Y-axis motor; 34. X-axis plate; 35. T-shaped guide rail; 36. X-axis mounting rotary seat; 37. X-axis lead screw; 38. X-axis moving seat; 39. X-axis motor; 40. Y-axis mounting rotary seat. Detailed Implementation

[0055] The following are specific embodiments of the present invention, which are described in conjunction with the accompanying drawings. However, the present invention is not limited to these embodiments.

[0056] like Figures 1-7 As shown, this intelligent parcel warehousing, storage, and error-proof management system based on multimodal information fusion includes a parcel locker 1. The interior of the parcel locker 1 is fixed with several partitions 4, dividing the interior into several storage compartments. Each partition 4 has an indicator light 5 on its front side. Two AI cameras 13 are located at the top of the parcel locker 1. A locker door 6 is hinged to the front of the parcel locker 1 via a spring-loaded hinge. The locker door 6 has a double-sided touch panel 12, an electric lock 10, and several monitoring cameras 3. The position and number of the monitoring cameras 3 correspond to the storage compartments, and each monitoring camera 3 is directly facing its corresponding storage compartment. The electric lock 10 engages and locks the parcel locker 1. The inner side of the cabinet 6 is provided with a flipping mechanism 9, and a displacement holding platform 11 is provided on the flipping mechanism 9. One AI camera 13 is directly above the flipped and unfolded displacement holding platform 11, and the other AI camera 13 is directly in front of the express cabinet 1. The outer side of the cabinet door 6 is provided with a handle 14. The indicator light 5, monitoring camera 3, flipping mechanism 9, electric lock 10, displacement holding platform 11 and AI camera 13 are all electrically connected to the double-sided touch panel 12. The double-sided touch panel 12 includes a human-computer interaction touch screen module, a flashing indicator light and a voice alarm. The main board of the double-sided touch panel 12 is provided with a main control and binding module, a wireless transmission module, an AI processing module, an identity verification module and a pick-up and put-down verification module.

[0057] I. Parcel Inbound Process Identity verification and opening: The courier authenticates his identity (such as by entering his employee number) through the human-machine interaction touch screen module of the double-sided touch panel 12 on the cabinet door 6, or by facial recognition or scanning a code through the AI ​​camera 13 facing the front of the cabinet body 1. The identification and verification information is transmitted to the identity verification module. After the identity verification module successfully verifies the identity verification, the main control and binding module controls the electric lock 10 to open. The courier pulls the handle 14 to open the cabinet door 6. After the AI ​​camera 13 above the flipped and unfolded displacement receiving platform 11 detects that the cabinet door 6 is open, the main control and binding module controls the flipping mechanism 9 inside the cabinet door 6 to start, moving and flipping the displacement receiving platform 11 to a position that is horizontal and perpendicular to the cabinet door 6, so as to facilitate the placement of packages.

[0058] Visual acquisition: After the courier places the package on the displacement platform 11, the AI ​​camera 13, which is directly facing the unfolded displacement platform 11, takes a picture of the package. At this time, the AI ​​camera 13 identifies the package itself, confirms its existence and position, and transmits this information to the AI ​​processing module. The AI ​​processing module controls the flipping mechanism 9 to adjust the height of the displacement platform 11 and controls the displacement platform 11 to adjust the horizontal position of the package, so that the waybill on the package is directly facing the AI ​​camera 13. The AI ​​camera 13 collects, but is not limited to, text information on the waybill, the shape, color, and key features of the package. Package storage: The courier places the package into any empty storage compartment. The courier confirms the storage is complete via the human-machine interface touchscreen module of the double-sided touch panel 12. Then, the displacement platform 11 and the flipping mechanism 9 are reset in sequence. The courier closes the cabinet door 6, and the electric lock 10 automatically locks. The monitoring camera 3, which is positioned opposite the corresponding storage compartment, monitors whether a package is stored in that compartment and transmits the package information to the AI ​​processing module. The AI ​​processing module transmits the information to the main control and binding module. The main control and binding module binds the identified package information, recipient information, and the location information of the storage compartment and stores them in the database.

[0059] Sending pickup information: The main control and binding module extracts the stored information and transmits it to the wireless transmission module. The wireless transmission module sends the pickup information to the client, the system updates the status, and the warehousing is completed.

[0060] II. User pickup process Identity verification and request: The recipient can verify their identity (such as by entering a pickup code, mobile phone number, ID card number, etc.) through the human-computer interaction touch screen module of the double-sided touch panel 12 on the cabinet door 6, or by facial recognition through the AI ​​camera 13 facing the front of the express cabinet 1. The verification information is transmitted to the identity verification module. After the identity verification module verifies the identity, the electric lock 10 is opened, and the courier pulls the handle 14 to open the cabinet door 6. At the same time, the flipping mechanism 9 on the inside of the cabinet door 6 is activated, moving and flipping the displacement and placement platform 11 to a horizontal position perpendicular to the cabinet door 6 for easy placement of the package.

[0061] Package location and retrieval: After the identity verification module confirms the identity, the main control and binding modules locate the package corresponding to the recipient and its storage compartment based on the binding relationship, and the indicator light 5 of the storage compartment lights up.

[0062] Error prevention verification: When the package verification module is activated, the package is placed on the displacement platform 11. The AI ​​camera 13, which is directly opposite the flipped and unfolded displacement platform 11, takes pictures of the package and acquires images in real time.

[0063] The real-time image is compared with the package image features (or waybill information) bound at the time of entry into the warehouse.

[0064] The verification logic ensures that the package to be retrieved matches the system record, preventing accidental retrieval (e.g., due to human error in placing the package in the wrong compartment or system misjudgment).

[0065] If the wrong item is selected, the item selection and placement verification module will control the display light 5 and the flashing indicator light to flash, and the voice alarm will broadcast the incorrect placement information.

[0066] Retrieval and Delivery: After verification, the recipient takes the package and confirms the completion through the human-machine interaction touch screen module of the double-sided touch panel 12. Then, the displacement and placement platform 11 and the flipping mechanism 9 are reset in sequence, the recipient closes the cabinet door 6, and the electric lock 10 locks automatically.

[0067] Sending completion information: The main control and binding module extracts and transmits the stored information to the wireless transmission module. The wireless transmission module then sends the information to the sender, courier, and client. The system updates the status to idle, and the package retrieval is complete.

[0068] III. Auxiliary and Error Prevention Mechanisms Human-computer interaction: The entire process is guided by the screen, indicator lights and voice of the dual-sided touch panel 12, providing step-by-step guidance, status prompts and abnormal alarms (such as the risk of picking up the wrong item).

[0069] Multiple verifications: Integrating identity information (the recipient), visual information (AI image recognition), and physical perception information (monitoring camera 3) forms the basis for error prevention through multimodal information fusion.

[0070] Each of the four corners of the lower end of the express cabinet 1 is fixed with a support foot 2. A lock hole is opened on the left side wall inside the express cabinet 1, and the position of the lock hole corresponds to the locking pin of the electric lock 10.

[0071] The cabinet is raised by the support feet 2, which also helps prevent moisture. The position of the lock hole corresponds to the locking pin of the electric lock 10, and is used to lock the cabinet door 6.

[0072] The inner periphery of the cabinet door 6 is provided with a U-shaped reinforced cavity plate 7. The double-sided touch panel 12 is located in the middle of the reinforced cavity plate 7. Magnets 8 are symmetrically distributed vertically on the inner side of the reinforced cavity plate 7. The electric lock 10 is located inside the reinforced cavity plate 7. Two left-right symmetrical clearance holes are opened on the inner wall of the reinforced cavity plate 7.

[0073] The reinforced cavity plate 7 is a reinforcing frame embedded in the cabinet door 6, which enhances the overall structural strength and rigidity of the cabinet door 6. When the user closes the cabinet door 6 to a near-closed position, the symmetrically distributed magnets 8 attract the express cabinet body 1, so that the cabinet door 6 and the express cabinet body 1 are stably closed, ensuring that the locking pin of the electric lock 10 can be accurately aligned and inserted into the lock hole on the cabinet body. The clearance hole is a channel reserved for the components of the flipping mechanism 9.

[0074] The flipping mechanism 9 includes two fixed bases 15 arranged symmetrically on the left and right. The fixed bases 15 are fixed to the inside of the cabinet door 6, and both fixed bases 15 extend out of the clearance hole on the same side. Each fixed base 15 is provided with a flipping component 16, and a flipping mounting rod 17 is provided between the two flipping components 16. The flipping assembly 16 includes a guide plate 18, which is fixed to a fixed base 15 on the same side. The guide plate 18 has an L-shaped guide hole 19. Two symmetrically arranged fixed plates 23 are fixed to the inner sides of the guide plate 18. A flipping slide rail 26 is fixed between the two fixed plates 23. A flipping slide block 25 is slidably mounted on the flipping slide rail 26. A flipping electric push rod 28 is fixed to the upper fixed plate 23. The telescopic end of the flipping electric push rod 28 is fixedly connected to the flipping slide block 25. A flipping bearing seat 22 is fixed to the flipping slide block 25. A flipping shaft 24 is fixed inside the inner ring of the flipping bearing seat 22. A flipping link 21 is fixed to the end of the shaft 24. The flipping link 21 is perpendicular to the axis of the flipping shaft 24. A guide roller 20 is rotatably provided at the end of the flipping link 21. The guide roller 20 is rolled inside the L-shaped guide hole 19. Limiters 27 are fixed on both fixed plates 23. The limiters 27 on the two fixed plates 23 face each other and are located on the upper and lower sides of the flipping bearing seat 22. The flipping mounting rod 17 is C-shaped, and the two ends of the flipping mounting rod 17 are respectively fixed to the ends of the flipping shaft 24 of the two flipping components 21. The limiters 27 and the flipping electric push rod 28 are electrically connected to the double-sided touch panel 12.

[0075] The fixed base 15 provides a stable mounting point. All fixed bases 15 extend out of the clearance holes on the same side to ensure that the flipping assembly 16 works stably. When it is necessary to flip and displace the platform 11, the flipping electric push rod 28 receives the command and directly drives the flipping slide 25 fixed to it to make vertical linear movement on the flipping slide rail 26.

[0076] The linear motion of the flip slide 25 is transmitted to the flip link 21, which is fixed perpendicularly to the flip shaft 24, through the flip bearing seat 22 and the flip shaft 24 on it. The guide roller 20 at the end of the flip link 21 is restricted to rolling within the L-shaped guide hole 19.

[0077] First stage (start / retraction): When the flip slide 25 is at one end of the stroke (such as the top end), the guide roller 20 is at the end of the "short side" of the L-shaped guide hole. At this time, the flip mounting rod 17 is parallel to the cabinet door 6 (the displacement support platform 11 is retracted).

[0078] Second stage (flipping process): The flipping electric push rod 28 pushes the flipping slide 25 downward. Due to the constraint of the L-shaped hole "corner" and "long side" on the guide roller 20, it forces the flipping linkage 21 not only to follow the downward slide, but also to rotate around the flipping axis 24.

[0079] This constraint forces the vertical linear motion of the flip slide 25 into the rotational motion of the flip shaft 24 (and the C-shaped flip mounting rod 17 fixed thereto).

[0080] Third stage (in place): When the flip slide 25 moves to the other end of the stroke (lowest end), the guide roller 20 reaches the other end of the "long side" of the L-shaped hole. At this time, the flip shaft 24 rotates exactly 90 degrees, driving the flip mounting rod 17 and the displacement support platform 11 to be in a state perpendicular to the cabinet door 6 (platform unfolds).

[0081] The symmetrically arranged limiters 27 correspond to the extreme positions of the vertical movement of the flip slide 25. When the slide touches the limiter, the system senses that the flipping action has reached the correct endpoint (0 degrees or 90 degrees) and then stops the flipping electric push rod 28 to ensure accurate flipping angle and prevent mechanical overtravel, thus protecting the equipment safety.

[0082] The displacement platform 11 includes a platform base plate 29, a Y-axis plate 30, and an X-axis plate 34. The platform base plate 29, Y-axis plate 30, and X-axis plate 34 have equal widths and their lengths decrease sequentially. One end of the platform base plate 29 is fixed to the upper end of a flip-mounting rod 17. T-shaped guide rails 35 are provided at the upper center of both the platform base plate 29 and the Y-axis plate 30, and these guide rails are perpendicular to each other. The Y-axis plate 30 is slidably mounted on the T-shaped guide rail 35 of the platform base plate 29, and the X-axis plate 34 is slidably mounted on the T-shaped guide rail 35 of the Y-axis plate 30. Two symmetrically arranged Y-axis mounting pivots 40 are fixed to the other end of the platform base plate 29. A Y-axis lead screw 31 rotatably connects the two Y-axis mounting pivots 40. One of the Y-axis mounting pivots 40... A Y-axis motor 33 is fixedly mounted on the Y-axis plate 30. The output shaft of the Y-axis motor 33 is fixedly connected to the Y-axis lead screw 31. A Y-axis shifter 32 is fixedly mounted on the middle of the side of the Y-axis plate 30. The Y-axis shifter 32 is connected to the Y-axis lead screw 31 in a transmission engagement. Two symmetrically arranged X-axis mounting rotary seats 36 are fixedly mounted on the side of the Y-axis plate 30. An X-axis lead screw 37 is rotatably mounted between the two X-axis mounting rotary seats 36. An X-axis motor 39 is fixedly mounted on one of the X-axis mounting rotary seats 36. The output shaft of the X-axis motor 39 is fixedly connected to the X-axis lead screw 37. An X-axis shifter 38 is fixedly mounted on the middle of the side of the X-axis plate 34. The X-axis shifter 38 is connected to the X-axis lead screw 37 in a transmission engagement. When the displacement support platform 11 is placed horizontally, one of the AI ​​cameras 13 is directly opposite the middle of the X-axis plate 34. Both the Y-axis motor 33 and the X-axis motor 39 are electrically connected to the double-sided touch panel 12.

[0083] Platform substrate 29 (bottom layer, fixed on the flip mounting rod) → Y-axis plate 30 (middle layer) → X-axis plate 34 (top layer, directly supporting the package).

[0084] Motion transmission path: Platform base plate 29 is fixed; Y-axis motor 33 drives Y-axis plate 30 to move back and forth (defined as Y-axis) along the T-shaped rail on platform base plate 29; X-axis motor 39 drives X-axis plate 34 to move left and right (defined as X-axis) along the T-shaped rail on Y-axis plate 30. The two motions are orthogonal and do not interfere with each other.

[0085] Y-axis motion: The rotation of the Y-axis motor 33 drives the Y-axis lead screw 31 to rotate. The Y-axis shifter 32 (fixed on the Y-axis plate 30) that engages with the Y-axis lead screw 31 through a threaded connection converts the rotational motion of the lead screw into the accurate linear movement of the Y-axis plate 30.

[0086] X-axis motion: Similarly, the rotation of the X-axis motor 39 drives the X-axis lead screw 37 to rotate, which in turn drives the X-axis shifter 38 (fixed on the X-axis plate 34) that meshes with the lead screw, thereby achieving accurate linear movement of the X-axis plate 34.

[0087] The T-shaped guide rail 35 plays a key supporting and guiding role in the movement of each layer, ensuring smooth and stable movement without shaking and strong load-bearing capacity.

[0088] Synergy with the core functions of the system: When the package is placed on the X-axis plate 34, the AI ​​camera 13 above will take a picture. The AI ​​processing module analyzes the image, and if the label is not in the correct position or is not in the center of the field of view, it will generate control commands.

[0089] The instruction is sent to the displacement platform, and the Y-axis motor 33 and X-axis motor 39 work together to drive the X-axis plate 34 (along with the package) to move in the horizontal plane until the label area on the package is accurately aligned with the center of the best field of view of the AI ​​camera 13; improving the success rate and accuracy of label recognition is the foundation for ensuring the reliability of key processes such as subsequent information binding and package verification.

[0090] The main control and binding module serves as the central processing and coordination hub of the system. It includes an information binding and storage unit and a system process control unit. The information binding and storage unit receives package information (waybill, features) from the AI ​​processing module and physical location identifiers (storage compartment information) from the spatial perception module. It uniquely binds the three (package, recipient, and storage compartment) and stores them in the database. The system process control unit acts as the general commander, coordinating and controlling the start, stop, and sequence of all sub-modules such as identity verification, flipping mechanism, displacement platform, AI recognition, and verification according to preset logic, driving the complete warehousing and retrieval process.

[0091] The wireless transmission module is responsible for data communication between the system and the external network and remote data exchange: after the package is put into storage, it sends the pickup information (such as pickup code and package location) to the recipient's client; after the pickup is completed, it sends the status completion information to the sender, the courier and the client to achieve information synchronization.

[0092] The AI ​​processing module is responsible for intelligent analysis of image information, including a package detection unit and a waybill recognition unit. The package detection unit runs the first AI model to identify the outline and location of the package from the image and confirm its existence. The waybill recognition unit runs the second AI model to accurately locate the waybill area from the package image and extract key text information such as the recipient's mobile phone number, address, and waybill number.

[0093] The identity verification module is responsible for verifying the legitimacy of the operator's identity. It includes a front-end interaction unit and a back-end verification unit. The front-end interaction unit collects identity information by receiving the operator's employee number, pickup code, and mobile phone number through the touch screen of the double-sided touchpad, keyboard, or front-facing AI camera, or by collecting their facial features and QR code information. The back-end verification unit judges the legitimacy of the identity by comparing the collected information with the reserved information in the database to verify the identity of the courier or recipient and decide whether to authorize the opening of the cabinet door.

[0094] The package verification module is responsible for real-time verification during the storage and retrieval process. It includes a real-time image analysis unit and a comparison and logic judgment unit. The real-time image analysis unit controls the AI ​​camera to take real-time photos of the packages on the platform when storing (before placing) or retrieving (after taking) a package from the storage compartment. The logic judgment unit performs error prevention comparison and alarm, comparing the features of the real-time image (or the identified waybill information) with the original records bound in the main control module. If there is a discrepancy (wrong compartment placed or wrong package retrieved), an audible and visual alarm (flash indicator light, voice alarm) is immediately triggered to stop the current process and prevent errors from occurring.

[0095] like Figure 8 As shown, this intelligent parcel warehousing, storage, and retrieval error prevention method based on multimodal information fusion includes the following steps: Step 1: Package Inbound Binding Process Step 1.1, Operator Authentication and System Startup: The courier enters their employee ID / password through the cabinet door interactive screen, or performs facial recognition / scanning through the front-facing AI camera; the authentication module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, the courier pulls the handle to open the cabinet door; after opening the cabinet door, the flipping mechanism is activated, flipping the displacement receiving platform horizontally to a vertical position outside the cabinet door, ready to receive the package; Step 1.2, Intelligent Collection and Active Positioning of Package Visual Information: The courier places the package on the displacement platform; visual collection: an AI camera located above the platform photographs the package; AI recognition and feedback control: the AI ​​processing module runs in parallel, the package detection unit identifies the package itself, confirms its presence and approximate position, and the waybill recognition unit attempts to read the text on the waybill; active focusing: if the waybill is not in the optimal recognition position, the AI ​​processing module sends instructions to the main control module, controlling the X-axis and Y-axis motors of the displacement platform to coordinate their movements, fine-tuning the horizontal position of the package until the waybill is stably located in the center of the camera's field of view; information extraction: in the optimal position, the camera captures high-definition images, and the waybill recognition unit accurately extracts key text information such as the waybill number and the recipient's mobile phone number; Step 1.3, Package Storage: The courier will pick up the package from the displacement platform and place it into any empty storage compartment. The courier will confirm the storage through the human-machine interface touchscreen module on the double-sided touch panel. Then, the displacement platform and the flipping mechanism will reset in sequence, the courier will close the cabinet door, and the electric lock will automatically lock. The monitoring camera facing the corresponding storage compartment will monitor whether a package is stored in that compartment and transmit the package information to the AI ​​processing module. The AI ​​processing module will then transmit the information to the main control and binding module. The main control and binding module will strongly associate and bind the package / recipient information (visual modality) extracted in Step 1.2 with the storage compartment information (spatial perception modality) generated in Step 1.3 to generate an entry record and store it in the database. Step 1.4: Send pickup information: The main control and binding module extracts the stored information and transmits it to the wireless transmission module. The wireless transmission module sends the pickup information to the client, the system updates the status, and the warehousing is completed. Step 2: Error Prevention Process for User Package Pickup: Step 2.1, Recipient Identity Verification and System Preparation: The recipient enters the pickup code / mobile phone number through the human-computer interaction touch screen module of the double-sided touch panel, or performs facial recognition through the front AI camera; the identity verification module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, and the courier pulls the handle to open the cabinet door; at the same time, the flipping mechanism is activated to flip the displacement and placement platform to a convenient handover position; Step 2.2, Package Location Guidance: The main control module queries the binding records in the database based on the recipient's identity information, finds the corresponding package and its storage compartment, and lights up the indicator light of the storage compartment; Step 2.3: Retrieve the package and place it on the verification platform: The recipient retrieves the package from the corresponding storage compartment according to the indicator light and places it on the extended displacement platform; the package retrieval and placement verification module controls the AI ​​camera above the platform to take real-time photos of the package on the platform; the verification module quickly compares the real-time image features (or the waybill information identified from them) with the original image features / waybill information stored when the package was bound to the warehouse; the error prevention logic is executed, and the core is to determine whether the "package on the current platform" is consistent with the "package that should be picked up by the current recipient in the system record"; Step 2.4, Verification Result Decision and Execution: Scenario 1, Verification Successful: The human-machine interaction module prompts "Verification successful, please pick up the package"; the recipient picks up the package from the platform and confirms completion on the touch screen; subsequent actions include the main control module resetting the control mechanism, the cabinet door closing and locking, the wireless transmission module sending a pickup completion notification to the shipping end, the express delivery end, and the client, and the database updating the inventory status. Scenario 2, verification fails (e.g., wrong package was picked up): The system executes, and the package pick-up and drop-off verification module immediately triggers an audible and visual alarm (flashing indicator light and sounding voice alarm), and displays an error warning on the human-machine interface screen; the system locks the current process, preventing the next normal operation, and requires administrator intervention.

[0096] In summary, this system is fully automated and intelligent: the system integrates electromechanical actuators for lifting, flipping, and accurate positioning, realizing full automation from identity verification, package handover, information collection to automatic storage and retrieval, reducing manual operation and improving efficiency.

[0097] This system has multiple proactive error prevention mechanisms: through dual AI visual verification of visual binding upon entry and real-time verification upon retrieval, combined with physical space verification by storage compartment sensors, it forms a proactive error prevention system that combines pre-emptive prevention and post-event verification, resulting in high reliability.

[0098] The system features a precise and reliable structural design: the tilting mechanism employs a clever L-shaped guide hole connecting rod conversion principle, and the displacement platform uses orthogonal screw transmission to ensure accurate and stable operation. Reinforced cabinet doors and a magnetic suction design enhance the overall rigidity and closure reliability of the equipment.

[0099] This system is highly integrated and adaptable to various environments: it integrates control, AI computing, sensing, execution, and interaction into the cabinet.

[0100] This method employs a closed-loop process design to eliminate information gaps: it links "information collection - binding - guidance - verification" into a closed loop. Upon entering the warehouse, a strong binding between visual information and physical location is established; upon retrieval, this is used as the basis for reverse verification, ensuring the continuity and accuracy of information flow throughout the process.

[0101] This method employs deep fusion decision-making based on multimodal information: it creatively integrates three types of heterogeneous information—identity (person), visual (package features / text), and spatial (storage compartment)—for joint binding and decision-making, which offers higher security and fault tolerance than verification methods based solely on a single information source (such as relying solely on the pickup code).

[0102] This method introduces key steps of "active focusing" and "transfer point verification": in the information acquisition stage, the recognition input is optimized by actively adjusting the package pose; and a mandatory verification step is set at the physical transfer point (platform) with the highest risk.

[0103] This method clearly divides the system's automatic execution part (identification, location, binding, and verification) and the manual operation part (placement and retrieval), and places the error prevention verification after the manual operation is completed and before the final confirmation, so as to achieve optimal risk control of human-machine collaboration.

[0104] The process logic (binding and verification) of this method is independent of specific hardware and can be adapted to access devices of different specifications. Based on an AI vision and multi-sensor framework, it provides a reusable methodology for standardized intelligent access processes.

[0105] The specific embodiments described herein are merely illustrative of the spirit of the invention. Those skilled in the art to which this invention pertains may make various modifications or additions to the described specific embodiments or use similar methods to substitute them, without departing from the spirit of the invention or exceeding the scope defined by the appended claims.

Claims

1. A multi-modal information fusion-based intelligent package warehousing, storage and pick-up error prevention management system, comprising a courier cabinet body (1), characterized in that, The express cabinet (1) has several fixed partitions (4) inside, which divide the interior of the express cabinet (1) into several storage compartments. Each partition (4) has an indicator light (5) on its front side. The upper part of the express cabinet (1) has two AI cameras (13). The front side of the express cabinet (1) is hinged to a cabinet door (6) via a spring-loaded hinge. The cabinet door (6) has a double-sided touch panel (12), an electric lock (10), and several monitoring cameras (3). The position and number of the monitoring cameras (3) correspond to the storage compartments, and the monitoring cameras (3) are directly opposite the corresponding storage compartments. The electric lock (10) is engaged and locked with the express cabinet (1). The inside of the cabinet door (6) has a flipping mechanism (9). (9) is equipped with a displacement holding platform (11), one of which is an AI camera (13) facing the top of the displacement holding platform (11) after it is flipped and unfolded, and the other AI camera (13) is facing the front of the express cabinet (1). The cabinet door (6) is equipped with a handle (14) on the outside. The display light (5), monitoring camera (3), flipping mechanism (9), electric lock (10), displacement holding platform (11) and AI camera (13) are all electrically connected to the double-sided touch panel (12). The double-sided touch panel (12) includes a human-computer interaction touch screen module, a flashing indicator light and a voice alarm. The main board of the double-sided touch panel (12) is equipped with a main control and binding module, a wireless transmission module, an AI processing module, an identity verification module and a pick-up and drop-off verification module. 2.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion according to claim 1, characterized in that, The lower four corners of the express cabinet (1) are all fixed with support feet (2), and a lock hole is opened on the inner left side wall of the express cabinet (1), the position of the lock hole is corresponding to the locking pin of the electric lock (10). 3.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion of claim 2, characterized in that, The cabinet door (6) has a reinforced cavity plate (7) in the shape of a square on its inner periphery. The double-sided touch panel (12) is located in the middle of the reinforced cavity plate (7). Magnets (8) are fixed on the inner side of the reinforced cavity plate (7) and are symmetrically distributed. The electric lock (10) is located inside the reinforced cavity plate (7). Two clearance holes are provided on the inner wall of the reinforced cavity plate (7) and are symmetrically arranged on the left and right. 4.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion of claim 3, characterized in that, The flipping mechanism (9) includes two fixed bases (15) arranged symmetrically on the left and right. The fixed bases (15) are fixed on the inner side of the cabinet door (6), and the fixed bases (15) extend out of the clearance hole on the same side. Each fixed base (15) is provided with a flipping component (16), and a flipping mounting rod (17) is provided between the two flipping components (16). The flipping assembly (16) includes a guide plate (18), which is fixed on a fixed base (15) on the same side. The guide plate (18) has an L-shaped guide hole (19). Two fixed plates (23) are fixed on the inner side of the guide plate (18) and arranged symmetrically. A flipping slide rail (26) is fixed between the two fixed plates (23). A flipping slide block (25) is slidably mounted on the flipping slide rail (26). A flipping electric push rod (28) is fixed on the upper fixed plate (23). The telescopic end of the flipping electric push rod (28) is fixedly connected to the flipping slide block (25). A flipping bearing seat (22) is fixed on the flipping slide block (25). A flipping shaft (24) is fixed inside the inner ring of the flipping bearing seat (22). A flipping link (21) is fixed at the end of the rotating shaft (24). The flipping link (21) is perpendicular to the axis of the flipping shaft (24). A guide roller (20) is rotatably provided at the end of the flipping link (21). The guide roller (20) is rolled inside the L-shaped guide hole (19). Limiters (27) are fixed on the fixed plates (23). The limiters (27) on the two fixed plates (23) face each other and are located on the upper and lower sides of the flipping bearing seat (22). The flipping mounting rod (17) is C-shaped, and the two ends of the flipping mounting rod (17) are respectively fixed to the ends of the flipping shaft (24) of the two flipping components (21). The limiters (27) and the flipping electric push rod (28) are electrically connected to the double-sided touch panel (12). 5.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion of claim 4, characterized in that, The displacement support platform (11) includes a platform base plate (29), a Y-axis plate (30), and an X-axis plate (34). The widths of the platform base plate (29), the Y-axis plate (30), and the X-axis plate (34) are equal, and their lengths decrease sequentially. One end of the platform base plate (29) is fixed to the upper end of the flip mounting rod (17). T-shaped guide rails (35) are provided in the middle of the upper ends of both the platform base plate (29) and the Y-axis plate (30). The T-shaped guide rail (29) and the T-shaped guide rail (35) on the Y-axis plate (30) are vertically arranged. The Y-axis plate (30) is slidably mounted on the T-shaped guide rail (35) on the platform base plate (29). The X-axis plate (34) is slidably mounted on the T-shaped guide rail (35) on the Y-axis plate (30). Two symmetrically arranged Y-axis mounting rotary seats (40) are fixed on the other side of the platform base plate (29). A Y-axis lead screw (31) is rotatably provided between the two Y-axis mounting rotary seats (40). One of the Y-axis mounting rotary seats... A Y-axis motor (33) is fixed on the base (40). The output shaft of the Y-axis motor (33) is fixedly connected to the Y-axis lead screw (31). A Y-axis shifter (32) is fixed at the middle position of the side of the Y-axis plate (30). The Y-axis shifter (32) is connected to the Y-axis lead screw (31) through a transmission fit. Two symmetrically arranged X-axis mounting rotary seats (36) are fixed on the side of the Y-axis plate (30). An X-axis lead screw (37) is rotatably arranged between the two X-axis mounting rotary seats (36). One of the X-axis mounting rotary seats (36) An X-axis motor (39) is fixed on the top, and the output shaft of the X-axis motor (39) is fixedly connected to the X-axis lead screw (37). An X-axis shifter (38) is fixed at the middle position of the side of the X-axis plate (34). The X-axis shifter (38) is connected to the X-axis lead screw (37) through transmission. When the displacement support platform (11) is placed horizontally, one of the AI ​​cameras (13) is directly opposite the middle position of the X-axis plate (34). The Y-axis motor (33) and the X-axis motor (39) are both electrically connected to the double-sided touch panel (12). 6.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion of claim 5, characterized in that, The main control and binding module, serving as the central processing and coordination hub of the system, includes an information binding and storage unit and a system process control unit. The information binding and storage unit receives package information from the AI ​​processing module and physical location identifiers from the spatial perception module, uniquely binding the three and storing them in the database. The system process control unit, acting as the overall commander, coordinates and controls the start, stop, and sequence of all sub-modules—identification, flipping mechanism, displacement platform, AI recognition, and verification—according to preset logic, driving the complete warehousing and retrieval process. The wireless transmission module is responsible for data communication between the system and external networks, enabling remote data exchange: after warehousing, retrieval information is sent to the recipient's client; after retrieval, status completion information is sent to the sender, courier, and client, achieving information synchronization. The AI ​​processing module is responsible for intelligent analysis of image information, including a package detection unit and a waybill recognition unit. The package detection unit runs a first AI model to identify the package's outline and location from the image, confirming its existence. The waybill recognition unit runs a second AI model to accurately locate the waybill area from the package image and extract key text information such as the recipient's mobile phone number, address, and waybill number. 7.The intelligent package warehouse-in, storage and pick error management system based on multi-modal information fusion of claim 6, wherein, The identity verification module is responsible for verifying the legitimacy of the operator's identity. It includes a front-end interaction unit and a back-end verification unit. The front-end interaction unit collects identity information by receiving the operator's employee number, pickup code, and mobile phone number through the touch screen of the double-sided touch panel, keyboard, or front-facing AI camera, or by collecting their facial features and QR code information. The back-end verification unit judges the legitimacy of the identity by comparing the collected information with the reserved information in the database to verify the identity of the courier or recipient and decide whether to authorize the opening of the cabinet door.

8. The intelligent parcel warehousing, storage, and error-proofing management system based on multimodal information fusion according to claim 7, characterized in that, The package retrieval and verification module is responsible for real-time verification during the storage and retrieval process. It includes a real-time image analysis unit and a comparison and logic judgment unit. When storing or retrieving a package, the real-time image analysis unit controls the AI ​​camera to take real-time photos of the package on the platform. The logic judgment unit performs error prevention comparison and alarm, comparing the features of the real-time image with the original records bound in the main control module. If they do not match, an audible and visual alarm is immediately triggered to stop the current process and prevent errors from occurring.

9. A management method for an intelligent parcel warehousing, storage, and error-proofing management system based on multimodal information fusion as described in claim 8, characterized in that, Including the following steps, Step 1: Package Inbound Binding Process Step 1.1, Operator Authentication and System Startup: The courier enters their employee ID / password through the cabinet door interactive screen, or performs facial recognition / scanning through the front-facing AI camera; the authentication module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, the courier pulls the handle to open the cabinet door; after opening the cabinet door, the flipping mechanism is activated, flipping the displacement receiving platform horizontally to a vertical position outside the cabinet door, ready to receive the package; Step 1.2, Intelligent Collection and Active Positioning of Package Visual Information: The courier places the package on the displacement platform; visual collection: the AI ​​camera above the platform photographs the package; AI recognition and feedback control: the AI ​​processing module runs in parallel, the package detection unit identifies the package itself, confirms its presence and approximate position, and the waybill recognition unit attempts to read the text on the waybill; active focusing: if the waybill is not in the optimal recognition position, the AI ​​processing module sends instructions to the main control module, controlling the X-axis and Y-axis motors of the displacement platform to coordinate their movements, fine-tuning the horizontal position of the package until the waybill is stably located in the center of the camera's field of view; information extraction: in the optimal position, the camera captures high-definition images, and the waybill recognition unit accurately extracts key text information such as the waybill number and the recipient's mobile phone number; Step 1.3, Package Storage: The courier will pick up the package from the displacement platform and place it into any empty storage compartment. The courier will confirm the storage through the human-machine interface touchscreen module on the double-sided touch panel. Then, the displacement platform and the flipping mechanism will reset in sequence. The courier will close the cabinet door, and the electric lock will automatically lock. The monitoring camera facing the corresponding storage compartment will monitor whether a package is stored in that compartment and transmit the package information to the AI ​​processing module. The AI ​​processing module will then transmit the information to the main control and binding module. The main control and binding module will strongly associate the package / recipient information extracted in Step 1.2 with the storage compartment information generated in Step 1.3 to generate an entry record and store it in the database. Step 1.4: Send pickup information: The main control and binding module extracts the stored information and transmits it to the wireless transmission module. The wireless transmission module sends the pickup information to the client, the system updates the status, and the warehousing is completed. Step 2: User package pickup error prevention process: Step 2.1, Recipient Identity Verification and System Preparation: The recipient enters the pickup code / mobile phone number through the human-computer interaction touch screen module of the double-sided touch panel, or performs facial recognition through the front AI camera; the identity verification module verifies the information, and after successful verification, the main control module issues an instruction: open the electric lock, and the courier pulls the handle to open the cabinet door; at the same time, the flipping mechanism is activated to flip the displacement and placement platform to a convenient handover position; Step 2.2, Package Location Guidance: The main control module queries the binding records in the database based on the recipient's identity information, finds the corresponding package and its storage compartment, and lights up the indicator light of the storage compartment; Step 2.3: Retrieve the package and place it on the verification platform: The recipient retrieves the package from the corresponding storage compartment according to the indicator light and places it on the extended displacement platform; the package retrieval and placement verification module controls the AI ​​camera above the platform to take real-time photos of the package on the platform; the verification module quickly compares the real-time image features with the original image features / waybill information stored when the package was bound to the warehouse; the error prevention logic judges and executes the verification logic, the core of which is to determine whether the "package on the current platform" is consistent with the information "the package that should be picked up by the current recipient in the system record"; Step 2.4, Verification Result Decision and Execution: Scenario 1, Verification Successful: The human-machine interaction module prompts "Verification successful, please pick up the package"; the recipient picks up the package from the platform and confirms completion on the touch screen; subsequent actions include the main control module resetting the control mechanism, the cabinet door closing and locking, the wireless transmission module sending a pickup completion notification to the shipping end, the express delivery end, and the client, and the database updating the inventory status. Scenario 2, verification fails: Upon system execution, the item pick-up and drop-off verification module immediately triggers an audible and visual alarm and displays an error warning on the human-machine interface screen; the system locks the current process, preventing further normal operation, and requires administrator intervention.

Images