Video order tracing system, method and device

Abstract

The embodiment of the invention provides a video order tracing system, method and device. The system comprises a management device, a mobile robot and a video acquisition device. The management equipment is used for generating a carrying task for realizing the carrying order in response to the carrying order, and scheduling the mobile robot to execute the carrying task; the mobile robot is used for executing a carrying task under the scheduling of the management equipment, and sending point location information for identifying a point location where the mobile robot arrives to the management equipment every time the mobile robot arrives at a point location of a preset type in the process of executing the carrying task; the management equipment is also used for sending the point location information and order information for identifying the carrying order to the video acquisition equipment; the video acquisition equipment is used for shooting the process of executing the carrying task by the mobile robot in real time to obtain a tracing video; and correspondingly storing the tracing video and the received order information and point location information. By applying the technical scheme provided by the embodiment of the invention, the order tracing efficiency can be improved.

Description

Technical Field

[0001] The present application relates to the field of warehouse logistics automation technology, and in particular to a video order tracing system, method and device. Background Art

[0002] Order tracing refers to the process of using video surveillance technology to trace the entire inbound and outbound order records. It is primarily used in warehousing operations, including outbound and inbound operations, tallying, and inventory. Currently, order tracing is achieved by installing multiple cameras in the warehouse, which continuously capture video data throughout the day. To subsequently trace a specific order, the video data captured by each camera must be reviewed. While this camera-based approach in related technologies can achieve order tracing, it requires manual querying of the individual video data sets, making order tracing inefficient. Summary of the Invention

[0003] The purpose of the embodiments of this application is to provide a video order tracing system, method, and device to improve order tracing efficiency. The specific technical solution is as follows:

[0004] In a first aspect, an embodiment of the present application provides a video order tracing system, the system comprising a management device, a mobile robot, and a video acquisition device;

[0005] The management device is configured to generate, in response to a transport order, a transport task for implementing the transport order, and schedule the mobile robot to perform the transport task;

[0006] The mobile robot is configured to perform the transport task under the scheduling of the management device; and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point it has reached to the management device;

[0007] The management device is further configured to send the location information and order information for identifying the transport order to the video acquisition device;

[0008] The video acquisition device is used to shoot the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly save the tracing video and the received order information and point information.

[0009] In some embodiments, the mobile robot is further configured to, in response to a falling object being carried, send a drop message indicating that an object has fallen to the management device during the execution of the carrying task;

[0010] The management device is specifically configured to send the drop information and the order information to the video acquisition device;

[0011] The video acquisition device is specifically used to store the traceability video and the received order information and drop information.

[0012] In some embodiments, the mobile robot is further configured to send location information identifying its own location to the management device whenever it reaches a preset type of point during the execution of the transport task;

[0013] The management device is specifically used to send the point information, the location information, and the order information to the video acquisition device;

[0014] The video acquisition device is specifically used to store the traceability video and the received order information, point information and location information.

[0015] In some embodiments, the mobile robot is further configured to send container information identifying the container being carried to the management device whenever it reaches a preset type of point during the execution of the carrying task;

[0016] The management device is specifically used to send the point information, the container information, and the order information to the video acquisition device;

[0017] The video acquisition device is specifically used to store the traceability video and the received order information, location information and container information.

[0018] In some embodiments, the system further comprises an item collection device disposed at the point;

[0019] The item collection device is used to collect item information of the items carried to the location by the mobile robot; and send the item information to the management device;

[0020] The management device is specifically used to send the point information, the item information, and the order information to the video acquisition device;

[0021] The video acquisition device is specifically used to store the traceability video and the received order information, location information and item information.

[0022] In some embodiments, the item information includes any one or more of the following information: item identification, item quantity, and item image.

[0023] In some embodiments, the system further comprises a terminal scanning device disposed at the point;

[0024] The terminal scanning device is used to scan the items carried to the location by the movable robot to obtain order information of the order to which the items belong.

[0025] In some embodiments, the order information includes job type, order type, order number, item number, operator number, job start time, and job end time.

[0026] In a second aspect, an embodiment of the present application further provides a video order tracing method, which is applied to a video order tracing system. The system includes a management device, a mobile robot, and a video acquisition device. The method includes:

[0027] The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches the mobile robot to perform the transport task;

[0028] The mobile robot performs the transport task under the scheduling of the management device; and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point reached by itself to the management device;

[0029] The management device sends the point information and the order information for identifying the transport order to the video acquisition device;

[0030] The video acquisition device records the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly saves the tracing video and the received order information and location information.

[0031] In some embodiments, the mobile robot sends drop information indicating that an item has fallen to the management device in response to the item being transported falling during the process of performing the transport task;

[0032] The management device sends the drop information and the order information to the video acquisition device;

[0033] The video acquisition device correspondingly stores the tracing video and the received order information and the drop information.

[0034] In some embodiments, when the mobile robot is performing the transport task, each time it reaches a preset type of point, it sends location information for identifying its own location to the management device;

[0035] The management device sends the point information, the location information, and the order information to the video acquisition device;

[0036] The video acquisition device stores the traceability video and the received order information, point information and location information.

[0037] In some embodiments, when the mobile robot is performing the transport task, each time it reaches a preset point, it sends container information identifying the container it is transporting to the management device;

[0038] The management device sends the point information, the container information, and the order information to the video acquisition device;

[0039] The video acquisition device correspondingly stores the traceability video and the received order information, location information, and container information.

[0040] In some embodiments, the system further comprises an item collection device disposed at the point;

[0041] The item collection device collects item information of the items carried to the location by the mobile robot; and sends the item information to the management device;

[0042] The management device sends the point information, the item information, and the order information to the video acquisition device;

[0043] The video acquisition device stores the traceability video and the received order information, location information, and item information.

[0044] In some embodiments, the item information includes any one or more of the following information: item identification, item quantity, and item image.

[0045] In some embodiments, the system further comprises a terminal scanning device disposed at the point;

[0046] The terminal scanning device scans the items carried to the location by the movable robot to obtain order information of the order to which the items belong.

[0047] In some embodiments, the order information includes job type, order type, order number, item number, operator number, job start time, and job end time.

[0048] In a third aspect, an embodiment of the present application further provides a video order tracing device, which is applied to a video order tracing system. The system includes a management device, a mobile robot, and a video acquisition device. The device includes:

[0049] a task generation module, configured to drive the management device to generate a transport task for implementing the transport order in response to the transport order, and to dispatch the mobile robot to perform the transport task;

[0050] a task execution module, configured to drive the mobile robot to perform the transport task under the scheduling of the management device; and, in the process of performing the transport task, whenever the mobile robot reaches a preset type of point, send point information for identifying the point reached by the mobile robot to the management device;

[0051] An information sending module, configured to drive the management device to send the location information and order information for identifying the transport order to the video acquisition device;

[0052] The video generation module is used to drive the video acquisition device to shoot the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly save the tracing video and the received order information and point information.

[0053] In some embodiments, the task execution module is further used to drive the mobile robot to send drop information indicating that an item has dropped to the management device in response to the drop of the item being transported during the execution of the transport task; the information sending module is further used to drive the management device to send the drop information and the order information to the video acquisition device; the video generation module is further used to drive the video acquisition device to correspondingly save the tracing video and the received order information, as well as the drop information; and / or,

[0054] The task execution module is further configured to drive the mobile robot to send location information for identifying its own location to the management device whenever it reaches a preset type of point during the execution of the transport task; the information sending module is further configured to drive the management device to send the point information, the location information, and the order information to the video acquisition device; the video generation module is further configured to drive the video acquisition device to correspondingly save the tracing video and the received order information, point information, and location information; and / or,

[0055] The task execution module is further configured to drive the mobile robot to send container information for identifying the container being carried to the management device whenever the mobile robot reaches a preset type of point during the execution of the carrying task; the information sending module is further configured to drive the management device to send the point information, the container information, and the order information to the video acquisition device; the video generation module is further configured to drive the video acquisition device to correspondingly save the tracing video and the received order information, point information, and container information; and / or,

[0056] The system further includes an item collection device provided at the location; the task execution module is further configured to drive the item collection device to collect item information of the items transported to the location by the mobile robot; and send the item information to the management device; the information sending module is further configured to drive the management device to send the location information, the item information, and the order information to the video collection device; the video generation module is further configured to drive the video collection device to correspondingly save the tracing video and the received order information, location information, and item information; and / or,

[0057] The item information includes any one or more of the following information: item identification, item quantity, item image; and / or,

[0058] The system further includes a terminal scanning device provided at the location; the apparatus further includes an information scanning module for driving the terminal scanning device to scan the items carried to the location by the mobile robot to obtain order information of the order to which the items belong; and / or,

[0059] The order information includes job type, order type, order number, item number, operator number, job start time, and job end time.

[0060] Beneficial effects of the embodiments of the present application:

[0061] In the technical solution provided by the embodiment of the present application, in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at each time it reaches a preset type of point to the management device; thereby, the management device sends the point information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information and point information. Thus, both the point information and the order information can be used as indexes for retrieving the tracing video, and there is no need to manually search for the required video clips from a large number of videos. In this way, the technical solution provided by the embodiment of the present application improves the efficiency of order tracing.

[0062] Of course, it is not necessary to achieve all the advantages described above at the same time when implementing any product or method of the present application. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other embodiments can also be obtained based on these drawings.

[0064] Figure 1 A schematic diagram of a process flow of warehousing business in the related technology provided in the embodiments of this application;

[0065] Figure 2 A schematic diagram of the structure of the video order tracing system provided in an embodiment of the present application;

[0066] Figure 3 A schematic diagram of the first process interaction of the order tracing system provided in an embodiment of the present application;

[0067] Figure 4 A schematic diagram of the second process interaction of the video order tracing system provided in an embodiment of the present application;

[0068] Figure 5 A schematic diagram of the third process interaction of the video order tracing system provided in an embodiment of the present application;

[0069] Figure 6 A schematic diagram of the fourth process interaction of the video order tracing system provided in an embodiment of the present application;

[0070] Figure 7 A schematic diagram of a search interface provided in an embodiment of the present application;

[0071] Figure 8 A flowchart of a video order tracing method provided in an embodiment of the present application;

[0072] Figure 9 A schematic diagram of the structure of a video order tracing device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0073] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field based on this application are within the scope of protection of this application.

[0074] Order traceability is typically accomplished through the collaborative work of an intelligent warehouse management system (IWMS), a warehouse control system (WCS), a robot control server (RCS), a client, and robots. The IWMS is primarily responsible for business and inventory management, as well as task initiation. The RCS is primarily responsible for scheduling warehouse robots, specifically, dispatching mobile robots to complete shelf handling tasks. The WCS is primarily responsible for device docking. The client, primarily installed on a workstation's personal computer (PC), interacts with staff to complete warehouse task prompts and execution.

[0075] In the field of warehouse logistics automation, the basic operational process for warehousing operations is as follows: the intelligent warehouse management system initiates tasks, including outbound and inbound operations, inventory counting, and tallying. The robot control service dispatches robots to move the corresponding shelves to the workstation. The workstation client receives and displays the task details, and the operator completes the corresponding task according to the client's prompts. Once the task is completed, the robot control service dispatches the robot again to move the shelves back to the warehouse.

[0076] For details, please refer to the following Figure 1 The example shown, Figure 1 A schematic flow chart of a warehousing business process in the related technology provided in the embodiments of the present application may include the following steps:

[0077] Step S11: The intelligent warehouse management system initiates a task;

[0078] It can be understood that the above step S11 is the first step executed after the warehousing business starts in this example. In some examples, the warehousing business in the related art may execute other steps before starting to execute step S11.

[0079] Step S12: The intelligent warehouse management system determines that the initiated task is a shelf outbound transportation task;

[0080] Step S13: In response to the shelf outbound transport task issued by the intelligent warehouse management system, the robot control service dispatches the robot to transport the shelf to the workstation;

[0081] Step S14: In response to the dispatch of the robot control service, the robot transports the shelf to the workstation;

[0082] Step S15: After the robot transports the shelf to the workstation, the client obtains the task details;

[0083] Step S16: The client executes the task;

[0084] Regarding the above steps S15 and S16, it can be understood that in this example, the task issued by the intelligent warehouse management system is a shelf outbound transportation task. Therefore, in the above step S15, the task details obtained by the client should be the detailed information for outbound transportation of the shelf; in the above step S16, the task performed by the client should be outbound transportation of the shelf based on the task details obtained in step S15.

[0085] Step S17: After the task is completed, the client sends a message to the intelligent warehouse management system to inform the intelligent warehouse management system that the task is completed;

[0086] Step S18: After the intelligent warehouse management system determines that the shelf outbound transportation task is completed, it issues the shelf inbound transportation task;

[0087] Step S19: In response to the shelf warehousing and transporting task issued by the intelligent warehouse management system, the robot control service dispatches the robot to transport the shelf into the warehouse;

[0088] Step S110: In response to the dispatch of the robot control service, the robot moves the shelf into the warehouse;

[0089] The above step S110 is the last step executed before the warehousing business in this example is completed. In some examples, the warehousing business in the related art may execute other steps after completing step S110.

[0090] It is understandable that during warehousing operations, in order to facilitate warehouse management personnel to grasp the status of orders in real time and detect abnormal situations in a timely manner, thereby improving warehouse management efficiency and reducing order processing time, in related technologies, multiple cameras are usually set up in the warehouse, and the cameras are used to continuously shoot video data throughout the day, thereby facilitating operators to have real-time control over the order processing process. Moreover, if it is necessary to trace a certain order later, it can also be achieved through the previously shot video data. However, if the order is to be traced through the previously shot video data, it is necessary to go back and view the video data shot by each camera, that is, the operator needs to query in each video data, which results in low efficiency of order tracing.

[0091] In this regard, in the field of industrial automation, related technologies have disclosed an industrial Internet platform and method for production process management. In this method, workshop equipment uses cameras to collect product production images on the production line. Afterwards, the industrial Internet platform performs visual recognition on the product production images collected by the camera to generate effective production data, and obtains the final basic data through data fusion processing, thereby realizing real-time production supervision and data statistics. As a result, all process data and records of the entire life cycle of the product can be traced.

[0092] However, in this method, the product production image needs to be identified in order to obtain valid production data. Therefore, the clarity requirement for the product production image is relatively high. That is, this method has high requirements for the camera and the cost of setting up the equipment is relatively high. Moreover, in this method, the valid production data obtained through identification is used to trace all process data and records of the entire life cycle of the product. In this way, when tracing, only the valid production data or related product production images obtained through image recognition can be traced, and the video data of the product at a certain stage cannot be accurately retrieved, that is, visual tracing is impossible.

[0093] Therefore, to solve the problem of low efficiency of order tracing, the embodiment of the present application provides an order tracing system, see Figure 2 , Figure 2 A schematic diagram of the structure of the video order tracing system provided in the embodiment of the present application is shown as follows: Figure 2 As shown, the video order tracing system includes a management device 21, a mobile robot 22 and a video acquisition device 23.

[0094] Among them, the management device 21 is used to generate a handling task and dispatch the mobile robot 22 to perform the generated handling task; the mobile robot 22 is used to perform the generated handling task under the dispatch of the management device 21, and, in the process of performing the handling task, sends relevant information (such as the point information, drop information, etc. mentioned below) to the management device 21. Then, the management device 21 can send the relevant information uploaded by the mobile robot 22 and the order information of the order executed by the mobile robot 22 to the video acquisition device 23, and the video acquisition device 23 will shoot the process of the mobile robot 22 performing the handling task in real time, generate a tracing video, and correspondingly save the tracing video and the received order information and related information. In this way, when it is necessary to trace an order, it is only necessary to enter the relevant information or order information to be queried in the system to retrieve the corresponding video to achieve order tracing, thereby improving the efficiency of order tracing.

[0095] For details, see the following Figure 3 , Figure 3 The first process interaction diagram of the order tracing system provided in the embodiment of the present application may include the following steps:

[0096] Step S31: The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches a mobile robot to perform the transport task;

[0097] Step S32: The mobile robot performs the transport task under the scheduling of the management device, and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point it has reached to the management device;

[0098] Step S33: The management device sends the point information and the order information for identifying the transport order to the video capture device;

[0099] Step S34: The video acquisition device captures the process of the mobile robot performing the transport task in real time to obtain a tracing video, and correspondingly saves the tracing video and the received order information and location information.

[0100] In the technical solution provided by the embodiment of the present application, in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at each time it reaches a preset type of point to the management device; thereby, the management device sends the point information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information and point information. Thus, both the point information and the order information can be used as indexes for retrieving the tracing video, and there is no need to manually search for the required video clips from a large number of videos. In this way, the technical solution provided by the embodiment of the present application improves the efficiency of order tracing.

[0101] In the above step S31, the management device is used to generate a transport task for implementing the transport order in response to the transport order, and schedule the mobile robot to perform the transport task.

[0102] Specifically, in some embodiments, the management equipment includes an intelligent warehouse management system (i.e., IWMS), a robotic control and monitoring system (RCMS), and a robotic control service (i.e., RCS). The IWMS is a business relationship system used to manage material batches, inventory, inbound and outbound orders, inventory, tallying, and other functions; the RCMS is used to manage warehouse maps and equipment (such as mobile robots, charging stations, etc.). After obtaining a handling task, the RCMS can decompose the task and send the decomposed subtasks to the RCS; the RCS is used to schedule and control the movement of autonomous mobile robots (AMRs), that is, to perform handling tasks.

[0103] Therefore, in this example, the management device generates a handling task for implementing the handling order in response to the handling order, and dispatches the mobile robot to perform the handling task. It can be understood that: IWMS receives tasks such as warehousing and inventory orders (i.e., handling orders) issued by upstream Enterprise Resource Planning (ERP) system / Manufacturing Execution System (MES) / System Applications and Products (SAP) system / Warehouse Management System (WMS) and other systems, and then sends the received order tasks to RCMS, which converts the order tasks into subtasks (i.e., handling tasks), such as shelf inbound and return tasks, and sends the converted subtasks to RCS. After receiving the subtasks issued by RCMS, RCS performs secondary decomposition of the subtasks, and dispatches AMR to transport the shelves to the corresponding locations through algorithms such as path planning and task allocation, that is, dispatches AMR to perform handling tasks.

[0104] In addition, in the embodiments of the present application, mobile robots may also be referred to as warehouse robots, which can perform operations such as picking, placing, and transporting goods in the warehouse. Common mobile robots include lurking robots and pick-and-place robots. Common lurking robots include automated guided vehicles (AGVs), which are primarily used for transporting shelves; and common pick-and-place robots include container transfer units (CTUs), which can directly pick, place, and transport containers in the warehouse.

[0105] In the above step S32, the mobile robot performs the corresponding transport task in response to the scheduling of the management device. During the process of performing the transport task, each time the mobile robot reaches a preset type of point, it sends point information to the management device.

[0106] The preset type of location is a location pre-set by the operator based on actual conditions, where the mobile robot is required to report location information to the management device. Specifically, in some embodiments, the preset type of location may be a conveyor line, elevator, workstation, charging pile, shelf, etc. The workstation is a work surface for warehousing operations, where outbound and inbound operations, inventory counting, and tallying operations can be performed.

[0107] In some embodiments, the mobile robot can capture images in real time through the image capture device it carries to determine whether it has reached a preset type of point; or, in other embodiments, the mobile robot can also determine whether it has reached a preset type of point based on the path and distance it has moved; or again, the mobile robot can determine whether it has reached a preset type of point through real-time positioning.

[0108] The point information is used to identify the point reached by the mobile robot itself. The specific representation of the point information can be pre-set by the operator. Specifically, in some embodiments, the point information can be represented in the form of a number. For example, the preset type of points includes transmission line 1 and workstation 1, and the transmission line 1 is pre-set to correspond to number 1 and the workstation 1 is pre-set to correspond to number 2. When the mobile robot reaches transmission line 1, the point information sent by the mobile robot to the management device is number 1; when the mobile robot reaches workstation 1, the point information sent by the mobile robot to the management device is number 2.

[0109] Alternatively, in other embodiments, the point information can also be directly represented by a point. For example, the preset type of point includes transmission line 1 and workstation 1. When the mobile robot reaches transmission line 1, the point information sent by the mobile robot to the management device is transmission line 1; when the mobile robot reaches workstation 1, the point information sent by the mobile robot to the management device is workstation 1.

[0110] In the above step S33, after the management device receives the point information sent by the mobile robot, it sends the point information and the order information of the handling order to the video acquisition device. Among them, the order information is understood to be information used to uniquely identify the order, such as the order information can be an order number. Among them, the order information can be only the order number, or it can be information including the order number and other information. Specifically, in some embodiments, in addition to the order number, the order information can also include any one or more of the following information: operation type, order type, item number, operator number, operation start time and operation end time. Among them, the operation type can be a warehousing operation, an outbound operation, an inventory operation, a picking operation, etc.; the order type can be an inbound order, an outbound order, an inventory order, etc.

[0111] The following two examples illustrate how the management device obtains order information.

[0112] In some embodiments, the video order tracing system also includes a terminal scanning device set at a point, that is, the terminal scanning device can be set in advance at any one or more preset types of points. For example, if the preset types of points are transmission lines, workstations, and shelves, the terminal scanning device can be set in advance on the transmission line, or the terminal scanning device can be set on the transmission line and the workstation at the same time.

[0113] The terminal scanning device is used to scan items carried to the location by the mobile robot and obtain order information for the order to which the items belong. It is understood that after the terminal scanning device obtains the order information through scanning, it can send the order information to the management device. In this way, the management device obtains the order information that needs to be sent to the video capture device along with the location information.

[0114] For example, a terminal scanning device is provided on workstation 1. When the movable robot arrives at workstation 1, it scans through the terminal scanning device to obtain the item number, order number, operator number, job type, order type, job start time and job end time. Thus, the terminal scanning device can send the scanned item number, order number, operator number, job type, order type, job start time and job end time to the management device. Then, the management device can send the received item number, order number, operator number, job type, order type, job start time and job end time as order information together with the point information sent by the movable robot to the video acquisition device.

[0115] Among them, information such as the item number, order number, operator number, and job type can be directly obtained after the terminal scanning device scans the items carried by the mobile robot; or the terminal scanning device only obtains part of the information by scanning the carried items, and the remaining information is obtained after querying the system through the scanned information. For example, the terminal scanning device only obtains the order type, order number, and item number by scanning the carried items, while the operator number, job type, job start time, and job end time are obtained after querying the system based on the order type, order number, and item number obtained by the scan.

[0116] Alternatively, in other embodiments, when the mobile robot sends point information to the management device, it may also carry information that can be used to query order information in the system, such as the item number of the item carried by the mobile robot itself, the order number of the order executed by the mobile robot itself, etc. In this way, the management device can query relevant order information in the system based on this information to obtain the order information.

[0117] For example, when the mobile robot sends point information to the management device, it also carries the order number of the order it is executing. The management device can then query the system in accordance with the order number to obtain the item number of the item currently carried by the mobile robot, the order type and job type of the task performed by the mobile robot, and the operator number, job start time, and job end time of the operator at the point where the mobile robot is located. Furthermore, the management device can use the obtained order number, as well as the queried item number, job type, order type, operator number, job start time, and end time as order information.

[0118] In the above step S34, the video capture device is used to capture the process of the mobile robot performing the transport task in real time. The process of obtaining the tracing video through the video capture device can be seen in the following description:

[0119] In some embodiments, the video capture device records the process of the mobile robot performing the handling task in real time, and segments the video according to a preset duration. Each video segment is a tracing video, and each tracing video segment and the received order information and point information are saved accordingly. The correspondence between the tracing video, order information and point information here can be understood as, for each tracing video segment, the point information and order information corresponding to the tracing video are the point information and order information received by the video capture device during the tracing video shooting process. Specifically, for example, the preset duration is ten minutes, and the video acquisition device starts recording at 10:00. During the period from 10:00 to 10:10, the point information 1 and order information 1 sent by the management device are received only at 10:05. After completing the video shooting from 10:00 to 10:10, the video from 10:00 to 10:10 (i.e., the tracing video) is saved in correspondence with the point information 1 and order information 1. Subsequently, during the period from 10:10 to 10:20, the point information 2 and order information 2 sent by the management device are received at 10:11, and the point information 3 and order information 3 sent by the management device are received at 10:16. After completing the video shooting from 10:10 to 10:20, the video from 10:10 to 10:20 (i.e., the tracing video) is saved in correspondence with the point information 2 and order information 2, and the video from 10:10 to 10:20 (i.e., the tracing video) is saved in correspondence with the point information 3 and order information 3.

[0120] Alternatively, in other embodiments, the video capture device records the process of the mobile robot performing the transport task in real time. During this process, each time the video capture device obtains a point information and order information, it stores the corresponding time in correspondence with the point information and order information. Specifically, for example, if the video capture device starts recording at 10:00 and receives point information 1 and order information 1 from the management device at 10:05, 10.05 is stored in correspondence with point information 1 and order information 1; then, if the video capture device receives point information 2 and order information 2 from the management device at 10:11, 10:11 is stored in correspondence with point information 2 and order information 2.

[0121] It is understood that in this example, the traceability video is a video shot during a time period that includes the corresponding time. For example, if the video capture device receives point information 1 and order information 1 from the management device at 10:05, the video capture device will associate 10:05 with point information 1 and order information 1, and will use the video from 10:05 forward and backward for 5 minutes (i.e., the video from 10:00 to 10:10) as the traceability video. The time corresponding to the point information and order information can be the time when the video capture device receives the point information and order information. However, considering the time delay of information transmission, the time corresponding to the point information and order information can also be set to the time that is a preset time interval before the time when the video capture device receives the point information and order information. For example, if the preset time interval is 2 seconds, and the time when the video capture device receives point information 1 and order information 1 is 10:05, then the time corresponding to point information 1 and order information 1 is 10:02.

[0122] In the embodiment of the present application, it can be understood that after the corresponding tracing video and the received order information and point information are saved, when tracing the order, the operator can determine the corresponding tracing video by searching based on the order information and point information, thereby avoiding retrieving the required video clips from a large video and improving the efficiency of video tracing. Specifically, for example, if there is a corresponding tracing video 1 and transmission line 1 (i.e. point information) and order number 1 (i.e. order information), and a tracing video 2 and workstation 1 (i.e. point information) and order number 2 (i.e. order information), the operator can search the tracing video on the retrieval interface (see below) to find the tracing video. Figure 7 ) by entering the corresponding point information and order number to determine the corresponding traceability video. For example, if the operator wants to view the processing status of the order with order number 1 on transmission line 1, the operator can retrieve the corresponding traceability video 1 by entering transmission line 1 and order number 1 on the search interface. Based on the traceability video 1, the operator can understand the processing status of the order with order number 1 on transmission line 1.

[0123] In some embodiments, when the order information also carries other information besides the order number, the tracing video can also be retrieved through the other information. For example, if the order information includes the operator number in addition to the order number, then if the operator needs to view the situation of a certain operator handling a certain order, the operator can enter the operator number and the order number of the order in the search interface. Based on this, the tracing video saved corresponding to the operator number and the order number can be retrieved, that is, the video of the operator handling the order. Alternatively, in some embodiments, if the operator needs to view the working status of a certain operator, the operator can also enter only the operator number of the operator in the input interface. In this way, all tracing videos saved corresponding to the operator number can be retrieved, that is, all videos about the operator.

[0124] Based on the above Figure 3 As shown in the example, in some embodiments, the mobile robot is also used to send drop information indicating that the item has dropped to the management device in response to the item being transported dropping during the execution of the transport task. That is, if the mobile robot detects that the item being transported has dropped during the execution of the transport task, it can send drop information indicating that the item has dropped to the management device. After receiving the item drop information, the management device sends the item drop information and the order information to the video capture device, and the video capture device correspondingly saves the tracing video and the received order information and drop information. Here, the principle of correspondingly saving the tracing video and the received order information and drop information is the same as the principle of correspondingly saving the tracing video and the received point information and order information in the above step S34. It is only necessary to replace the point information in the above step S34 with the drop information.

[0125] Alternatively, in some embodiments, the movable robot may detect that the carried item has fallen when reaching a preset type of point. At this time, the movable robot will simultaneously send the point information and the drop information to the management device. After receiving the point information and the drop information, the management device will simultaneously send the point information, the drop information and the order information to the video acquisition device, thereby correspondingly saving the tracing video and the point information, the drop information and the order information. Here, the corresponding manner of saving the tracing video and the point information, the drop information and the order information is the same as the principle of correspondingly saving the tracing video and the received point information and the order information in the above step S34. It is only necessary to replace the point information in the above step S34 with the point information and the drop information.

[0126] It is understandable that, based on the embodiments of the present application, the information saved corresponding to the tracing video may be point information and order information, or it may be drop information and order information, or it may be point information, drop information and order information. Therefore, in subsequent search pages, order information, point information and drop information can all be used as indexes to retrieve related tracing videos. Which one or more indexes are used is determined according to the needs of the operator. For example, if the operator is only concerned about the situation where the items fall during the transportation process, the drop information can be used as an index alone to retrieve the tracing video saved corresponding to the drop information.

[0127] In some embodiments, the drop information can be represented in the form of numbers, serial numbers, etc. For example, if the drop information is represented in digital form, it is preset that if an item is detected to have dropped, the number 1 is sent. In this way, when subsequently retrieving the video saved corresponding to the drop information, the number 1 can be selected in the drop information corresponding to the search interface. In addition, in some embodiments, when the drop information is sent to the management device, it also carries information indicating the location where the item dropped. For example, after detecting that the item has dropped, the mobile robot can collect the serial number of the nearby shelf. In this way, when the drop information is sent to the management device, it also carries the serial number of the collected shelf. In this way, when an alarm is triggered by the drop information, the operator can promptly find the specific location where the item dropped and handle it.

[0128] In some embodiments, the mobile robot can also be used to send location information for identifying its own location to the management device whenever it reaches a preset type of point during the execution of the transport task. For details, see Figure 4 , Figure 4 The second process interaction diagram of the video order tracing system provided in the embodiment of the present application may include the following steps:

[0129] Step S41: The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches a mobile robot to perform the transport task;

[0130] Step S42: The mobile robot performs the transport task under the scheduling of the management device, and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point it has reached and position information for identifying its location to the management device;

[0131] Step S43: The management device sends the point information, location information, and order information to the video acquisition device;

[0132] Step S44: The video acquisition device records the process of the mobile robot performing the handling task in real time to obtain a tracing video, and correspondingly saves the tracing video and the received order information, point information, and location information.

[0133] In the technical solution provided by the embodiment of the present application: in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at, and position information for identifying its own position to the management device every time it reaches a preset type of point; thereby, the management device sends the point information, position information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information, point information and position information. Therefore, the point information, position information and order information can all be used as indexes to retrieve the tracing video, eliminating the need to manually search for the required video clips from a large number of videos, thereby improving the efficiency of order tracing.

[0134] The above step S41 and Figure 3 The above steps S31 are the same as those in the above steps S31, which will not be described again here. Only the above steps S42 to S44 will be described below.

[0135] In the above step S42, the position information is information used to indicate the position of the movable robot itself. Specifically, the position information can be expressed in the form of coordinates or the like.

[0136] In this embodiment, when the mobile robot is performing a transport task, each time it reaches a preset type of point, the mobile robot can send point information and position information to the management device. The preset type of point and point information can be found in the description of step S32 above. Step S42 is executed in the same manner as step S32 above, with the point information and position information simply being replaced with the point information in step S32.

[0137] In the above step S43, after the management device receives the point information and position information sent by the movable robot, it sends the point information, position information and order information to the video acquisition device. For the description of the order information, please refer to the relevant content in the above step S33. In addition, the execution principle of step S43 is the same as that of the above step S33. It is only necessary to replace the point information in the above step S33 with the point information and position information.

[0138] In step S44, the video capture device stores the traceability video and the received order information, location information, and location information. It will be appreciated that the method for determining the traceability video and the corresponding order information, location information, and location information in this step is similar to the method for determining the traceability video and the corresponding location information and order information in step S34. Simply replace the location information in step S34 with the location information and location information.

[0139] It is understood that in some embodiments, in the above Figure 4 Based on the example shown, when the mobile robot is performing a transport task, in response to the transported item falling, it can also send drop information to the management device to indicate that an item has fallen. Therefore, after receiving the item drop information, the management device can send the drop information and order information to the video capture device, and the video capture device will correspondingly save the tracing video and the drop information and order information. Alternatively, if the mobile robot detects that an item has fallen when moving to a preset type of point, the mobile robot can simultaneously send the point information, position information, and drop information to the management device. Then, the management device will simultaneously send the drop information, position information, point information, and order information to the video capture device, and the video capture device will correspondingly save the tracing video and the drop information, position information, point information, and order information. Therefore, the drop information, position information, point information, and order information can all be used as indexes to enable retrieval of the tracing video, thereby improving the efficiency of order tracing.

[0140] In some embodiments, the mobile robot can also be used to send container information for identifying the container it is carrying to the management device whenever it reaches a preset type of point during the execution of the transportation task. For details, see Figure 5 , Figure 5 The third process interaction diagram of the video order tracing system provided in the embodiment of the present application may include the following steps:

[0141] Step S51: The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches a mobile robot to perform the transport task;

[0142] Step S52: The mobile robot performs the transport task under the scheduling of the management device. During the transport task, whenever it reaches a preset location, it sends location information identifying the location it has reached and container information identifying the container it is transporting to the management device.

[0143] Step S53: The management device sends the location information, container information, and order information to the video capture device;

[0144] Step S54: The video acquisition device captures the process of the mobile robot performing the handling task in real time to obtain a tracing video, and correspondingly saves the tracing video and the received order information, location information, and container information.

[0145] In the technical solution provided by the embodiment of the present application: in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at, and container information for identifying the container it is transporting to the management device every time it reaches a preset type of point; thereby, the management device sends the point information, container information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information, point information and container information. Therefore, the point information, container information and order information can all be used as indexes to retrieve the tracing video, eliminating the need to manually search for the required video clips from numerous videos, thereby improving the efficiency of order tracing.

[0146] The above step S51 and Figure 3 The above steps S31 are the same as those in the above steps, which will not be described again here. Only steps S52 to S54 will be described below.

[0147] In step S52 above, the container information is used to identify the container being carried by the mobile robot. Specifically, in some implementations, the container information may be the container number of the container being carried by the mobile robot. For example, if the container being carried by the mobile robot upon reaching a predetermined location is a shelf, the container information may be the shelf number of the shelf. Alternatively, if the container being carried by the mobile robot upon reaching a predetermined location is a bin, the container information may be the bin number.

[0148] In this embodiment, when the mobile robot is performing a transport task, each time it reaches a preset location, it can send location information and container information to the management device. The preset location and location information can be found in the description of step S32 above. Step S52 is executed in the same manner as step S32 above, with the location information and container information being replaced by the location information in step S52.

[0149] In the above step S53, after the management device receives the point information and container information sent by the movable robot, it sends the point information, container information and order information to the video acquisition device. For the description of the order information, please refer to the relevant content in the above step S33. In addition, the execution principle of step S53 is the same as that of the above step S33. It is only necessary to replace the point information in the above step S33 with the point information and container information.

[0150] In step S54, the video capture device stores the traceability video and the received order information, location information, and container information. It should be understood that the method for determining the traceability video and the corresponding order information, location information, and container information in this step is similar to the method for determining the traceability video and the corresponding location information and order information in step S34. Simply replace the location information in step S34 with the location information and container information.

[0151] It is understood that in some embodiments, in the above Figure 5 Based on the example shown, when a mobile robot is performing a transport task and an item falls, it can also send drop information to a management device indicating that an item has fallen. Upon receiving the drop information, the management device can send the drop information and order information to a video capture device, which then stores the tracing video, drop information, and order information accordingly. Alternatively, if the mobile robot detects an item has fallen when moving to a preset location, the mobile robot can simultaneously send the location information, container information, and drop information to the management device. Furthermore, the management device simultaneously sends the drop information, container information, location information, and order information to the video capture device, which then stores the tracing video, drop information, container information, location information, and order information accordingly. Thus, the drop information, container information, location information, and order information can all be used as indexes to retrieve the tracing video, thereby improving the efficiency of order tracing.

[0152] Alternatively, in some embodiments, Figure 5 On the basis of what is shown, when the mobile robot reaches a preset type of point, it can also send location information to the management device to identify its own location. Therefore, when the mobile robot performs a handling task, each time it reaches a preset type of point, it can send point information, location information, and container information to the management device. Furthermore, the management device can send the point information, location information, container information, and order information to the video acquisition device. As a result, the video acquisition device can correspondingly save the traceability video and the point information, location information, container information, and order information. In this way, the container information, point information, location information, and order information can all be used as indexes to enable retrieval of the traceability video, thereby improving the efficiency of order tracing.

[0153] When the management device only receives drop information, it can send the drop information and order information to the video capture device, which will store the tracing video, drop information, and order information accordingly. When the management device only receives point information, location information, and container information, it can send the point information, location information, and container information to the video capture device at the same time, and the video capture device will store the tracing video, point information, location information, and container information accordingly. When the management device receives drop information, point information, location information, and container information at the same time, it can send the drop information, point information, location information, container information, and order information to the video capture device at the same time, and the video capture device will store the tracing video, drop information, point information, location information, container information, and order information accordingly. Therefore, the drop information, location information, point information, container information, and order information can all be used as indexes to retrieve the tracing video, thereby improving the efficiency of order tracing.

[0154] Alternatively, in some embodiments, in the above Figure 5 On the basis of the shown embodiment, the mobile robot can also send drop information to the management device in response to the drop of the transported item, and when it reaches a preset type of point, it sends position information for identifying its own position to the management device. That is, in response to the drop of the transported item, the mobile robot sends drop information to the management device, and when the mobile robot reaches a preset type of point, the mobile robot is also used to send point information, position information and container information to the management device. Therefore, when the management device only receives the drop information, it can send the drop information and order information to the video capture device, and the video capture device will correspondingly save the traceability video and drop information. When the management device only receives point information, location information, and container information, it can send the point information, location information, and container information to the video capture device at the same time, and the video capture device will save the tracing video and the point information, location information, and container information accordingly. When the management device receives drop information, point information, location information, and container information at the same time, it can send the drop information, point information, location information, container information, and order information to the video capture device at the same time, and the video capture device will save the tracing video and the drop information, point information, location information, container information, and order information accordingly. Therefore, the drop information, location information, point information, container information, and order information can all be used as indexes to realize the retrieval of the tracing video, thereby improving the efficiency of order tracing.

[0155] In some embodiments, the video order tracing system also includes an item collection device set at a point, which is used to collect item information of items transported to the point by the mobile robot and send the item information to the management device. Furthermore, the management device can send the point information, item information and order information to the video collection device, and the video collection device will correspondingly save the tracing video and the received order information, point information and item information.

[0156] Specifically, in an embodiment of the present application, the order tracing system includes preset points of different types. Operators can pre-install item collection devices at certain preset points (or all preset points) based on actual needs to collect item information. Item information includes any one or more of the following: item identification, item quantity, and item image. It is understood that the item collection device can be an image acquisition device or a video recording device. Thus, the item collection device can simultaneously capture item identification, item quantity, and item image. Of course, if the item information does not include item images, the item collection device can also be a radio frequency device capable of identifying item identification and item quantity.

[0157] Based on this, when the mobile robot arrives at a location where an item collection device is installed, it can simultaneously send the item information and the location information to the management device; when the mobile robot arrives at a location where an item collection device is not installed, it sends the location information to the management device. Furthermore, if the management device receives both the item information and the location information, the management device simultaneously sends the item information, the location information, and the order information to the video collection device, which then stores the traceability video and the location information, the item information, and the order information accordingly. Here, the method for determining the traceability video and the location information, the item information, and the order information corresponding to the traceability video is the same as the method for determining the traceability video and the location information and the order information corresponding to the traceability video in step S34 above. It is only necessary to replace the location information in step S34 above with the item information and the location information. If the management device only receives the location information, the location information and the order information are sent to the video collection device, which then stores the traceability video and the location information and the order information accordingly according to the content shown in step S34 above. Therefore, item information, location information, and order information can all be used as indexes to retrieve traceability videos, thereby improving the efficiency of order traceability.

[0158] It can be understood that based on the aforementioned multiple embodiments, in the present application, when the mobile robot is performing a transport task, each time it reaches a preset type of point, in addition to sending the point information to the management device, the mobile robot can also send any one or more of the following information to the management device: location information, container information, and item information. Furthermore, when the mobile robot is performing a transport task, if it detects that the transported item has fallen, it can also send the fall information to the management device; thereby, the management device sends the information sent by the mobile robot and the order information to the video capture device, which correspondingly saves the information sent by the mobile robot and the order information. Thus, the information sent by the mobile robot and the order information can be used as indexes to facilitate operators to retrieve the tracing video as needed, thereby improving the efficiency of order tracing.

[0159] Specifically, in some examples, each time the mobile robot reaches a preset type of point, it will send point information, position information, container information, and item information to the management device. Moreover, when the mobile robot detects that the transported item has fallen, it will send the falling information to the management device. Thus, the point information, position information, container information, item information, falling information, and order information can all be used as indexes. Based on this, if the operator needs to check the status of a certain order (such as order 1) at a certain point (such as transmission line 1), the operator enters the point (i.e., transmission line 1) and the order information in the search interface. The order number (i.e., the number of order 1) can be used to retrieve the tracing video saved corresponding to the point and the order number; similarly, if the operator needs to check whether a certain order (such as order 1) has dropped during the transportation process, the operator can enter the order number (i.e., the number of order 1) in the retrieval interface and select the drop information (such as selecting the number 1 in the drop information corresponding to the retrieval interface in the above example) to retrieve the corresponding tracing video. Specifically, the information saved corresponding to the tracing video includes the order number of the order (i.e., the order number of order 1) and the drop information. It can be seen that in the embodiment of the present application, the order can be traced by entering information for retrieval, and the operator does not need to search for the required content one by one in the numerous video contents, thereby improving the efficiency of order tracing.

[0160] At the same time, it can be understood that since the retrospective video in this application carries time information, the retrospective video can also be retrieved using time as an index.

[0161] See below. Figure 6 Taking the management equipment including the intelligent warehouse management system, the robot center management system and the robot control service as an example, the video order tracing system provided by the embodiment of the present application is described again through specific examples. Specifically, Figure 6 The following steps are included:

[0162] Step S61: The intelligent warehouse management system receives order information issued by the upstream system;

[0163] Among them, the upstream system can be ERP / MES / SAP / WMS system; the order information can include tasks such as warehousing and outbound tasks, inventory order tasks, etc., and the details can be found in the description of the above step S31.

[0164] Step S62: IWMS searches for the order and executes the task of issuing the order to the shelf. For details, please refer to the description of step S31 above.

[0165] Step S63: After receiving the task issued by the IWMS, the RCMS converts it into subtasks, such as shelf inbound and return tasks. For details, please refer to the description of step S31 above.

[0166] Step S64: RCMS sends the converted subtask to RCS. For details, please refer to the description of step S31 above.

[0167] Step S65: After receiving the subtasks issued by the RCMS, the RCS assigns mobile robots to each subtask (such as the aforementioned transport task) through path planning and task allocation algorithms. In other words, it schedules the mobile robots to perform the corresponding subtasks. For details, please refer to the description of step S31 above.

[0168] It is understood that when the mobile robot performs the corresponding subtask, each time it reaches a preset point type, it reports the point information, location information, container information, and item information to the IWMS. Moreover, when it detects that a transported item has fallen, it reports the drop information to the IWMS. Specifically, in this example, after detecting that an item has fallen, the mobile robot can change the depth of field of its own image acquisition device to obtain the nearby shelf number, and then report the obtained shelf number and the drop information to the IWMS. For details, please refer to the relevant descriptions of steps S32, S42, and S52 above.

[0169] Step S66: The IWMS reports the relevant information reported by the mobile robot (including point information, which may include location information, container information, item information, and drop information) and order information to the video capture device in real time. For details, please refer to the relevant descriptions of steps S33, S43, and S53 above.

[0170] It is understandable that when the mobile robot detects that an item has fallen, it can also inform the operator by pushing a message that the item being carried has fallen, so that the operator can handle it in time, thereby improving the processing efficiency of the warehousing business.

[0171] Step S67: The video capture device captures the mobile robot's execution of the corresponding subtask in real time and tags the video based on the information reported by the IWMS. This means that the IWMS-reported information and the tracing video are stored accordingly to facilitate subsequent refined indexing. For details, see the descriptions of steps S34, S44, and S54 above.

[0172] Step S68: The mobile robot notifies the RCS of the completion of the vehicle transport; the RCS notifies the RCMS of the arrival of the vehicle; and the RCMS notifies the IWMS of the arrival of the vehicle.

[0173] Step S69: The IWMS uses a handheld PDA (Personal Digital Assistant) or a workstation client to perform operations.

[0174] Step S610: The IWMS notifies the RCMS that the picking is completed, so that the RCMS issues a return task to the RCS. Then, the RCS dispatches the mobile robot to perform the return task, that is, to transport the vehicle back to the warehouse.

[0175] Next, Figure 7 The example shown is used as an example to illustrate the search interface involved in this application. Figure 7 It can be seen that the content on the page mainly includes two parts. The first part of the content is at the top of the search interface. This part of the content includes the four words "traceability order number", "belonging work station", "scene name", and "traceability time range", and there is an input box under each word. In addition, there are the words "query" and "reset" on the left side above the interface. It can be understood that the operator can enter the content to be searched in the input box at the top of the page, and then click the "query" button to realize the search. If you need to search again, you can click the "reset" button to clear the content in the input box to facilitate entering new search conditions.

[0176] The second section is located in the center of the search interface, and its content is organized in rows and columns. Each row can be understood as a traceability video. Specifically, the content in the middle is divided into seven columns: the first column is "Traceability Order Number," the second column is "Work Station," the third column is "Scenario Name," the fourth column is "Area," the fifth column is "Traceability Start Time," the sixth column is "Traceability End Time," and the seventh column is "Operation." Therefore, the second row of data in the second section represents a traceability video for order number Q221624E000152521A, work station RBS102-B05-2, any scenario name, and area 2F, Building B, Operation Desk. The start time of this traceability video is 2024-08-27 13:06:00, and the end time is 2024-08-27 13:14:05. Click the play button in the seventh column of the second row to play the traceability video represented in the first row. The meanings of other lines in the second part can be deduced similarly.

[0177] Based on the above embodiments of the present application, it can be seen that the video order tracing system provided by the embodiments of the present application can achieve the following beneficial effects:

[0178] 1) Compared with the traditional on-site picking model, the video order tracing system provided by the embodiment of the present application has a high degree of automation integration, strong real-time response capability, high fault tolerance, and can effectively improve warehouse operation efficiency and reduce warehouse personnel input;

[0179] 2) In the video order tracing system provided in the embodiment of the present application, the mobile robot records the entire process of order information during the operation phase. During each operation, the order information, location information, point information, item information, and drop information are pushed to the video acquisition device. Combined with the video acquisition device, the purpose of tracing orders quickly and accurately is achieved;

[0180] 3) Set up a video surveillance point in the entire robot operation area, such as a conveyor line workstation. When workers are doing inventory counting and inbound and outbound operations, they report operation information to the monitoring system at regular intervals or at key points. The monitoring system obtains employee operation information and marks the video for subsequent tracing.

[0181] 4) When the mobile robot moves in the warehouse, it reports information to the monitoring system in real time. The system records the robot's dynamics in real time and can capture the falling of goods in real time. When goods fall, the monitoring system can change the depth of field to record shelf information and send a message to the warehouse management staff in time for timely processing;

[0182] 5) The technical solution provided in the embodiment of the present application can customize the reported message type (i.e., customize the information used for indexing) to facilitate flexible adjustment of the tracing method, reduce the redundancy of the monitoring system collection, and achieve detailed tracing.

[0183] Corresponding to the above-mentioned video order tracing system, the embodiment of the present application also provides a video order tracing method, which is applied to the video order tracing system. The system includes a management device, a mobile robot and a video acquisition device. Specifically, Figure 8 As shown, the method may include the following steps:

[0184] Step S81: The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches a mobile robot to perform the transport task;

[0185] Step S82: The mobile robot performs the transport task under the scheduling of the management device; and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point it has reached to the management device;

[0186] Step S83: The management device sends the point information and the order information for identifying the transport order to the video capture device;

[0187] Step S84: The video acquisition device records the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly saves the tracing video and the received order information and point information.

[0188] In the technical solution provided by the embodiment of the present application, in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at each time it reaches a preset type of point to the management device; thereby, the management device sends the point information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information and point information. Thus, both the point information and the order information can be used as indexes for retrieving the tracing video, and there is no need to manually search for the required video clips from a large number of videos. In this way, the technical solution provided by the embodiment of the present application improves the efficiency of order tracing.

[0189] In some embodiments, in the process of performing a transport task, the mobile robot sends drop information indicating that an item has fallen to the management device in response to the item being transported falling.

[0190] The management device sends the drop information and order information to the video capture device;

[0191] The video acquisition device stores the traceability video and received order information, as well as the drop information;

[0192] In some embodiments, when the mobile robot is performing a transport task, whenever it reaches a preset type of point, it sends location information for identifying its own location to the management device;

[0193] The management device sends point information, location information, and order information to the video acquisition device;

[0194] The video acquisition device stores the traceability video and the received order information, point information, and location information.

[0195] In some embodiments, when the mobile robot is performing a transport task, whenever it reaches a preset type of point, it sends container information for identifying the container it is transporting to the management device;

[0196] The management device sends the location information, container information, and order information to the video acquisition device;

[0197] The video acquisition device stores the traceability video and the received order information, location information, and container information.

[0198] In some embodiments, the video order tracing system further includes an item collection device disposed at a point;

[0199] The item collection device collects item information of the items carried to the location by the mobile robot and sends the item information to the management device;

[0200] The management device sends the location information, item information, and order information to the video acquisition device;

[0201] The video acquisition device stores the traceability video and the received order information, location information, and item information.

[0202] In some embodiments, the item information includes any one or more of the following information: item identification, item quantity, and item image.

[0203] In some embodiments, the video order tracing system further includes a terminal scanning device provided at a point;

[0204] The terminal scanning device scans the items carried to the location by the mobile robot and obtains the order information of the order to which the items belong;

[0205] In some embodiments, the order information includes job type, order type, order number, item number, operator number, job start time, and job end time.

[0206] Corresponding to the above-mentioned video order tracing system, the embodiment of the present application further provides a video order tracing device, which is applied to the video order tracing system. The system includes a management device, a mobile robot and a video acquisition device; Figure 9 As shown, the device includes:

[0207] A task generation module 91 is used to drive the management device to respond to the transport order, generate a transport task for implementing the transport order, and schedule the mobile robot to perform the transport task;

[0208] The task execution module 92 is configured to drive the mobile robot to perform the transport task under the control of the management device; and in the process of performing the transport task, whenever the mobile robot reaches a preset point, it sends point information identifying the point it has reached to the management device;

[0209] An information sending module 93 is used to drive the management device to send the point information and order information for identifying the transport order to the video acquisition device;

[0210] The video generation module 94 is used to drive the video acquisition device to shoot the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly save the tracing video and the received order information and point information.

[0211] In the technical solution provided by the embodiment of the present application, in response to a transport order, a management device generates a transport task for implementing the transport order, and dispatches a mobile robot to perform the transport task; in the process of performing the transport task, the mobile robot sends point information for identifying the point it has arrived at each time it reaches a preset type of point to the management device; thereby, the management device sends the point information and order information to the video acquisition device, and the video acquisition device correspondingly saves the tracing video and the received order information and point information. Thus, both the point information and the order information can be used as indexes for retrieving the tracing video, and there is no need to manually search for the required video clips from a large number of videos. In this way, the technical solution provided by the embodiment of the present application improves the efficiency of order tracing.

[0212] In some embodiments, the task execution module 92 is further configured to drive the mobile robot to send a drop information indicating that an item has fallen to the management device in response to the item falling during the process of performing the transport task;

[0213] The information sending module 93 is also used to drive the management device to send the drop information and order information to the video acquisition device;

[0214] The video generation module 94 is also used to drive the video acquisition device to correspondingly save the traceability video and the received order information and drop information;

[0215] In some embodiments, the task execution module 92 is further configured to drive the mobile robot to send location information identifying its own location to the management device whenever it reaches a preset type of point during the process of performing the transport task;

[0216] The information sending module 93 is also used to drive the management device to send point information, location information, and order information to the video acquisition device;

[0217] The video generation module 94 is further used to drive the video acquisition device to correspondingly save the traceability video and the received order information, point information and location information.

[0218] In some embodiments, the task execution module 92 is further configured to drive the mobile robot to send container information identifying the container being carried to the management device whenever the mobile robot reaches a preset type of point during the process of performing the carrying task;

[0219] The information sending module 93 is also used to drive the management device to send the point information, container information, and order information to the video acquisition device;

[0220] The video generation module 94 is further used to drive the video acquisition device to correspondingly save the traceability video and the received order information, location information, and container information.

[0221] In some embodiments, the video order tracing system further includes an item collection device disposed at a point;

[0222] The task execution module 92 is further configured to drive the item collection device to collect item information of the items carried to the location by the mobile robot; and send the item information to the management device;

[0223] The information sending module 93 is also used to drive the management device to send point information, item information, and order information to the video acquisition device;

[0224] The video generation module 94 is also used to drive the video acquisition device to correspondingly save the traceability video and the received order information, location information, and item information.

[0225] In some embodiments, the item information includes any one or more of the following information: item identification, item quantity, and item image.

[0226] In some embodiments, the video order tracing system further includes a terminal scanning device provided at a point; the device further includes,

[0227] The information scanning module is used to drive the terminal scanning device to scan the items carried to the location by the mobile robot to obtain the order information of the order to which the items belong.

[0228] In some embodiments, the order information includes job type, order type, order number, item number, operator number, job start time, and job end time.

[0229] In another embodiment provided by the present application, a computer-readable storage medium is further provided, in which a computer program is stored. When the computer program is executed by a processor, the steps of any of the above-mentioned video order tracing systems are implemented.

[0230] In another embodiment provided by the present application, a computer program product including instructions is also provided, which, when executed on a computer, enables the computer to execute the steps of any video order tracing system in the above embodiments.

[0231] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present application is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website, computer, server or data center to another website, computer, server or data center via a wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) method. The computer-readable storage medium can be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrations. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a solid-state drive (SSD).

[0232] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0233] Each embodiment in this specification is described in a related manner. Similar portions between the various embodiments can be referenced to each other. Each embodiment focuses on the differences between the other embodiments. In particular, since the methods, apparatuses, computer-readable storage media, and computer program products are generally similar to the method embodiments, their descriptions are relatively simplified. For relevant portions, reference can be made to the descriptions of the method embodiments.

[0234] The above description is only a preferred embodiment of the present application and is not intended to limit the scope of protection of the present application. Any modifications, equivalent replacements, improvements, etc. made within the spirit and principles of the present application are included in the scope of protection of the present application.

Claims

1. A video order tracing system, characterized in that: The system includes a management device, a mobile robot and a video acquisition device; The management device is configured to generate, in response to a transport order, a transport task for implementing the transport order, and schedule the mobile robot to perform the transport task; The mobile robot is configured to perform the transport task under the scheduling of the management device; And in the process of executing the transport task, whenever the preset type of point is reached, the point information for identifying the point reached by the vehicle is sent to the management device; The management device is further configured to send the location information and order information for identifying the transport order to the video acquisition device; The video acquisition device is used to shoot the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly save the tracing video and the received order information and point information.

2. The system according to claim 1, wherein: The mobile robot is further configured to, in response to a falling object being carried, transmit drop information indicating that the object has fallen to the management device during the execution of the carrying task; The management device is specifically configured to send the drop information and the order information to the video acquisition device; The video acquisition device is specifically used to store the traceability video and the received order information and drop information.

3. The system according to claim 1, wherein: The mobile robot is further configured to send location information identifying its own location to the management device whenever it reaches a preset point type during the execution of the transport task; The management device is specifically used to send the point information, the location information, and the order information to the video acquisition device; The video acquisition device is specifically used to store the traceability video and the received order information, point information and location information.

4. The system according to claim 1, wherein: The mobile robot is further configured to send container information identifying the container being carried by the mobile robot to the management device whenever the mobile robot reaches a preset point during the execution of the carrying task; The management device is specifically used to send the point information, the container information, and the order information to the video acquisition device; The video acquisition device is specifically used to store the traceability video and the received order information, location information and container information.

5. The system according to claim 1, wherein: The system further includes an item collection device disposed at the point; The item collection device is used to collect item information of the items carried to the location by the mobile robot; and send the item information to the management device; The management device is specifically used to send the point information, the item information, and the order information to the video acquisition device; The video acquisition device is specifically used to store the traceability video and the received order information, location information and item information.

6. The system according to claim 5, characterized in that The item information includes any one or more of the following information: item identification, item quantity, and item image.

7. The system according to claim 1, wherein: The system further includes a terminal scanning device disposed at the point; The terminal scanning device is used to scan the items carried to the location by the movable robot to obtain order information of the order to which the items belong.

8. The system according to claim 7, characterized in that The order information includes the job type, order type, order number, item number, operator number, job start time, and job end time.

9. A video order tracing method, characterized in that: Applied to a video order tracing system, the system includes a management device, a mobile robot, and a video acquisition device, and the method includes: The management device generates a transport task for implementing the transport order in response to the transport order, and dispatches the mobile robot to perform the transport task; The mobile robot performs the transport task under the scheduling of the management device; and in the process of performing the transport task, whenever it reaches a preset type of point, it sends point information for identifying the point reached by itself to the management device; The management device sends the point information and the order information for identifying the transport order to the video acquisition device; The video acquisition device records the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly saves the tracing video and the received order information and location information.

10. A video order tracing device, characterized in that: Applied to a video order tracing system, the system includes a management device, a mobile robot, and a video acquisition device; the device includes: a task generation module, configured to drive the management device to generate a transport task for implementing the transport order in response to the transport order, and to dispatch the mobile robot to perform the transport task; a task execution module, configured to drive the mobile robot to perform the transport task under the scheduling of the management device; and, during the process of performing the transport task, whenever the mobile robot reaches a preset type of point, send point information for identifying the point reached by the mobile robot to the management device; An information sending module, configured to drive the management device to send the location information and order information for identifying the transport order to the video acquisition device; The video generation module is used to drive the video acquisition device to shoot the process of the mobile robot performing the handling task in real time to obtain a tracing video; and correspondingly save the tracing video and the received order information and point information.

11. The device according to claim 10, characterized in that The task execution module is further configured to drive the mobile robot to send drop information indicating that an item has dropped to the management device in response to the item being transported dropping during the execution of the transport task; the information sending module is further configured to drive the management device to send the drop information and the order information to the video acquisition device; the video generation module is further configured to drive the video acquisition device to correspondingly save the tracing video and the received order information, as well as the drop information; and / or, The task execution module is further configured to drive the mobile robot to send location information for identifying its own location to the management device whenever it reaches a preset type of point during the execution of the transport task; the information sending module is further configured to drive the management device to send the point information, the location information, and the order information to the video acquisition device; the video generation module is further configured to drive the video acquisition device to correspondingly save the tracing video and the received order information, point information, and location information; and / or, The task execution module is further configured to drive the mobile robot to send container information for identifying the container being carried to the management device whenever the mobile robot reaches a preset type of point during the execution of the carrying task; the information sending module is further configured to drive the management device to send the point information, the container information, and the order information to the video acquisition device; the video generation module is further configured to drive the video acquisition device to correspondingly save the tracing video and the received order information, point information, and container information; and / or, The system further includes an item collection device provided at the location; the task execution module is further configured to drive the item collection device to collect item information of the items transported to the location by the mobile robot; and send the item information to the management device; the information sending module is further configured to drive the management device to send the location information, the item information, and the order information to the video collection device; the video generation module is further configured to drive the video collection device to correspondingly save the tracing video and the received order information, location information, and item information; and / or, The item information includes any one or more of the following information: item identification, item quantity, item image; and / or, The system further includes a terminal scanning device provided at the location; the apparatus further includes an information scanning module for driving the terminal scanning device to scan items carried to the location by the mobile robot to obtain order information of the order to which the items belong; and / or, The order information includes the job type, order type, order number, item number, operator number, job start time, and job end time.

Images