Storage system control method and system and storage medium

Abstract

The invention discloses a control method of a warehousing system, the warehousing system and a storage medium, and the method comprises the steps: receiving a field image collected by a warehousing robot based on a visual sensor, and a field state parameter collected by a state sensor disposed at the warehousing robot and / or a warehousing field, and then carrying out the control of the warehousing system based on the field image; the method comprises the steps of generating a virtual field picture according to a field state parameter and a recognition identifier of a robot, sending the virtual field picture to a user side, receiving a user operation responded by the user side based on the virtual field picture, generating a robot control instruction based on an instruction conversion rule corresponding to the recognition identifier and the user operation, and sending the robot control instruction to the user side. And the control instruction is sent to the storage robot so as to control the storage robot to execute a corresponding action. Therefore, the problem that related technologies highly depend on rules is solved, and the effect of improving the adaptation degree between the solution and the problem is achieved.

Description

Technical Field

[0001] The present application relates to the field of data processing of a warehousing system, and in particular to a control method of a warehousing system, a warehousing system and a storage medium. Background Art

[0002] In modern warehousing scenarios, although artificial intelligence and robotics have significantly improved efficiency, this has also caused the intelligent warehousing system to be highly dependent on preset rules and difficult to deal with sudden anomalies without historical data. When faced with some non-standardized problems, the need for manpower to make emergency responses based on comprehensive considerations still exists. For example, when a warehouse catches fire, the cause and condition of the fire are unpredictable, and the same situation usually does not recur. When the system has no historical data as a basis, it is difficult to make effective emergency responses based on the current situation. Therefore, when faced with non-standardized problems, the decision-making solutions provided by the warehousing systems provided by related technologies are poorly adapted to actual problems. Summary of the invention

[0003] The main purpose of this application is to provide a control method for a warehousing system, a warehousing system and a storage medium, aiming to solve the technical problem in the related art that the warehousing system is too dependent on rules.

[0004] To achieve the above-mentioned purpose, an embodiment of the present application provides a control method of a warehousing system, which is applied to a control terminal, one side of which is communicatively connected to a warehousing robot, and the other side of which is communicatively connected to a user terminal. The control method of the warehousing system includes:

[0005] Receiving the on-site image collected by the storage robot based on the visual sensor, and the on-site status parameters collected by the status sensors arranged on the storage robot and / or the storage site;

[0006] Generate a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and send the virtual scene picture to the user terminal;

[0007] Receiving a user operation of the user terminal based on the virtual scene image response, and generating a robot control instruction based on an instruction conversion rule corresponding to the identification mark and the user operation;

[0008] The control instruction is sent to the storage robot to control the storage robot to perform corresponding actions.

[0009] In the embodiment of the present application, the step of receiving the user operation of the user terminal based on the virtual scene screen response, and generating the robot control instruction based on the instruction conversion rule corresponding to the identification mark and the user operation includes:

[0010] Determine a task objective according to the user operation;

[0011] Determine the executable action of the storage robot based on the identification mark, and obtain the instruction conversion rule;

[0012] Decomposing the task objective according to the task objective and the executable actions to generate at least one subtask;

[0013] In the instruction conversion rule, the robot control instruction corresponding to the subtask is retrieved.

[0014] In the embodiment of the present application, the step of determining the task target according to the user operation includes:

[0015] Determine, according to the user operation and the virtual scene picture, a state change of at least one of a virtual commodity, a virtual device, and a virtual equipment in the virtual scene picture;

[0016] The task objective is determined according to the state change.

[0017] In the embodiment of the present application, the step of determining the task target according to the state change includes:

[0018] If the virtual cargo position status is updated in the virtual scene screen, it is determined that the task is to move the cargo;

[0019] Determining the initial position and the end position of the virtual goods;

[0020] The task objective is set to move the goods in the warehouse from the initial position to the end position.

[0021] In the embodiment of the present application, the step of determining the task target according to the state change includes:

[0022] If the trigger state of the virtual device is updated, the task target is set to update the trigger state of the alarm device corresponding to the virtual device.

[0023] In the embodiment of the present application, the step of determining the task target according to the state change includes:

[0024] If the state of the virtual device changes, obtaining the disassembly steps and / or assembly steps of the virtual device;

[0025] The task target is set as a maintenance task, and operation steps corresponding to the maintenance task are generated based on the disassembly step and / or the assembly step.

[0026] In the embodiment of the present application, the step of determining the state change of at least one of the virtual goods, virtual devices, and virtual equipment in the virtual scene according to the user operation and the virtual scene includes:

[0027] Determine an operation object corresponding to the task target based on the current mode, wherein the operation object includes at least one of goods, an alarm device, and equipment to be repaired;

[0028] The step of determining a state change of at least one of virtual goods, virtual devices, and virtual equipment in the virtual scene screen according to the user operation and the virtual scene screen is performed based on the operation object.

[0029] In the embodiment of the present application, before the step of determining the operation object corresponding to the task target based on the current mode, the method further includes:

[0030] Sending a mode selection interface to the user terminal, wherein the mode selection interface includes at least one optional mode among a shipping mode, an alarm mode and a maintenance mode

[0031] A mode selection result of the user terminal based on the mode selection interface response is received, and the current mode is determined according to the mode selection result.

[0032] The present application also provides a storage system, which includes:

[0033] A receiving module, used to receive the on-site image collected by the storage robot based on the visual sensor, and the on-site status parameters collected by the status sensors arranged on the storage robot and / or the storage site;

[0034] A data processing module, for generating a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and sending the virtual scene picture to the user terminal; and receiving a user operation of the user terminal based on the virtual scene response, and generating a robot control instruction based on an instruction conversion rule corresponding to the identification mark and the user operation;

[0035] The sending module is used to send the control instruction to the storage robot to control the storage robot to perform corresponding actions.

[0036] An embodiment of the present application further provides a storage medium, which is a computer-readable storage medium. A computer program is stored on the storage medium. When the computer program is executed by a processor, the steps of the control method of the warehousing system as described above are implemented.

[0037] An embodiment of the present application discloses a control method for a warehousing system, wherein the control method for the warehousing system is configured such that the control end first receives a scene image and status parameters, generates a virtual scene picture based on the scene image, the scene status parameters and an identification mark of a robot, and sends the virtual scene picture to the user end, and then generates a robot control instruction according to the user operation collected by the user end and the instruction conversion rules corresponding to the above identification mark, thereby achieving the purpose of controlling the warehousing robot to realize the artificial control intention through the robot control instruction.

[0038] In the above method, since the intelligent terminal can generate robot control instructions based on the instruction conversion rules and the user's operations on the virtual scene screen, when a new warehouse robot is added or the warehouse robot is updated, the method can realize the control of any warehouse robot by simply updating the corresponding instruction conversion rules. There is no need to re-arrange and design a new remote control system. At the same time, since the virtual scene screen is generated based on the scene image and scene status parameters, the remote controller can obtain more accurate on-site conditions based on it, and then make more accurate remote control actions. In addition, the solution achieves the purpose of remotely controlling the warehouse robot under specific circumstances, so that the warehouse system can make manual decisions and realize equipment control when facing non-standardized problems. This improves the adaptability of the decision-making plan to the actual problem to a certain extent. BRIEF DESCRIPTION OF THE DRAWINGS

[0039] Figure 1 It is a schematic diagram of the architecture of the storage system involved in the embodiment of the present application;

[0040] Figure 2 It is a flow chart of an embodiment of a control method for a storage system according to an embodiment of the present application;

[0041] Figure 3 This is a schematic diagram of the structure of the control device of the warehousing system of this application;

[0042] Figure 4 This is a schematic diagram of the modular structure of the warehousing system for this application.

[0043] The realization of the purpose, functional features and advantages of this application will be further explained in conjunction with embodiments and with reference to the accompanying drawings. DETAILED DESCRIPTION

[0044] It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0045] In modern warehousing scenarios, although artificial intelligence and robotics have significantly improved efficiency, this has also caused the intelligent warehousing system to be highly dependent on preset rules and difficult to deal with sudden anomalies without historical data. When faced with some non-standardized problems, the need for manpower to make emergency responses based on comprehensive considerations still exists. For example, when a warehouse catches fire, the cause and condition of the fire are unpredictable, and the same situation usually does not recur. When the system has no historical data as a basis, it is difficult to make effective emergency responses based on the current situation. Therefore, when faced with non-standardized problems, the decision-making solutions provided by the warehousing systems provided by related technologies are poorly adapted to actual problems.

[0046] In the related art, there are also some solutions that can remotely control storage equipment or storage robots to perform corresponding work tasks. For example, a communication module and a camera are set on the smart forklift, and the on-site image is collected by the camera and displayed on the computer. At the same time, a control interface is arranged on the computer, so that the user can remotely send corresponding control instructions to the smart forklift through the control interface. Alternatively, for a storage robot, its travel video can be customized on the remote control terminal so that it can travel according to the manually set route. However, the above scheme still has defects.

[0047] First, existing remote control solutions all require the development of corresponding control systems and command synchronization systems for designated equipment. Therefore, when any equipment in the intelligent warehousing system is updated or any new equipment is added, a corresponding remote control system needs to be developed for the equipment. Such a solution will make the overall system increasingly complicated during the update and iteration process, which will increase the probability of failure and the difficulty of troubleshooting when failure occurs. Secondly, if a new system needs to be developed every time an equipment is updated, there is a defect of high development cost.

[0048] Second, the remote control solutions in related technologies are all based on surveillance video rather than the first-person perspective of the device. Moreover, simple video solutions will ignore many on-site details, causing the remote operator to make wrong operations or decisions.

[0049] To solve the above-mentioned defects existing in the related art, an embodiment of the present application provides a control method for a warehousing system implemented based on a new warehousing system. The warehousing system is provided with a warehousing robot, a control terminal and a user terminal which are connected in sequence. Among them, the user terminal is configured to display on-site information and obtain user control operations, and the control terminal is used to convert user operations into instructions executable by the warehousing robot, and the warehousing robot implements the user's control intention based on the instructions. The control method of the warehousing system is configured such that the control terminal first receives the on-site image and state parameters, and then generates a virtual on-site picture based on the on-site image, on-site state parameters and the robot's identification mark, and sends the virtual on-site picture to the user terminal, and then generates a robot control instruction according to the user operation collected by the user terminal and the instruction conversion rules corresponding to the above identification mark, thereby achieving the purpose of controlling the warehousing robot to achieve the artificial control intention through the robot control instruction.

[0050] In the above method, since the intelligent terminal can generate robot control instructions based on the instruction conversion rules and the user's operations on the virtual scene screen, when a new warehouse robot is added or the warehouse robot is updated, the method can realize the control of any warehouse robot by simply updating the corresponding instruction conversion rules. There is no need to re-arrange and design a new remote control system. At the same time, since the virtual scene screen is generated based on the scene image and scene status parameters, the remote controller can obtain more accurate on-site conditions based on it, and then make more accurate remote control actions. In addition, the solution achieves the purpose of remotely controlling the warehouse robot under specific circumstances, so that the warehouse system can make manual decisions and realize equipment control when facing non-standardized problems. This improves the adaptability of the decision-making plan to the actual problem to a certain extent.

[0051] For ease of understanding, the technical solutions in the embodiments of the present application will be described below in conjunction with the drawings in the embodiments of the present application.

[0052] Please refer to Figure 1, an embodiment of the present application provides a warehousing system, which includes a control terminal, a warehousing robot terminal and a user terminal. Among them, the user terminal includes virtual reality (VR, Virtual Reality), so that when the user terminal receives the virtual scene picture sent by the control terminal, it can be realized through the virtual reality device. The user terminal is also provided with a user operation acquisition device. Among them, the user operation acquisition device can be a handle, or a user behavior detection device. Among them, the user behavior detection device can determine the user's intention by detecting the user's body movements. Then, the user can control the control object in the virtual scene picture through body movements and perform corresponding actions in the virtual scene picture. Alternatively, the user operation acquisition device can also be other devices that can collect the user's operation intentions so that the user can interact with virtual objects in the virtual scene picture. This embodiment does not limit this.

[0053] Optionally, the control end includes a warehouse robot side communication module, a data processing module and a user-end communication module. Among them, the warehouse robot communication module is deployed with one or more communication methods. The control terminal can be connected to different robots based on user selection. It can be understood that the warehouse robot side communication module can establish communication with one or more machines at the same time. Moreover, the warehouse robots that establish communication with it at the same time can be robots of the same type or robots of different types. The data processing module is used to generate a virtual scene picture based on the data received by the warehouse robot side communication module, and send it to the user end through the user-end communication module. It is also configured to generate control instructions based on the user operation received by the user-end communication module, and send it to the warehouse robot through the warehouse robot side communication module.

[0054] It should be noted that in a remote control scenario, a warehouse robot entity can establish communication with a user-end entity to form a communication link between the warehouse robot, the control end, and the user end. The control end can also connect to multiple warehouse robots through the warehouse robot side communication module, and connect to multiple user ends through the user end communication module. This allows multiple warehouse robot-control end-user end communication links to exist at the same time. That is, in a remote control scenario, the warehouse robot corresponds to the user end one-to-one. The control end and the warehouse robot, or the control end and the user end can both be one-to-many communications.

[0055] Optionally, the storage robot may be a storage device that can execute remote control instructions, such as a robotic arm, an autonomous mobile robot, an inventory drone, a cross-belt sorter, a shuttle car, or an intelligent forklift, etc., which can be remotely controlled storage devices.

[0056] Please refer to Figure 1In an optional implementation scheme, the control method of the storage system includes the following steps:

[0057] The control method of the warehousing system provided in this embodiment can be executed on the control end of the warehousing system, which can be a background server or a distributed data processing node. In one embodiment, the control method of the warehousing system includes steps S10 to S40.

[0058] S10: receiving the on-site image collected by the storage robot based on the visual sensor, and the on-site status parameters collected by the status sensors arranged on the storage robot and / or the storage site.

[0059] The storage system is provided with an automatic mode and a passive mode. The automatic mode and the passive mode can be triggered based on conditions or can be triggered actively by the user. When the passive mode is triggered, the control node first sends a data acquisition request to the storage robot corresponding to the currently triggered passive mode, that is, the controlled object in the subsequent process, to obtain the scene image collected by the visual sensor, and sends a data acquisition request to the communication module corresponding to the state sensor, to obtain the scene state parameters collected by the state sensor arranged on the storage robot and / or the storage site.

[0060] It should be noted that in some variant implementation schemes, the collection of on-site images can also be based on other image sensors in the space where the controlled object is located, and it is not limited to the image acquisition sensor being arranged only on the storage robot. The solution provided in this embodiment is described by controlling an autonomous mobile robot as an example. Therefore, in order to provide a better first visual experience, it is preferred to use the on-site images collected by the storage robot based on the visual sensor, but this is not used to limit it from using the on-site images collected by the visual sensors deployed at other locations. For example, for a cross sorting machine, it is only necessary to determine the channel in which the goods flow during the control process. Therefore, the images collected by the cameras deployed at a higher level may have a better visual experience, and its data stream can be set to the images collected by the cameras outside its body. The main purpose of the solution given in this embodiment is to determine which device collects the data stream as the on-site image collected by the storage robot based on the visual sensor through a pre-set image data stream correspondence. It is not limited to the specific deployment position of the visual sensor.

[0061] S20: generating a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and sending the virtual scene picture to the user terminal;

[0062] After receiving the live image, data preprocessing can be performed first, that is, based on the camera internal parameter matrix, the barrel distortion of the wide-angle lens can be eliminated. Then the depth map is converted into a three-dimensional point cloud (XYZ coordinates + RGB colors) and downsampled to n cm accuracy using voxel filtering. Where n can be set to 3CM-15CM based on the system computing power. Multi-sensor spatiotemporal synchronization is then achieved through hardware timestamps (PTP protocol).

[0063] After completing the data preprocessing, modeling can be performed based on the environment semantics and dynamic objects can be tracked, so that spatial mapping and rendering can be performed based on the modeling and tracking results.

[0064] For example, after obtaining the scene image and scene state parameters, the data collected by multiple sensors can be synchronized through data preprocessing to achieve data spatiotemporal synchronization. Then, modeling is performed based on the synchronized data stream to construct a scene real scene model. Then, in order to have a good visual experience, the pre-established robot model can be obtained based on the robot's identification mark, and the virtual and real space mapping (the robot is virtual, and the scene is real) is performed to render a virtual scene picture.

[0065] After the virtual scene picture is generated, it is updated in the form of streaming data based on the real-time collected scene images and status parameters, and sent to the client in the form of a data stream, so that when the client is performing reality based on the VR device, the picture seen can change with the real changes on the scene.

[0066] It should be noted that the field status parameters mentioned in this embodiment can be any other sensor parameters used to enrich the virtual field screen. There is no specific limitation here. For example, it can be the robot body state parameters: such as position, posture, speed, mechanical arm joint angle, etc., which directly affect the movement and behavior of the virtual robot model. Environmental physical parameters: such as light, temperature and humidity, sound, which affect the visual and auditory performance of the virtual environment. Dynamic object parameters: such as the position and speed of other mobile devices (AGV, forklift), and the status of the goods (whether it is being transported and placed), which ensure that the dynamic elements in the virtual environment are synchronized with the real one. Sensor data: such as obstacle information of distance sensors and contact feedback of force / torque sensors, which enhance the realism and safety of interaction. Time synchronization parameters: ensure that the timestamps of all data are consistent to avoid delays or asynchrony in virtual images.

[0067] S30: receiving a user operation of the user terminal based on the virtual scene response, and generating a robot control instruction based on the instruction conversion rule corresponding to the identification mark and the user operation;

[0068] After sending the virtual scene image to the user end, the user end can collect the user's operation based on the user behavior detection device. And send the collected user operation to the control end. So that the control end can determine the task target according to the user operation, and then determine the executable action of the storage robot based on the identification mark, and obtain the instruction conversion rule, and decompose the task target according to the task target and the executable action, generate at least one subtask, and finally retrieve the robot control instruction corresponding to the subtask in the instruction conversion rule.

[0069] It should be noted that the above-mentioned task objectives refer to the user inputting control actions through the user behavior detection device based on the virtual scene screen, thereby controlling the actions performed by the robot in the virtual scene screen. Since the interactive behaviors that the user can control are limited based on the generated virtual scene screen, after receiving the user operation, the control terminal can control the virtual model (i.e., the virtual model of the warehouse robot) to perform actions according to the user operation. However, this action is currently not mapped to the actual scene and only occurs in the virtual world. Therefore, the control terminal also needs to convert it into a robot control instruction that can be executed by the warehouse robot, and send it to the robot for execution before the action can be completed in the actual space.

[0070] The behavior mapping between virtual and real space is not realized. This embodiment provides two optional implementation schemes. One is real-time control based on user manipulation behavior, and the other is segmented control based on user manipulation behavior. The two schemes can be selectively set during implementation based on different warehouse robots and application scenarios.

[0071] First, in the implementation of the control scheme, a real-time control data stream is generated according to the user's real-time control behavior to control the warehouse robot and synchronously perform the corresponding actions. For example, when the user controls the robot in the virtual scene to raise its right arm, the control terminal detects the movement trajectory and speed of the right arm. Based on the movement trajectory and speed, the corresponding real-time control instructions are generated through the instruction conversion rules. The real-time control instructions are then sent to the warehouse robot to control it to raise its right arm synchronously. Therefore, the task goal here is to raise the right arm, and the executable actions are the amplitude and angle of rotation of each joint. The generated robot control instructions are to control the motors corresponding to each joint to respond. Such a solution has better real-time performance, but it requires higher computing power for the system.

[0072] Secondly, in the segmented control scheme, the storage robot and the virtual robot in the virtual scene screen do not run synchronously. That is, according to the user operation and the virtual scene screen, the state change of at least one of the virtual goods, virtual devices, and virtual equipment in the virtual scene screen is determined, and the task target is determined according to the state change.

[0073] Taking the equipment maintenance scenario as an example, if the state of the virtual device changes, the disassembly steps and / or assembly steps of the virtual device can be obtained first, and then the task target can be set as a maintenance task, and the operation steps corresponding to the maintenance task can be generated based on the disassembly steps and / or the assembly steps. Among them, the equipment to be repaired exists in the virtual scene in the form of a virtual model. It allows users to repair the virtual equipment to be repaired based on interactive operations. When the entire maintenance process or part of the maintenance process is completed. The control end records the steps in the maintenance process. Then the maintenance task is decomposed according to the steps. A series of robot control instructions are generated and sent to the execution object in the form of an instruction stream. The execution object can control itself to complete the maintenance process by executing the instructions in the instruction stream in sequence.

[0074] It should be noted that in the maintenance scenario, the virtual scene screen may include at least two virtual models. One is the virtual model corresponding to the equipment to be repaired. The scene image used for the virtual scene screen can be collected in real time or in advance. In this way, by reducing the real-time nature of the operation, a longer available computing time is obtained. In this way, the system can reduce the computing power requirements in application scenarios where the computing power is insufficient or high real-time performance is not required.

[0075] Optionally, in an alarm scenario, a virtual device corresponding to a real alarm device can be set based on a virtual scene screen. This allows the user to trigger the virtual device in the virtual scene screen by interacting with the virtual scene screen. When the control terminal receives an action to trigger the virtual device, it sets the task target to update the trigger state of the alarm device corresponding to the virtual device. Then, the trigger instruction can be sent to the corresponding real alarm device to trigger an alarm in the alarm system through the alarm device. Of course, in some optional schemes, if the control terminal can directly communicate with the alarm system to trigger an alarm, then after the virtual device is triggered, an alarm can also be directly triggered in the alarm system. This embodiment does not limit this.

[0076] Optionally, in the cargo handling scenario, the user can first control the virtual cargo in the virtual scene to move in the virtual scene. Then the system can trigger synchronization based on the completion of an action, or it can be set to trigger synchronization based on a preset synchronization time. The preset synchronization time can be customized to 5s to 60s according to needs. For example, it is set to 10s, 20s, 30s or 45s, etc. So that the control section can determine the starting position and end position of the virtual cargo in the virtual scene when the preset synchronization time is reached. Then based on the revealed position and end position, generate the movement task of the transfer robot in the future time period. And after generating the instruction set based on the movement task, it is sent to the transfer robot to control it to move from the starting position to the end position in the real scene.

[0077] Optionally, in this embodiment, before generating the virtual scene picture, the current mode of the user terminal can also be determined. Because in different modes, the necessary virtual objects in the virtual scene picture are different. Through the current mode, unnecessary virtual models in the virtual scene picture can be filtered out. That is, the operation object corresponding to the task target is determined based on the current mode, wherein the operation object includes at least one of goods, alarm devices and equipment to be repaired. Then, in the rendering process of the virtual scene picture, the operation object is rendered as a virtual object. In this way, it is possible to perform the state change of at least one of the virtual goods, virtual devices and virtual equipment in the virtual scene picture based on the operation object according to the user operation and the virtual scene picture. At the same time, this can also effectively reduce the amount of calculation in the process of generating the virtual scene picture.

[0078] Optionally, the mode of the user terminal can be customized. For example, a mode selection interface can be sent to the user terminal first, wherein the mode selection interface includes at least one optional mode of a shipping mode, an alarm mode, and a maintenance mode, and then a mode selection result of the user terminal based on the mode selection interface response is received, and the current mode is determined according to the mode selection result.

[0079] In some other optional implementation schemes, the user's authority can be determined based on the user's login information, and the user's mode can be automatically selected based on the user's authority. Or the user's mode can be automatically selected based on some other mode selection conditions. For example, the image content can be recognized for the scene image. Then, the user's mode is automatically selected based on the matching degree between the image content recognition result and the mode. Of course, other schemes can also be set, which will not be repeated here.

[0080] In this embodiment, the control method of the warehousing system is configured such that the control end first receives the on-site image and status parameters, generates a virtual on-site picture based on the on-site image, on-site status parameters and the identification mark of the robot, and sends the virtual on-site picture to the user end, and then generates robot control instructions according to the user operations collected by the user end and the instruction conversion rules corresponding to the above identification marks, thereby achieving the purpose of controlling the warehousing robot to realize the artificial control intention through the robot control instructions.

[0081] In the above method, since the intelligent terminal can generate robot control instructions based on the instruction conversion rules and the user's operations on the virtual scene screen, when a new warehouse robot is added or the warehouse robot is updated, the method can realize the control of any warehouse robot by simply updating the corresponding instruction conversion rules. There is no need to re-arrange and design a new remote control system. At the same time, since the virtual scene screen is generated based on the scene image and scene status parameters, the remote controller can obtain more accurate on-site conditions based on it, and then make more accurate remote control actions. In addition, the solution achieves the purpose of remotely controlling the warehouse robot under specific circumstances, so that the warehouse system can make manual decisions and realize equipment control when facing non-standardized problems. This improves the adaptability of the decision-making plan to the actual problem to a certain extent.

[0082] The present application provides a control device for a warehousing system, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor so that the at least one processor can execute the control method for the warehousing system in the above-mentioned embodiment one.

[0083] Reference below Figure 3 , which shows a schematic diagram of the structure of a control device of a warehousing system suitable for implementing the embodiment of the present application. The control device of the warehousing system in the embodiment of the present application may include, but is not limited to, a server and a computer. Figure 3 The control device of the warehousing system shown is merely an example and should not bring any limitation to the functions and scope of use of the embodiments of the present application.

[0084] like Figure 3As shown, the control device of the storage system may include a processing device 1001 (e.g., a central processing unit, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 to a random access memory (RAM) 1004. In the random access memory 1004, various programs and data required for the operation of the control device of the storage system are also stored. The processing device 1001, the read-only memory 1002, and the random access memory 1004 are connected to each other through a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Generally, the following systems can be connected to the input / output interface 1006: an input device 1007 including, for example, a touch screen, a touch pad, etc.; an output device 1008 including, for example, a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage device 1003 including, for example, a hard disk, etc.; and a communication device 1009. The communication device 1009 can allow the control device of the storage system to communicate with other devices wirelessly or by wire to exchange data. Although the control device of the storage system with various systems is shown in the figure, it should be understood that it is not required to implement or have all the systems shown. More or fewer systems can be implemented or provided instead.

[0085] In particular, according to the embodiments disclosed in the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, the embodiments disclosed in the present application include a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a program code for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from a network through a communication device, or installed from a storage device 1003, or installed from a read-only memory 1002. When the computer program is executed by the processing device 1001, the above-mentioned functions defined in the method of the embodiment disclosed in the present application are executed.

[0086] The control device of the storage system provided by the present application adopts the control method of the storage system in the above embodiment to solve the technical problem that the solution provided by the storage system is not compatible with the actual problem. Compared with the related art, the beneficial effects of the control device of the storage system provided by the present application are the same as the beneficial effects of the control method of the storage system provided by the above embodiment, and the other technical features of the control device of the storage system are the same as the features disclosed in the method of the previous embodiment, which will not be repeated here.

[0087] It should be understood that the various parts disclosed in this application can be implemented by hardware, software, firmware or a combination thereof. In the description of the above embodiments, specific features, structures, materials or characteristics can be combined in any one or more embodiments or examples in a suitable manner.

[0088] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any person skilled in the art who is familiar with the present technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

[0089] Please refer to Figure 4 The present application provides a storage system, the storage system 100 comprising:

[0090] A receiving module 110 receives a scene image collected by the storage robot based on a visual sensor, and a scene state parameter collected by a state sensor arranged on the storage robot and / or the storage site;

[0091] The data processing module 120 generates a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and sends the virtual scene picture to the user terminal; and receives the user operation of the user terminal based on the virtual scene picture, and generates a robot control instruction based on the instruction conversion rule corresponding to the identification mark and the user operation;

[0092] The sending module 130 sends the control instruction to the storage robot to control the storage robot to perform corresponding actions.

[0093] The present application provides a computer-readable storage medium having computer-readable program instructions (ie, computer programs) stored thereon, and the computer-readable program instructions are used to execute the control method of the warehousing system in the above-mentioned embodiment.

[0094] The computer-readable storage medium provided in the present application may be, for example, a USB flash drive, but is not limited to electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, systems or devices, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM, Erasable Programmable Read Only Memory or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM, CD-Read Only Memory), an optical storage device, a magnetic storage device, or any suitable combination of the above. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in combination with an instruction execution system, system or device. The program code contained on the computer-readable storage medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination of the above.

[0095] The computer-readable storage medium may be included in the control device of the warehousing system; or may exist independently without being assembled into the control device of the warehousing system.

[0096] The computer-readable storage medium carries one or more programs. When the one or more programs are executed by the control device of the warehousing system, the control device of the warehousing system can improve the warehousing efficiency of the warehouse based on the method.

[0097] Computer program code for performing the operations of the present application may be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a separate software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0098] The flow chart and block diagram in the accompanying drawings illustrate the possible architecture, function and operation of the system, method and computer program product according to various embodiments of the present application. In this regard, each square box in the flow chart or block diagram can represent a module, a program segment or a part of a code, and the module, the program segment or a part of the code contains one or more executable instructions for realizing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the square box can also occur in a sequence different from that marked in the accompanying drawings. For example, two square boxes represented in succession can actually be executed substantially in parallel, and they can sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each square box in the block diagram and / or flow chart, and the combination of the square boxes in the block diagram and / or flow chart can be implemented with a dedicated hardware-based system that performs a specified function or operation, or can be implemented with a combination of dedicated hardware and computer instructions.

[0099] The modules involved in the embodiments described in the present application may be implemented by software or hardware, wherein the name of the module does not constitute a limitation on the unit itself in some cases.

[0100] The readable storage medium provided by the present application is a computer-readable storage medium, which stores computer-readable program instructions (i.e., computer programs) for executing the control method of the above-mentioned storage system, and can solve the technical problem that the solution provided by the storage system is not compatible with the actual problem. Compared with the related art, the beneficial effects of the computer-readable storage medium provided by the present application are the same as the beneficial effects of the control method of the storage system provided by the above-mentioned embodiment, and will not be repeated here.

[0101] An embodiment of the present application provides a computer program product, including a computer program, which implements the steps of the control method of the warehousing system as described above when the computer program is executed by a processor.

[0102] The computer program product provided in this application can solve the technical problem that the solution provided by the storage system is not compatible with the actual problem. Compared with the related art, the beneficial effects of the computer program product provided in the embodiment of this application are the same as the beneficial effects of the control method of the storage system provided in the above embodiment, which will not be repeated here.

[0103] The above are only preferred embodiments of the present application, and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made by using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent scope of the present application.

[0104] It should be noted that, in this article, the terms "include", "comprises" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or system including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or system. In the absence of further restrictions, an element defined by the sentence "including a ..." does not exclude the existence of other identical elements in the process, method, article or system including the element. In the intervals given in this application, the boundary values ​​are all included unless explicitly defined.

[0105] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method.

[0106] The above are only preferred embodiments of the present application, and are not intended to limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A control method for a warehousing system, characterized in that: Applied to a control end, one side of the control end is connected to a storage robot for communication, and the other side of the control end is connected to a user end for communication. The control method of the storage system includes: Receiving the on-site image collected by the storage robot based on the visual sensor, and the on-site status parameters collected by the status sensors arranged on the storage robot and / or the storage site; Generate a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and send the virtual scene picture to the user terminal; Receiving a user operation of the user terminal based on the virtual scene image response, and generating a robot control instruction based on an instruction conversion rule corresponding to the identification mark and the user operation; The control instruction is sent to the storage robot to control the storage robot to perform corresponding actions.

2. The control method of the storage system according to claim 1, characterized in that: The step of receiving the user operation of the user terminal based on the virtual scene image response and generating the robot control instruction based on the instruction conversion rule corresponding to the identification mark and the user operation includes: Determine a task objective according to the user operation; Determine the executable action of the storage robot based on the identification mark, and obtain the instruction conversion rule; Decomposing the task objective according to the task objective and the executable actions to generate at least one subtask; In the instruction conversion rule, the robot control instruction corresponding to the subtask is retrieved.

3. The control method of the storage system according to claim 2, characterized in that: The step of determining the task target according to the user operation comprises: Determine, according to the user operation and the virtual scene picture, a state change of at least one of a virtual commodity, a virtual device, and a virtual equipment in the virtual scene picture; The task objective is determined according to the state change.

4. The control method of the storage system according to claim 3, characterized in that: The step of determining the task target according to the state change comprises: If the virtual cargo position status is updated in the virtual scene screen, it is determined that the task is to move the cargo; Determining the initial position and the end position of the virtual goods; The task objective is set to move the goods in the warehouse from the initial position to the end position.

5. The control method of the storage system according to claim 3, characterized in that: The step of determining the task target according to the state change comprises: If the trigger state of the virtual device is updated, the task target is set to update the trigger state of the alarm device corresponding to the virtual device.

6. The control method of the storage system according to claim 3, characterized in that: The step of determining the task target according to the state change comprises: If the state of the virtual device changes, obtaining the disassembly steps and / or assembly steps of the virtual device; The task target is set as a maintenance task, and operation steps corresponding to the maintenance task are generated based on the disassembly step and / or the assembly step.

7. The control method of the storage system according to claim 3, characterized in that: The step of determining a state change of at least one of a virtual commodity, a virtual device, and a virtual equipment in the virtual scene according to the user operation and the virtual scene includes: Determine an operation object corresponding to the task target based on the current mode, wherein the operation object includes at least one of goods, an alarm device, and equipment to be repaired; The step of determining a state change of at least one of virtual goods, virtual devices, and virtual equipment in the virtual scene screen according to the user operation and the virtual scene screen is performed based on the operation object.

8. The control method of the storage system according to claim 7, characterized in that: Before the step of determining the operation object corresponding to the task target based on the current mode, the method further includes: Sending a mode selection interface to the user terminal, wherein the mode selection interface includes at least one optional mode among a shipping mode, an alarm mode and a maintenance mode A mode selection result of the user terminal based on the mode selection interface response is received, and the current mode is determined according to the mode selection result.

9. A storage system, characterized in that: The storage system comprises: A receiving module, used to receive the on-site images collected by the storage robot based on the visual sensor, and the on-site status parameters collected by the status sensors arranged on the storage robot and / or the storage site; A data processing module, for generating a virtual scene picture based on the scene image, the scene state parameters and the identification mark of the robot, and sending the virtual scene picture to the user end; and receiving a user operation of the user end based on the virtual scene response, and generating a robot control instruction based on an instruction conversion rule corresponding to the identification mark and the user operation; The sending module is used to send the control instruction to the storage robot to control the storage robot to perform corresponding actions.

10. A storage medium, characterized in that: The storage medium is a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the steps of the control method of the warehousing system as described in any one of claims 1 to 8 are implemented.

Images