Safety monitoring system, method and equipment of robot and medium

Abstract

The invention provides a safety monitoring system, method and equipment for a robot and a medium, and the system comprises a monitoring region setting module which is used for setting a safety monitoring region which is used for carrying out the safety protection of a working region of the robot; the point cloud projection module is used for acquiring a point cloud of the safety monitoring area and projecting the point cloud into a pixel plane, and the pixel plane comprises a depth value of each point; the point cloud monitoring module is used for continuously monitoring the point cloud of the safety monitoring area and mapping the monitored point cloud to the pixel plane; and the intrusion judgment module is used for comparing the depth values of the plurality of monitored point clouds and the adjacent domains with the depth values of the corresponding pixels of the pixel plane to determine whether an invaded moving object exists in the safety monitoring area. Safe operation of the mechanical arm can be achieved, and collision between the mechanical arm and other objects is avoided.

Description

Technical Field

[0001] This invention relates to mobile robots, and more specifically, to a safety monitoring system, method, device, and medium for robots. Background Technology

[0002] Stacking is indispensable in industries such as retail, manufacturing, and logistics. It refers to arranging materials of different sizes and shapes on a pallet for storage, transportation, or distribution. The goal is to efficiently build a stable stack, maximizing pallet space utilization while ensuring the materials are not damaged. Manual stacking is slow and inefficient, and manually handling heavy boxes poses significant safety risks. Therefore, the demand for robotic stacking is increasing.

[0003] One of the main challenges of traditional robotic arm offline palletizing is the need for significant investment in infrastructure, particularly in sequencers, to ensure materials arrive in the correct order for palletizing operations. Mobile robot palletizing solutions, on the other hand, do not require a specific material arrival order and can perform stable, densely packed palletizing operations, making them more flexible and cost-effective than traditional palletizing robots.

[0004] Mobile robots can solve the problem of high robot installation costs and cope with the challenges of varying sorting line traffic. The number of mobile robots can be increased or decreased according to different business needs, and they can be customized and deployed at the end of various conveyor lines. However, how to ensure the safety of mobile robots is a problem that urgently needs to be solved. Summary of the Invention

[0005] In view of the deficiencies in the prior art, the purpose of this invention is to provide a safety monitoring system, method, device and medium for robots.

[0006] The robot safety monitoring system provided by the present invention includes the following steps:

[0007] The monitoring area setting module is used to set a safety monitoring area, which is used to protect the robot's working area.

[0008] A point cloud projection module is used to acquire the point cloud of the security monitoring area and project the point cloud onto a pixel plane, wherein the pixel plane includes the depth value of each point.

[0009] The point cloud monitoring module is used to continuously monitor the point cloud in the security monitoring area and map the monitored point cloud onto the pixel plane.

[0010] The intrusion detection module is used to determine whether there is an intruding moving object in the security monitoring area by comparing the depth values ​​of multiple monitored point clouds and their neighborhoods with the depth values ​​of the corresponding pixels in the pixel plane.

[0011] Preferably, the monitoring area setting module includes the following units:

[0012] A monitoring area setting unit is used to set a security monitoring area, which includes security monitoring sub-areas with multiple states.

[0013] The operation status acquisition unit is used to acquire the operation status of the robot, which includes the left pallet working mode, the right pallet working mode, and the double pallet working mode.

[0014] The safety monitoring and control unit is used to control several safety monitoring sub-areas to perform safety monitoring based on the robot's operating status.

[0015] Preferably, the point cloud projection module includes the following units:

[0016] A point cloud acquisition unit is used to acquire the point cloud of the security monitoring area and determine the point cloud of the security monitoring area into corresponding security monitoring sub-area point clouds.

[0017] A pixel plane establishment unit is used to acquire the point cloud of each of the security monitoring sub-regions and establish a pixel plane for each of the security monitoring sub-regions.

[0018] A point cloud projection unit is used to project the point cloud of each of the security monitoring sub-regions onto a corresponding pixel plane, wherein the pixel plane includes the depth value of each point.

[0019] Preferably, the point cloud monitoring module includes the following units:

[0020] The point cloud monitoring unit is used to acquire the sub-regions that need to be monitored in the security monitoring area, and to continuously monitor the point cloud of each sub-region that needs to be monitored.

[0021] The point cloud filtering unit is used to acquire the point cloud monitored in each security monitoring sub-area and filter the monitored point cloud to generate a target point cloud.

[0022] The point cloud mapping unit is used to map the filtered target point cloud onto the pixel plane corresponding to each security monitoring sub-region.

[0023] Preferably, the intrusion detection module includes the following units:

[0024] A depth acquisition unit is used to acquire the depth values ​​of the monitored point clouds and their neighborhoods;

[0025] The depth comparison unit compares the depth values ​​of multiple detected point clouds and their neighbors with the depth values ​​of the corresponding pixels relative to the pixel plane. If the depth values ​​of the multiple detected point clouds and their neighbors are greater than or equal to the depth value of the corresponding pixels of the pixel plane, the data is returned to the depth acquisition unit; otherwise, the depth change monitoring unit is triggered.

[0026] The depth change monitoring unit is used to determine the depth change of the multiple point clouds and their neighborhoods within a preset time threshold, and to determine the motion direction of the moving object based on the depth change.

[0027] Preferably, the depth contrast unit includes:

[0028] For each security monitoring sub-region, the depth values ​​of multiple monitored point clouds and their neighborhoods are compared with the depth values ​​of the corresponding pixels relative to the pixel plane.

[0029] When the depth values ​​of multiple point clouds and their neighborhoods detected in any of the security monitoring sub-regions are greater than or equal to the depth value of the corresponding pixel in the pixel plane, the depth acquisition unit is returned.

[0030] The depth change monitoring unit is triggered when the depth values ​​of multiple point clouds and their neighbors detected in any of the aforementioned security monitoring sub-regions are less than the depth value of the corresponding pixel in the pixel plane.

[0031] Preferably, the security monitoring sub-area in multiple states includes:

[0032] A safety fence is used to form a dual-pallet simultaneous operation safety monitoring sub-area surrounding the robot and the two material pallets;

[0033] A first single-pallet safety monitoring sub-area is formed around the robot and the material pallet located on the right side, in conjunction with a safety fence;

[0034] A second single-pallet safety monitoring sub-area is formed around the robot and the material pallet located on the left, in conjunction with a safety fence.

[0035] The robot safety monitoring method provided by the present invention includes the following steps:

[0036] Step S1: Set up a safety monitoring area, which is used to protect the robot's working area.

[0037] Step S2: Obtain the point cloud of the security monitoring area and project the point cloud onto a pixel plane, wherein the pixel plane includes the depth value of each point;

[0038] Step S3: Continuously monitor the point cloud of the security monitoring area and map the monitored point cloud onto the pixel plane;

[0039] Step S4: Compare the depth values ​​of the monitored point clouds and their neighborhoods with the depth values ​​of the corresponding pixels in the pixel plane to determine whether there is an intruding moving object in the security monitoring area.

[0040] The robot safety monitoring device provided by the present invention includes:

[0041] processor;

[0042] A memory module that stores executable instructions of the processor;

[0043] The processor is configured to perform the steps of the robot's safety monitoring system by executing the executable instructions.

[0044] According to the computer-readable storage medium provided by the present invention, a program is used to store a program that, when executed, implements the steps of the safety monitoring system for the robot.

[0045] Compared with the prior art, the present invention has the following beneficial effects:

[0046] This invention first establishes a safety monitoring area to protect the robot's working area. Then, a point cloud of the safety monitoring area is acquired and projected onto a pixel plane, which includes the depth value of each point. The point cloud of the safety monitoring area is continuously monitored, and the monitored point cloud is mapped onto the pixel plane. The depth values ​​of multiple monitored point clouds and their neighbors are compared with the depth values ​​of corresponding pixels on the pixel plane to determine if an intruding moving object exists in the safety monitoring area. When an intruding moving object is detected in the safety monitoring area, the robot is controlled to stop operating, achieving effective safety protection during robot operation and preventing accidental injury to moving objects by the robotic arm. Attached Figure Description

[0047] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are merely embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on the provided drawings without creative effort. Other features, objects, and advantages of the present invention will become more apparent by reading the following detailed description of non-limiting embodiments with reference to the accompanying drawings:

[0048] Figure 1 This is a schematic diagram of the robot safety monitoring system in an embodiment of the present invention;

[0049] Figure 2 This is a schematic diagram of the monitoring area setting module in an embodiment of the present invention;

[0050] Figure 3 This is a schematic diagram of the point cloud projection module in an embodiment of the present invention;

[0051] Figure 4 This is a schematic diagram of the point cloud monitoring module in an embodiment of the present invention;

[0052] Figure 5 This is a schematic diagram of the intrusion detection module in an embodiment of the present invention;

[0053] Figure 6 This is a schematic diagram illustrating a state for safety monitoring of a robot in an embodiment of the present invention;

[0054] Figure 7 This is a schematic diagram illustrating another state for safety monitoring of the robot in an embodiment of the present invention;

[0055] Figure 8 This is a flowchart illustrating the steps of the robot safety monitoring method in an embodiment of the present invention;

[0056] Figure 9 This is a structural schematic diagram of the robot's safety monitoring device in an embodiment of the present invention; and

[0057] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium in an embodiment of the present invention. Detailed Implementation

[0058] The present invention will now be described in detail with reference to specific embodiments. These embodiments will help those skilled in the art to further understand the present invention, but do not limit the invention in any way. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the present invention. These all fall within the scope of protection of the present invention.

[0059] It should be noted that when a component is referred to as "fixed to" or "set on" another component, it can be directly on or indirectly on that other component. When a component is referred to as "connected to" another component, it can be directly connected to or indirectly connected to that other component. Furthermore, a connection can be for both fixing and circuit connection purposes.

[0060] It should be understood that the terms "length", "width", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", and "outer" indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing the embodiments of the present invention and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on the present invention.

[0061] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of embodiments of the present invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0062] Figure 1 This is a schematic diagram of the robot safety monitoring system in an embodiment of the present invention, such as... Figure 1 As shown, the robot safety monitoring system provided by this invention includes the following modules:

[0063] The monitoring area setting module is used to set a safety monitoring area, which is used to protect the robot's working area.

[0064] A point cloud projection module is used to acquire the point cloud of the security monitoring area and project the point cloud onto a pixel plane, wherein the pixel plane includes the depth value of each point.

[0065] The point cloud monitoring module is used to continuously monitor the point cloud in the security monitoring area and map the monitored point cloud onto the pixel plane.

[0066] The intrusion detection module is used to determine whether there is an intruding moving object in the security monitoring area by comparing the depth values ​​of multiple monitored point clouds and their neighborhoods with the depth values ​​of the corresponding pixels in the pixel plane.

[0067] Figure 2 This is a schematic diagram of the monitoring area setting module in an embodiment of the present invention, as shown below. Figure 2 As shown, the monitoring area setting module includes the following units:

[0068] A monitoring area setting unit is used to set a security monitoring area, which includes security monitoring sub-areas with multiple states.

[0069] The operation status acquisition unit is used to acquire the operation status of the robot, which includes the left pallet working mode, the right pallet working mode, and the double pallet working mode.

[0070] The safety monitoring and control unit is used to control several safety monitoring sub-areas to perform safety monitoring based on the robot's operating status.

[0071] In this embodiment of the invention, the security monitoring sub-area specifically includes a left side security monitoring sub-area, a right side security monitoring sub-area, and a rear side security monitoring sub-area.

[0072] When the left tray is in working mode, the rear side safety monitoring sub-area and the right side safety monitoring sub-area are controlled to perform safety monitoring.

[0073] When the right tray is in working mode, the rear side safety monitoring sub-area and the left side safety monitoring sub-area are controlled to perform safety monitoring.

[0074] In the dual-tray operating mode, the rear side safety monitoring sub-area is controlled and safety monitoring is performed.

[0075] Each of the aforementioned security monitoring sub-areas is planar.

[0076] Figure 3 This is a schematic diagram of the point cloud projection module in an embodiment of the present invention, as shown below. Figure 3 As shown, the point cloud projection module includes the following units:

[0077] A point cloud acquisition unit is used to acquire the point cloud of the security monitoring area and determine the point cloud of the security monitoring area into corresponding security monitoring sub-area point clouds.

[0078] A pixel plane establishment unit is used to acquire the point cloud of each of the security monitoring sub-regions and establish a pixel plane for each of the security monitoring sub-regions.

[0079] A point cloud projection unit is used to project the point cloud of each of the security monitoring sub-regions onto a corresponding pixel plane, wherein the pixel plane includes the depth value of each point.

[0080] In this embodiment of the invention, the pixel plane can be set to be coplanar with the security monitoring sub-region, or the pixel plane can be set to be parallel to the security monitoring sub-region.

[0081] Figure 4 This is a schematic diagram of the point cloud monitoring module in an embodiment of the present invention, as shown below. Figure 4 As shown, the point cloud monitoring module includes the following units:

[0082] The point cloud monitoring unit is used to acquire the sub-regions that need to be monitored in the security monitoring area, and to continuously monitor the point cloud of each sub-region that needs to be monitored.

[0083] The point cloud filtering unit is used to acquire the point cloud monitored in each security monitoring sub-area and filter the monitored point cloud to generate a target point cloud.

[0084] The point cloud mapping unit is used to map the filtered target point cloud onto the pixel plane corresponding to each security monitoring sub-region.

[0085] In this embodiment of the invention, when filtering the monitored point cloud to generate the target point cloud, the process includes removing ground point clouds and some flying points, etc.

[0086] Figure 5 This is a schematic diagram of the intrusion detection module in an embodiment of the present invention, as shown below. Figure 5 As shown, the intrusion detection module includes the following units:

[0087] A depth acquisition unit is used to acquire the depth values ​​of the monitored point clouds and their neighborhoods;

[0088] The depth comparison unit compares the depth values ​​of multiple detected point clouds and their neighbors with the depth values ​​of the corresponding pixels relative to the pixel plane. If the depth values ​​of the multiple detected point clouds and their neighbors are greater than or equal to the depth value of the corresponding pixels of the pixel plane, the data is returned to the depth acquisition unit; otherwise, the depth change monitoring unit is triggered.

[0089] The depth change monitoring unit is used to determine the depth change of the multiple point clouds and their neighborhoods within a preset time threshold, and to determine the motion direction of the moving object based on the depth change.

[0090] In this embodiment of the invention, when the depth values ​​of multiple point clouds and their neighborhoods do not change within a preset time threshold, and when the depth values ​​of the multiple point clouds and their neighborhoods decrease relative to the depth values ​​of the corresponding pixels in the pixel plane, and the corresponding values ​​of the point clouds in the pixel plane change continuously, it is determined that the invading moving object is perpendicular to the direction of the radar laser emission.

[0091] When multiple point clouds and their neighborhood depth values ​​are continuously decreasing within a preset time threshold, it is determined that the intruding moving object is moving in the direction of the radar laser emission.

[0092] When multiple point clouds and their neighborhood depth values ​​are continuously increasing within a preset time threshold, it is determined that the intruding moving object is moving away from the direction from which the radar laser is emitted.

[0093] In this embodiment of the invention, the depth comparison unit includes:

[0094] For each security monitoring sub-region, the depth values ​​of multiple monitored point clouds and their neighborhoods are compared with the depth values ​​of the corresponding pixels relative to the pixel plane.

[0095] When the depth values ​​of multiple point clouds and their neighborhoods detected in any of the security monitoring sub-regions are greater than or equal to the depth value of the corresponding pixel in the pixel plane, the depth acquisition unit is returned.

[0096] The depth change monitoring unit is triggered when the depth values ​​of multiple point clouds and their neighbors detected in any of the aforementioned security monitoring sub-regions are less than the depth value of the corresponding pixel in the pixel plane.

[0097] Figure 6 This is a schematic diagram illustrating a state for safety monitoring of a robot in an embodiment of the present invention, such as... Figure 6 As shown, a material tray is provided on each side of the robot, and a safety fence 2 is provided on the outside of the material tray; the safety fence 2 is provided with a safe passage to facilitate the entry and exit of the tray;

[0098] When a moving object is detected in the security monitoring area, the robot is controlled to stop operating.

[0099] Figure 7 This is another schematic diagram illustrating the safety monitoring of the robot in an embodiment of the present invention, such as... Figure 7 As shown, the safety radar is used in conjunction with the safety fence 2 to form a safety monitoring area surrounding the first robot and the material pallet.

[0100] The security monitoring sub-areas with multiple states include:

[0101] The safety fence 2 forms a dual-pallet simultaneous operation safety monitoring sub-area surrounding the robot and the two material pallets;

[0102] The safety fence 2 forms a first single-pallet safety monitoring sub-area surrounding the robot and the material pallet located on the right.

[0103] The safety fence 2 forms a second single-pallet safety monitoring sub-area surrounding the robot and the material pallet located on the left.

[0104] When the dual-pallet simultaneous operation safety monitoring sub-area is in the state, as long as a moving object enters the dual-pallet simultaneous operation safety monitoring sub-area, the robot will be controlled to stop working.

[0105] When in the first single-pallet safety monitoring sub-area state, the material pallet located on the left side of the robot can be replaced, and the robot can operate at a low speed in the first single-pallet safety monitoring sub-area; however, as soon as a moving object enters the first single-pallet safety monitoring sub-area, the robot will be controlled to stop operating.

[0106] In the second single-pallet safety monitoring sub-area state, the material pallet located on the right side of the robot can be replaced. The robot can operate at a low speed in the second single-pallet safety monitoring sub-area; however, if any moving object enters the second single-pallet safety monitoring sub-area, the robot will stop operating.

[0107] In this embodiment of the invention, the safety monitoring area in multiple states can be set to switch automatically according to the robot's operating state, or it can be set to switch via a control switch on the console. After switching, the current safety monitoring area status is displayed on the human-machine interface of the console.

[0108] Figure 8 This is a flowchart illustrating the steps of the robot safety monitoring method in an embodiment of the present invention, as follows: Figure 8 As shown, the robot safety monitoring method provided by the present invention includes the following steps:

[0109] Step S1: Set up a safety monitoring area, which is used to protect the robot's working area.

[0110] Step S2: Obtain the point cloud of the security monitoring area and project the point cloud onto a pixel plane, wherein the pixel plane includes the depth value of each point;

[0111] Step S3: Continuously monitor the point cloud of the security monitoring area and map the monitored point cloud onto the pixel plane;

[0112] Step S4: Compare the depth values ​​of the monitored point clouds and their neighborhoods with the depth values ​​of the corresponding pixels in the pixel plane to determine whether there is an intruding moving object in the security monitoring area.

[0113] This invention also provides a robot safety monitoring device, including a processor and a memory. The memory stores executable instructions for the processor. The processor is configured to execute robot safety monitoring method steps by executing the executable instructions.

[0114] As described above, in this embodiment, a safety monitoring area is first set up to protect the robot's working area. Then, a point cloud of the safety monitoring area is acquired and projected onto a pixel plane. The pixel plane includes the depth value of each point. The point cloud of the safety monitoring area is continuously monitored, and the monitored point cloud is mapped onto the pixel plane. The depth values ​​of the monitored point clouds and their neighbors are compared with the depth values ​​of the corresponding pixels on the pixel plane to determine whether there is an intruding moving object in the safety monitoring area. When an intruding moving object is detected in the safety monitoring area, the robot is controlled to stop working, thus achieving effective safety protection during robot operation and avoiding accidental injury to the moving object by the robotic arm.

[0115] Those skilled in the art will understand that various aspects of the present invention can be implemented as systems, methods, or program products. Therefore, various aspects of the present invention can be specifically implemented in the following forms: a completely hardware implementation, a completely software implementation (including firmware, microcode, etc.), or a combination of hardware and software aspects, collectively referred to herein as a "circuit," "module," or "platform."

[0116] Figure 9 This is a structural schematic diagram of the robot's safety monitoring device in an embodiment of the present invention. See below for reference. Figure 9 To describe an electronic device 600 according to this embodiment of the present invention. Figure 9 The electronic device 600 shown is merely an example and should not impose any limitations on the functionality and scope of use of the embodiments of the present invention.

[0117] like Figure 9 As shown, the electronic device 600 is presented in the form of a general-purpose computing device. The components of the electronic device 600 may include, but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different platform components (including storage unit 620 and processing unit 610), a display unit 640, etc.

[0118] The storage unit stores program code, which can be executed by the processing unit 610 to perform the steps described in the section on robot safety monitoring methods of this specification, according to various exemplary embodiments of the present invention. For example, the processing unit 610 can perform actions such as... Figure 8 The steps are shown in the figure.

[0119] Storage unit 620 may include a readable medium in the form of a volatile storage unit, such as random access memory (RAM) 6201 and / or cache memory 6202, and may further include a read-only memory (ROM) 6203.

[0120] Storage unit 620 may also include a program / utility 6204 having a set (at least one) program module 6205, such program module 6205 including but not limited to: operating system, one or more application programs, other program modules and program data, each or some combination of these examples may include an implementation of a network environment.

[0121] Bus 630 can represent one or more of several types of bus structures, including a memory cell bus or memory cell controller, a peripheral bus, a graphics acceleration port, a processing unit, or a local bus using any of the various bus structures.

[0122] Electronic device 600 can also communicate with one or more external devices 700 (e.g., keyboard, pointing device, Bluetooth device, camera, depth camera, etc.), and with one or more devices that enable a user to interact with electronic device 600, and / or with any device that enables electronic device 600 to communicate with one or more other computing devices (e.g., router, modem, etc.). This communication can be performed via input / output (I / O) interface 650. Furthermore, electronic device 600 can also communicate with one or more networks (e.g., local area network (LAN), wide area network (WAN), and / or public networks, such as the Internet) via network adapter 660. Network adapter 660 can communicate with other modules of electronic device 600 via bus 630. It should be understood that, although... Figure 9 As not shown in the diagram, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage platforms.

[0123] This invention also provides a computer-readable storage medium for storing a program, which, when executed, implements the steps of a robot safety monitoring method. In some possible implementations, various aspects of the invention can also be implemented as a program product comprising program code, which, when run on a terminal device, causes the terminal device to perform the steps described in the above-described robot safety monitoring method section of this specification according to various exemplary embodiments of the invention.

[0124] As shown above, when the program of the computer-readable storage medium of this embodiment is executed, it first sets up a safety monitoring area for the safety protection of the robot's working area. Then, it acquires the point cloud of the safety monitoring area and projects the point cloud onto a pixel plane, which includes the depth value of each point. It continuously monitors the point cloud of the safety monitoring area and maps the monitored point cloud onto the pixel plane. It compares the depth values ​​of the monitored point clouds and their neighbors with the depth values ​​of the corresponding pixels on the pixel plane to determine whether there is an intruding moving object in the safety monitoring area. When an intruding moving object is detected in the safety monitoring area, it controls the robot to stop working, thereby achieving effective safety protection during robot operation and avoiding accidental injury to the moving object by the robotic arm.

[0125] Figure 10 This is a schematic diagram of the structure of a computer-readable storage medium according to an embodiment of the present invention. (Reference) Figure 10 As shown, a program product 800 for implementing the above-described method according to an embodiment of the present invention is described. This product may employ a portable compact disc read-only memory (CD-ROM) and include program code, and may run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, the readable storage medium may be any tangible medium containing or storing a program that may be used by or in conjunction with an instruction execution system, apparatus, or device.

[0126] The program product may employ any combination of one or more readable media. A readable medium may be a readable signal medium or a readable storage medium. A readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples (a non-exhaustive list) of readable storage media include: electrical connections having one or more wires, portable disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof.

[0127] Computer-readable storage media may include data signals propagated in baseband or as part of a carrier wave, carrying readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A readable storage medium may also be any readable medium other than a readable storage medium that can transmit, propagate, or transfer a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the readable storage medium may be transmitted using any suitable medium, including but not limited to wireless, wired, optical fiber, RF, etc., or any suitable combination thereof.

[0128] Program code for performing the operations of this invention can be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java and C++, and conventional procedural programming languages ​​such as C or similar languages. The program code can execute entirely on the user's computing device, partially on the user's computing device, as a standalone software package, partially on the user's computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device can be connected to the user's computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or it can be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0129] In this embodiment of the invention, a safety monitoring area is first set up to protect the robot's working area. Then, a point cloud of the safety monitoring area is acquired and projected onto a pixel plane, which includes the depth value of each point. The point cloud of the safety monitoring area is continuously monitored, and the monitored point cloud is mapped onto the pixel plane. The depth values ​​of the monitored point clouds and their neighbors are compared with the depth values ​​of the corresponding pixels on the pixel plane to determine whether there is an intruding moving object in the safety monitoring area. When an intruding moving object is detected in the safety monitoring area, the robot is controlled to stop working, thus achieving effective safety protection during robot operation and avoiding accidental injury to the moving object by the robotic arm.

[0130] The various embodiments described in this specification are presented in a progressive manner, with each embodiment focusing on its differences from other embodiments. Similar or identical parts between embodiments can be referred to interchangeably. The above description of the disclosed embodiments enables those skilled in the art to make or use the invention. Various modifications to these embodiments will be readily apparent to those skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the invention. Therefore, the invention is not to be limited to the embodiments shown herein, but is to be accorded the widest scope consistent with the principles and novel features disclosed herein.

[0131] The specific embodiments of the present invention have been described above. It should be understood that the present invention is not limited to the specific embodiments described above, and those skilled in the art can make various modifications or variations within the scope of the claims, which do not affect the essence of the present invention.

Claims

1. A safety monitoring system for a robot, characterized in that The method comprises the following modules: A monitoring area setting module is configured to set a safety monitoring area for safety protection of a working area of the robot. A point cloud projection module is configured to acquire point clouds of the safety monitoring area and project the point clouds into a pixel plane, which comprises depth values of each point. A point cloud monitoring module is configured to continuously monitor the point clouds of the safety monitoring area and map the monitored point clouds to the pixel plane. An intrusion judgment module is configured to compare the depth values of the monitored point clouds and their adjacent areas with the depth values of corresponding pixels of the pixel plane to determine whether there is an intruding moving object in the safety monitoring area.

2. The safety monitoring system of the robot according to claim 1, characterized in that, The monitoring area setting module comprises the following units: A monitoring area setting unit is configured to set a safety monitoring area, which comprises safety monitoring sub-areas in multiple states. A working state acquisition unit is configured to acquire a working state of the robot, which comprises a left tray working mode, a right tray working mode and a double tray working mode. A safety monitoring control unit is configured to control safety monitoring of a plurality of safety monitoring sub-areas according to the working state of the robot.

3. The safety monitoring system of the robot according to claim 2, wherein The point cloud projection module comprises the following units: A point cloud acquisition unit is configured to acquire point clouds of the safety monitoring area and determine corresponding safety monitoring sub-area point clouds of the safety monitoring area. A pixel plane establishment unit is configured to acquire each safety monitoring sub-area point cloud and establish a pixel plane corresponding to each safety monitoring sub-area. A point cloud projection unit is configured to project each safety monitoring sub-area point cloud into a corresponding pixel plane, which comprises depth values of each point.

4. The safety monitoring system of the robot according to claim 1, wherein The point cloud monitoring module comprises the following units: A point cloud monitoring unit is configured to acquire sub-areas in the safety monitoring area that need to be monitored and continuously monitor point clouds of each sub-area that needs to be monitored. A point cloud filtering unit is configured to acquire point clouds monitored by each safety monitoring sub-area and filter the monitored point clouds to generate target point clouds. A point cloud mapping unit is configured to map the filtered target point clouds to the pixel plane corresponding to each safety monitoring sub-area.

5. The safety monitoring system of the robot according to claim 1, wherein The intrusion judgment module comprises the following units: A depth acquisition unit is configured to acquire depth values of the monitored point clouds and their adjacent areas. A depth comparison unit is configured to compare the depth values of the monitored point clouds and their adjacent areas with the depth values of corresponding pixels of the pixel plane. When the depth values of the monitored point clouds and their adjacent areas are greater than or equal to the depth values of the corresponding pixels of the pixel plane, the depth acquisition unit is returned, otherwise, a depth change monitoring unit is triggered. The depth change monitoring unit is configured to judge depth changes of the monitored point clouds and their adjacent areas within a pre-set time threshold and determine a moving direction of the moving object according to the depth value changes. 6.The safety monitoring system of the robot according to claim 5, wherein, The depth comparison unit comprises: For each safety monitoring sub-area, compare the depth values of the monitored point clouds and their vicinities with the depth values of the corresponding pixels of the pixel plane; When the depth values of the monitored point clouds and their vicinities in any of the safety monitoring sub-areas are greater than or equal to the depth values of the corresponding pixels of the pixel plane, return a depth acquisition unit; When the depth values of the monitored point clouds and their vicinities in any of the safety monitoring sub-areas are less than the depth values of the corresponding pixels of the pixel plane, trigger a depth change monitoring unit.

7. The safety monitoring system of the robot according to claim 2, wherein The safety monitoring sub-areas in multiple states include: a double-tray simultaneous operation safety monitoring sub-area formed in cooperation with the safety fence to surround the robot and the two material trays; a first single-tray safety monitoring sub-area formed in cooperation with the safety fence to surround the robot and the material tray located on the right side; a second single-tray safety monitoring sub-area formed in cooperation with the safety fence to surround the robot and the material tray located on the left side.

8. A safety monitoring method of a robot, characterized by, The method comprises the following steps: Step S1: setting a safety monitoring area for safety protection of the working area of the robot; Step S2: acquiring point clouds of the safety monitoring area and projecting the point clouds into a pixel plane, the pixel plane including the depth values of each point; Step S3: continuously monitoring the point clouds of the safety monitoring area and mapping the monitored point clouds to the pixel plane; Step S4: determining whether there is an intruding moving object in the safety monitoring area according to the comparison of the depth values of the monitored point clouds and their vicinities with the depth values of the corresponding pixels of the pixel plane.

9. A safety monitoring device for a robot, characterized by It comprises: a processor; a memory module having executable instructions of the processor stored therein; wherein the processor is configured to execute the steps of the safety monitoring system of the robot of claim 8 by executing the executable instructions.

10. A computer readable storage medium for storing a program, characterized in that, The program is executed to implement the steps of the safety monitoring system of the robot of claim 8.

Images