Method, device, equipment and storage medium for detecting dangerous areas during robot driving

Abstract

The present invention discloses a method, device, equipment and storage medium for detecting dangerous areas during robot driving. The method comprises: obtaining a visual image and shooting time taken by the robot, determining the position information of an escalator in the image based on the visual image as the first position information, obtaining laser data matching the shooting time, and converting the laser data based on a preset data conversion method to obtain a laser image, performing human leg recognition according to the laser image to obtain human leg boundary information, and fusing the first position information, human leg boundary information and laser data to obtain accurate information of the escalator. The present invention can improve the accuracy of detecting dangerous areas during robot driving.

Description

Technical Field

[0001] The present invention relates to the field of safety monitoring, and in particular to a method, device, equipment and storage medium for detecting dangerous areas during robot driving. Background Art

[0002] When the robot is driving, it often loses its position due to many problems at the software and hardware levels. When the robot loses its position, its movement trajectory will change unexpectedly, and it may even fail to reach the end and fall into very dangerous areas such as pools and escalators, causing losses in many aspects. Therefore, it is crucial to judge these dangerous areas during the robot's driving process.

[0003] At present, for escalators, the common judgment method is to detect through the camera on the robot. When the escalator is detected by visual images, an instruction to report danger or stop movement is issued. However, when judging whether the robot is in the dangerous area at the edge of the escalator, relying only on the image captured by the robot's own camera, it can only get the signal that there is an escalator in front. Relying on the image can only get information about possible dangers, and cannot obtain the accurate position of the escalator, so as to perform accurate avoidance behavior. Therefore, when avoiding danger based on the image information collected by the robot's own camera, there will be more false alarms, which will cause the robot to alarm or stop moving for no reason when passing by the escalator. It not only affects the normal driving of the robot, but also brings a very bad experience to the user. Therefore, there is an urgent need for a method that can accurately detect dangerous areas during robot driving. Summary of the invention

[0004] Embodiments of the present invention provide a method, an apparatus, a computer device and a storage medium for detecting dangerous areas during robot driving, so as to improve the accuracy of detecting dangerous areas during robot driving.

[0005] A method for detecting dangerous areas during robot driving, comprising:

[0006] Obtain visual images taken by the robot and the shooting time;

[0007] Based on the visual image, determining position information of the escalator in the image as first position information;

[0008] Acquire laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image;

[0009] Performing human leg recognition according to the laser image to obtain human leg boundary information;

[0010] The first position information, the human leg boundary information and the laser data are fused to obtain accurate information of the escalator.

[0011] Optionally, the determining, based on the visual image, position information of the escalator in the image as first position information includes:

[0012] Input the visual image into a trained detection network, perform recognition detection through the trained detection network, and obtain the position (x1, y1, x2, y2) of the escalator in the visual image, where (x1, y1) is the coordinate of the upper left corner of the escalator bounding box, and (x2, y2) is the coordinate of the lower right corner of the escalator bounding box;

[0013] Based on the visual image, calculate the direction angle (θ1, θ2) of the escalator in the current perspective of the robot, where θ1 is the angle corresponding to the left perspective and θ2 is the angle corresponding to the right perspective;

[0014] The position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot are used as the first position information.

[0015] Optionally, the performing human leg recognition according to the laser image to obtain human leg boundary information includes:

[0016] Input the laser image into a recognition network, use the recognition network to recognize the laser image, and obtain a human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area;

[0017] Calculate the enclosing angles (q1, q2) of the left and right boundaries of the human leg recognition area relative to the laser image;

[0018] The human leg recognition area and the enclosing angle are used as the human leg boundary information.

[0019] Optionally, the fusing the first position information, the human leg boundary information and the laser data to obtain accurate information of the escalator includes:

[0020] From the laser data, data information between the direction angles of the escalator under the current viewing angle of the robot is obtained as valid data;

[0021] Acquire data information within an enclosed angle in the human leg boundary information from the laser data as interference data;

[0022] Eliminating the interference data from the valid data to obtain target data;

[0023] The shortest value of each laser distance is calculated through the target data, and accurate information of the escalator is generated based on the obtained result.

[0024] Optionally, converting the laser data based on a preset data conversion method to obtain a laser image includes:

[0025] According to a preset ratio, the laser data is converted into binary data;

[0026] A binarized image is generated based on the binarized data, and the binarized image is used as the laser image.

[0027] A device for detecting dangerous areas during robot driving, comprising:

[0028] A first acquisition module is used to acquire the visual image taken by the robot and the shooting time;

[0029] A position determination module, used to determine, based on the visual image, position information of the escalator in the image as first position information;

[0030] A second acquisition module is used to acquire laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image;

[0031] A boundary determination module, used to identify human legs according to the laser image to obtain human leg boundary information;

[0032] The accurate information generation module is used to fuse the first position information, the human leg boundary information and the laser data to obtain accurate information of the escalator.

[0033] Optionally, the location determination module includes:

[0034] A first boundary determination unit is used to input the visual image into a trained detection network, perform recognition detection through the trained detection network, and obtain a position (x1, y1, x2, y2) of the escalator in the visual image, wherein (x1, y1) is the coordinate of the upper left corner of the escalator boundary box, and (x2, y2) is the coordinate of the lower right corner of the escalator boundary box;

[0035] A first angle determination unit is used to calculate the direction angle (θ1, θ2) of the escalator in the current viewing angle of the robot based on the visual image, where θ1 is the angle corresponding to the left viewing angle and θ2 is the angle corresponding to the right viewing angle;

[0036] The escalator position determination unit is used to use the position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot as the first position information.

[0037] Optionally, the boundary determination module includes:

[0038] A second boundary determination unit is used to input the laser image into a recognition network, and use the recognition network to recognize the laser image to obtain a human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area;

[0039] A second angle calculation unit, used to calculate the enclosing angles (q1, q2) of the left and right boundaries of the human leg recognition area relative to the laser image;

[0040] The human foot frame determination unit is used to use the human leg recognition area and the enclosing angle as the human leg boundary information.

[0041] Optionally, the precise information generation module includes:

[0042] A valid data determination unit, used to obtain data information between the direction angles of the escalator under the current viewing angle of the robot from the laser data as valid data;

[0043] An interference data determination unit, used to obtain data information within an enclosed angle in the human leg boundary information from the laser data as interference data;

[0044] A target data determination unit, configured to remove the interference data from the valid data to obtain target data;

[0045] The accurate data generating unit is used to calculate the shortest value of each laser distance through the target data, and generate accurate information of the escalator based on the obtained result.

[0046] Optionally, the second acquisition module includes:

[0047] A data conversion unit, used for converting the laser data into binary data according to a preset ratio;

[0048] The image generating unit is used to generate a binary image based on the binary data, and use the binary image as the laser image.

[0049] A computer device comprises a memory, a processor and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the steps of the method for detecting dangerous areas during robot driving are implemented.

[0050] A computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the method for detecting dangerous areas during robot driving are implemented.

[0051] The method, device, computer equipment and storage medium for detecting dangerous areas during robot driving provided by the embodiments of the present invention obtain the visual image and shooting time taken by the robot, determine the position information of the escalator in the image based on the visual image, take the first position information, obtain laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image, perform human leg recognition based on the laser image to obtain human leg boundary information, fuse the first position information, human leg boundary information and laser data to obtain accurate information of the escalator. Accurate detection of dangerous areas during robot driving is achieved. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments of the present invention will be briefly introduced below. Obviously, the accompanying drawings in the following description are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative labor.

[0053] Figure 1 is a schematic diagram of an application of a method for detecting dangerous areas during robot travel provided by an embodiment of the present invention;

[0054] Figure 2 is a flowchart of a method for detecting dangerous areas during robot driving provided by an embodiment of the present invention;

[0055] Figure 3 is a schematic diagram of a dangerous area detection device for a robot during driving provided by an embodiment of the present invention;

[0056] Figure 4 It is a schematic diagram of a computer device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present invention to clearly and completely describe the technical solutions in the embodiments of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0058] See also Figure 1 , Figure 1 The application environment of the method for detecting dangerous areas during robot driving provided by an embodiment of the present invention is shown. Figure 1 The scattered rays in the figure are the binary images of the laser data. In the example diagram of the escalator obstacle avoidance, the square indicated by J1 represents the position of the robot. The two rays indicated by J2 represent the left and right angles of the escalator given by the visual image. The box indicated by J3 is the real area of ​​the escalator. The solid dot indicated by J4 is the accurate position of the escalator obtained after the fusion of the visual image and the laser data.

[0059] See also Figure 2 , Figure 2 A method for detecting dangerous areas during robot driving provided by an embodiment of the present invention is shown and described in detail as follows:

[0060] S201: Acquire the visual image taken by the robot and the shooting time.

[0061] Specifically, the robot collects images through its own camera. When it determines that there is an escalator in front through the collected images, it obtains the image at this time as a visual image, and records the time when the visual image is taken as the shooting time.

[0062] It should be understood that the image obtained at this time can only determine the existence of the escalator in front, but cannot determine the distance between the escalator and the robot, as well as the accurate relative position information.

[0063] S202: Based on the visual image, determine the position information of the escalator in the image as the first position information.

[0064] Specifically, by analyzing and processing the visual image, the position information of the escalator in the image is obtained as the first position information. The specific implementation process can refer to the description of subsequent embodiments, and will not be repeated here to avoid repetition.

[0065] S203: Acquire laser data that matches the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image.

[0066] Specifically, laser data that matches the shooting time, that is, collected at the same time as the shooting time, is obtained. It can be understood that the laser data needs to be analyzed and fused with the visual image later. When the visual image and the laser data are not at the same time, the fusion result will produce a large error. In this embodiment, laser data that matches the shooting time is obtained, and the laser data is converted based on a preset data conversion method to obtain a laser image, which is beneficial to subsequent image analysis and fusion processing.

[0067] S204: Perform human leg recognition based on the laser image to obtain human leg boundary information.

[0068] Specifically, after the laser image is obtained, the human legs present in the laser image are identified through a preset analysis method to obtain the boundary information of the human legs.

[0069] It should be noted that in the process of judging the escalator, the human leg laser data is interference data, and the human leg data needs to be excluded to obtain the accurate distance between the robot and the escalator.

[0070] The recognition of human legs can be achieved through a trained neural network model. The specific process can be referred to the description of the subsequent embodiments. To avoid repetition, it will not be described here.

[0071] S205: The first position information, the human leg boundary information and the laser data are integrated to obtain accurate information of the escalator.

[0072] In this embodiment, by obtaining the visual image and shooting time taken by the robot, based on the visual image, the position information of the escalator in the image is determined as the first position information, the laser data matching the shooting time is obtained, and based on the preset data conversion method, the laser data is converted to obtain a laser image, and the human leg is identified according to the laser image to obtain the human leg boundary information, and the first position information, the human leg boundary information and the laser data are fused to obtain the accurate information of the escalator. Accurate detection of dangerous areas during robot driving is achieved.

[0073] In a specific optional manner, in step S202, based on the visual image, the position information of the escalator in the image is determined, and the first position information includes:

[0074] Input the visual image into the trained detection network, perform recognition detection through the trained detection network, and obtain the position of the escalator in the visual image (x1, y1, x2, y2), where (x1, y1) is the coordinate of the upper left corner of the escalator bounding box, and (x2, y2) is the coordinate of the lower right corner of the escalator bounding box;

[0075] Based on the position of the escalator in the visual image, the direction angle (θ1, θ2) of the escalator in the current perspective of the robot is calculated, where θ1 is the angle corresponding to the left perspective and θ2 is the angle corresponding to the right perspective;

[0076] The position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot are used as the first position information.

[0077] The trained detection network may be a pre-trained neural network for human leg recognition or a machine learning network.

[0078] In a specific optional manner, in step S204, human leg recognition is performed according to the laser image, and the human leg boundary information obtained includes:

[0079] The laser image is input into the recognition network, and the recognition network is used to recognize the laser image to obtain the human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area;

[0080] Calculate the enclosing angles (q1, q2) of the left and right boundaries of the leg recognition area relative to the laser image;

[0081] The leg recognition area and enclosing angle are used as the leg boundary information.

[0082] The recognition network may specifically be a pre-trained neural network for human leg recognition.

[0083] In a specific optional manner, in step S205, the first position information, the human leg boundary information and the laser data are fused to obtain accurate information of the escalator including:

[0084] From the laser data, obtain the data information between the direction angles of the escalator under the current viewing angle of the robot as valid data;

[0085] From the laser data, obtain the data information within the enclosed angle in the boundary information of the human leg as interference data;

[0086] Eliminate interference data from valid data to obtain target data;

[0087] The shortest value of each laser distance is calculated through the target data, and accurate information of the escalator is generated based on the obtained results.

[0088] Specifically, during the image fusion process, the input includes the first position information of the escalator on the image, the human leg boundary information on the laser image, and the original laser data. First, according to the left and right angles given by the image detection results (the direction angle of the escalator under the current viewing angle of the robot), the effective data of the laser is extracted, that is, only the laser distance value within the left and right angle range of the escalator is extracted, and the laser data outside the range is discarded. Since the laser data is easily interfered by obstacles (such as pedestrians, etc.), it is necessary to use the human leg recognition results on the laser to minimize external interference. After obtaining the human leg recognition results on the laser, the distance values ​​within the corresponding angle range on the laser are discarded based on this information to achieve the effect of filtering pedestrian interference. Then, based on the left and right angles of the escalator detected by the image data, the filtered laser distance values ​​are taken out, and the shortest value of the laser distance is calculated as the result after the fusion of the escalator, which contains accurate position information and angle information, and serves as auxiliary information for the robot movement.

[0089] In this embodiment, by fusing the first position information, the human leg boundary information and the laser data, interference information is removed from the approximate area, accurate position information is obtained, and the positioning accuracy of the escalator is improved.

[0090] In a specific optional manner, in step S204, based on a preset data conversion method, converting the laser data to obtain a laser image includes:

[0091] According to the preset ratio, the laser data is converted into binary data;

[0092] Based on the binarized data, a binarized image is generated, and the binarized image is used as a laser image.

[0093] Among them, the preset ratio can be set according to actual needs and is not limited here.

[0094] Specifically, according to a preset ratio, the laser data is converted so that the collected laser data becomes binary data, and then a laser image is generated through the binary data. The laser data that is difficult to fuse with the image taken by the camera is converted into image data that can be fused, which is conducive to subsequent accurate positioning through image fusion.

[0095] In this embodiment, by binarizing the data and then generating a binary image, the laser data that is difficult to fuse with the image captured by the camera is converted into image data that can be fused, which is beneficial to improving the accuracy of dangerous area detection.

[0096] It should be understood that the order of execution of the steps in the above embodiment does not necessarily mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present invention.

[0097] In one embodiment, a device for detecting dangerous areas during robot driving is provided, and the device for detecting dangerous areas during robot driving corresponds to the method for detecting dangerous areas during robot driving in the above embodiment. Figure 3 As shown, Figure 3 The schematic diagram of the dangerous area detection device for the robot during driving is as follows: a first acquisition module 31, a position determination module 32, a second acquisition module 33, a boundary determination module 34 and an accurate information generation module 35. The detailed description of each functional module is as follows:

[0098] A first acquisition module 31 is used to acquire the visual image taken by the robot and the shooting time;

[0099] A position determination module 32, for determining position information of the escalator in the image based on the visual image as first position information;

[0100] The second acquisition module 33 is used to acquire laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image;

[0101] The boundary determination module 34 is used to identify the human leg according to the laser image and obtain the boundary information of the human leg;

[0102] The accurate information generating module 35 is used to fuse the first position information, the human leg boundary information and the laser data to obtain accurate information of the escalator.

[0103] Optionally, the location determination module 32 includes:

[0104] A first boundary determination unit is used to input the visual image into the trained detection network, perform recognition detection through the trained detection network, and obtain the position (x1, y1, x2, y2) of the escalator in the visual image, where (x1, y1) is the coordinate of the upper left corner of the escalator boundary box, and (x2, y2) is the coordinate of the lower right corner of the escalator boundary box;

[0105] A first angle determination unit is used to calculate the direction angle (θ1, θ2) of the escalator in the current viewing angle of the robot based on the visual image, where θ1 is the angle corresponding to the left viewing angle and θ2 is the angle corresponding to the right viewing angle;

[0106] The escalator position determination unit is used to use the position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot as the first position information.

[0107] Optionally, the boundary determination module 34 includes:

[0108] A second boundary determination unit is used to input the laser image into a recognition network, and use the recognition network to recognize the laser image to obtain a human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area;

[0109] A second angle calculation unit, used to calculate the enclosing angles (q1, q2) of the left and right boundaries of the human leg recognition area relative to the laser image;

[0110] The human foot frame determination unit is used to use the human leg recognition area and enclosing angle as the human leg boundary information.

[0111] Optionally, the accurate information generating module 35 includes:

[0112] The effective data determination unit is used to obtain data information between the direction angles of the escalator under the current viewing angle of the robot from the laser data as effective data;

[0113] An interference data determination unit is used to obtain data information within an enclosed angle in the boundary information of the human leg from the laser data as interference data;

[0114] A target data determination unit, used to remove interference data from valid data to obtain target data;

[0115] The accurate data generating unit is used to calculate the shortest value of each laser distance through the target data, and generate accurate information of the escalator based on the obtained result.

[0116] Optionally, the second acquisition module 33 includes:

[0117] A data conversion unit, used for converting laser data into binary data according to a preset ratio;

[0118] The image generating unit is used to generate a binary image based on the binary data, and use the binary image as a laser image.

[0119] For the specific definition of the robot's driving dangerous area detection device, please refer to the definition of the robot's driving dangerous area detection method above, which will not be repeated here. Each module in the above-mentioned robot's driving dangerous area detection device can be implemented in whole or in part by software, hardware and their combination. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0120] To solve the above technical problems, the present application also provides a computer device. Figure 4 , Figure 4 This is a basic structural block diagram of the computer device in this embodiment.

[0121] The computer device 4 includes a memory 41, a processor 42, and a network interface 43 that are interconnected through a system bus. It should be noted that the figure only shows a computer device 4 with components connected to the memory 41, the processor 42, and the network interface 43, but it should be understood that it is not required to implement all the components shown, and more or fewer components can be implemented instead. Among them, those skilled in the art can understand that the computer device here is a device that can automatically perform numerical calculations and / or information processing according to pre-set or stored instructions, and its hardware includes but is not limited to microprocessors, application specific integrated circuits (Application Specific Integrated Circuit, ASIC), programmable gate arrays (Field-Programmable Gate Array, FPGA), digital processors (Digital Signal Processor, DSP), embedded devices, etc.

[0122] The computer device may be a computing device such as a desktop computer, a notebook, a PDA, a cloud server, etc. The computer device may interact with a user through a keyboard, a mouse, a remote controller, a touch pad, or a voice control device.

[0123] The memory 41 includes at least one type of readable storage medium, and the readable storage medium includes flash memory, hard disk, multimedia card, card-type memory (for example, SD or D interface display memory, etc.), random access memory (RAM), static random access memory (SRAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), programmable read-only memory (PROM), magnetic memory, disk, optical disk, etc. In some embodiments, the memory 41 can be an internal storage unit of the computer device 4, such as a hard disk or memory of the computer device 4. In other embodiments, the memory 41 can also be an external storage device of the computer device 4, such as a plug-in hard disk equipped on the computer device 4, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. Of course, the memory 41 can also include both the internal storage unit of the computer device 4 and its external storage device. In this embodiment, the memory 41 is generally used to store the operating system and various application software installed on the computer device 4, such as the program code for controlling electronic files, etc. In addition, the memory 41 can also be used to temporarily store various types of data that have been output or are to be output.

[0124] The processor 42 may be a central processing unit (CPU), a controller, a microcontroller, a microprocessor, or other data processing chips in some embodiments. The processor 42 is generally used to control the overall operation of the computer device 4. In this embodiment, the processor 42 is used to run the program code stored in the memory 41 or process data.

[0125] The network interface 43 may include a wireless network interface or a wired network interface. The network interface 43 is generally used to establish a communication connection between the computer device 4 and other electronic devices.

[0126] The present application also provides another implementation, namely, providing a computer-readable storage medium, wherein the computer-readable storage medium stores an interface display program, and the interface display program can be executed by at least one processor to enable the at least one processor to perform the steps of the information access method as described above.

[0127] Through the description of the above implementation methods, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus a necessary general hardware platform, and of course by hardware, but in many cases the former is a better implementation method. Based on such an understanding, the technical solution of the present application, or the part that contributes to the prior art, can be embodied in the form of a software product, which is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk), and includes a number of instructions for a terminal device (which can be a mobile phone, computer, server, air conditioner, or network device, etc.) to execute the methods described in each embodiment of the present application.

[0128] Obviously, the embodiments described above are only some embodiments of the present application, rather than all embodiments. The preferred embodiments of the present application are given in the accompanying drawings, but they do not limit the patent scope of the present application. The present application can be implemented in many different forms. On the contrary, the purpose of providing these embodiments is to make the understanding of the disclosure of the present application more thorough and comprehensive. Although the present application is described in detail with reference to the aforementioned embodiments, for those skilled in the art, it is still possible to modify the technical solutions recorded in the aforementioned specific implementation methods, or to perform equivalent replacement of some of the technical features therein. Any equivalent structure made using the contents of the specification and drawings of this application, directly or indirectly used in other related technical fields, is similarly within the scope of patent protection of this application.

Claims

1. A method for detecting dangerous areas during robot driving, which is applied to determine whether the robot is in a dangerous area at the edge of an escalator, and is characterized in that: include: Obtain visual images taken by the robot and the shooting time; Based on the visual image, determining position information of the escalator in the image as first position information; Acquire laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image; Performing human leg recognition according to the laser image to obtain human leg boundary information; The first position information, the human leg boundary information and the laser data are fused to obtain accurate information of the escalator. The position information of the escalator in the image is determined based on the visual image, and the first position information includes: Input the visual image into a trained detection network, perform recognition detection through the trained detection network, and obtain the position (x1, y1, x2, y2) of the escalator in the visual image, where (x1, y1) is the coordinate of the upper left corner of the escalator bounding box, and (x2, y2) is the coordinate of the lower right corner of the escalator bounding box; Based on the position of the escalator in the visual image, calculate the direction angle (θ1, θ2) of the escalator in the current perspective of the robot, where θ1 is the angle corresponding to the left perspective, and θ2 is the angle corresponding to the right perspective; Using the position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot as the first position information; The performing of human leg recognition according to the laser image to obtain human leg boundary information comprises: Input the laser image into a recognition network, use the recognition network to recognize the laser image, and obtain a human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area; Calculate the enclosing angles (q1, q2) of the left and right boundaries of the human leg recognition area relative to the laser image; Using the human leg recognition area and the enclosing angle as the human leg boundary information; The step of fusing the first position information, the human leg boundary information and the laser data to obtain accurate information of the escalator includes: From the laser data, data information between the direction angles of the escalator under the current viewing angle of the robot is obtained as valid data; Acquire data information within an enclosed angle in the human leg boundary information from the laser data as interference data; Eliminating the interference data from the valid data to obtain target data; The shortest value of each laser distance is calculated through the target data, and accurate information of the escalator is generated based on the obtained result.

2. The method for detecting dangerous areas during robot driving as claimed in claim 1, characterized in that: The converting the laser data based on the preset data conversion method to obtain the laser image includes: According to a preset ratio, the laser data is converted into binary data; A binarized image is generated based on the binarized data, and the binarized image is used as the laser image.

3. A device for detecting dangerous areas during robot driving, used to determine whether the robot is in a dangerous area at the edge of an escalator, characterized in that: include: A first acquisition module is used to acquire the visual image taken by the robot and the shooting time; A position determination module, used to determine, based on the visual image, position information of the escalator in the image as first position information; A second acquisition module is used to acquire laser data matching the shooting time, and convert the laser data based on a preset data conversion method to obtain a laser image; A boundary determination module, used to identify human legs according to the laser image to obtain human leg boundary information; The accurate information generation module is used to fuse the first position information, the human leg boundary information and the laser data to obtain accurate information of the escalator; the position determination module includes: A first boundary determination unit is used to input the visual image into a trained detection network, perform recognition detection through the trained detection network, and obtain a position (x1, y1, x2, y2) of the escalator in the visual image, wherein (x1, y1) is the coordinate of the upper left corner of the escalator boundary box, and (x2, y2) is the coordinate of the lower right corner of the escalator boundary box; A first angle determination unit is used to calculate the direction angle (θ1, θ2) of the escalator in the current viewing angle of the robot based on the visual image, where θ1 is the angle corresponding to the left viewing angle and θ2 is the angle corresponding to the right viewing angle; The escalator position determination unit is used to use the position of the escalator in the visual image and the direction angle of the escalator in the current viewing angle of the robot as the first position information; the boundary determination module includes: A second boundary determination unit is used to input the laser image into a recognition network, and use the recognition network to recognize the laser image to obtain a human leg recognition area (a1, b1, a2, b2), where (a1, b1) is the upper left corner coordinate of the human leg recognition area, and (a2, b2) is the lower right corner coordinate of the human leg recognition area; A second angle calculation unit, used to calculate the enclosing angles (q1, q2) of the left and right boundaries of the human leg recognition area relative to the laser image; A human foot frame determination unit is used to use the human leg recognition area and the enclosing angle as the human leg boundary information, and the first position information, the human leg boundary information and the laser data are fused to obtain accurate information of the escalator, including: From the laser data, data information between the direction angles of the escalator under the current viewing angle of the robot is obtained as valid data; Acquire data information within an enclosed angle in the human leg boundary information from the laser data as interference data; Eliminating the interference data from the valid data to obtain target data; The shortest value of each laser distance is calculated through the target data, and accurate information of the escalator is generated based on the obtained result.

4. A computer device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the method for detecting dangerous areas during robot driving as described in any one of claims 1 to 2 is implemented.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the method for detecting dangerous areas during robot driving as described in any one of claims 1 to 2 is implemented.

Images