A control method of a robot chassis, a storage medium and an electronic device

Abstract

The application provides a control method of a robot chassis, a storage medium and an electronic device, and steps of the method include: the robot travels along a travel path; radar images collected by a radar sensor and video images collected by a camera at the same time are acquired; a qualified rate of the radar images is calculated based on a proportion of missing point clouds in the radar images, and a qualified rate of the video images is calculated based on a proportion of invalid pixel points in the video images; a main mode is determined based on the qualified rates, a standard image of the current main mode is acquired based on a position of the robot, and an abnormal area in the radar images or the video images is determined based on the standard image and the radar images or the video images of the current main mode; a preset template image is labeled to obtain a first rendering image; a second rendering image is constructed based on the first rendering images of multiple continuous time points, the second rendering image is input into a preset neural network model, and the neural network model outputs an adjustment instruction.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of robots, in particular to a control method of a robot chassis, a storage medium and an electronic device. BACKGROUND

[0002] The application of sensors in the field of robotics is crucial, as they act as the "senses" of the robot, providing it with the ability to perceive the external environment. Through different types of sensors, the robot can obtain a large amount of information about its surroundings, including distance, temperature, humidity, light, sound, pressure and magnetic field, etc.

[0003] In the aspect of autonomous navigation in the prior art, robots use lidar, ultrasonic sensors and infrared sensors, etc. to accurately measure the distance to surrounding obstacles, so as to plan the optimal moving path. In industrial production lines, pressure sensors and temperature sensors can monitor the running state of equipment in real time, ensuring the stability and safety of the production process. In addition, visual sensors and image recognition technology enable robots to recognize and process complex visual information, such as recognizing object shape, color, texture, etc., which is crucial for achieving precise grasping, sorting and assembly, etc.

[0004] Although the robot chassis in the prior art is equipped with radar or camera devices, in actual navigation and avoidance of the robot, due to environmental changes, the radar and camera devices may be affected by the environment, resulting in insufficient accuracy and difficulty in achieving precise control of the robot.

[0005] Therefore, the present application is proposed. SUMMARY

[0006] The purpose of the present application is to provide a control method of a robot chassis, a storage medium and an electronic device. The present application combines multiple modal devices, which can effectively avoid moving obstacles on the one hand, and can also re-plan the driving path to ensure precise control of the robot when the obstacle completely blocks the road.

[0007] The present application provides a control method of a robot chassis, the robot chassis is equipped with a radar sensor and a camera, the radar sensor and the camera collect in real time, and the steps of the method include:

[0008] Based on the received destination address and the current position of the robot, a driving path is constructed, and the robot drives along the driving path;

[0009] Obtain the radar image collected by the radar sensor and the video image collected by the camera at the same time;

[0010] calculate a qualified rate of the radar image based on a proportion of missing point clouds in the radar image, and calculate a qualified rate of the video image based on a proportion of invalid pixel points in the video image;

[0011] determine a main mode based on the qualified rates of the radar image and the video image, obtain a standard image of the current main mode based on the position of the robot, and determine an abnormal area in the radar image or the video image based on the standard image and the radar image or the video image of the current main mode;

[0012] label a preset template image based on the abnormal area to obtain a first rendering image;

[0013] construct a second rendering image based on the first rendering images of multiple continuous time points, input the second rendering image into a preset neural network model, and output an adjustment instruction from the neural network model, wherein the adjustment instruction comprises a turning avoidance instruction and a path re-planning instruction, and if the adjustment instruction is the path re-planning instruction, re-construct a driving path, and the robot drives along the re-constructed driving path.

[0014] With the above scheme, the robot in the scheme is equipped with a radar sensor and a camera at the same time. In the driving process of the vehicle, the accuracy of the two may decrease due to environmental factors. The scheme can determine the main mode by comparing the data collected by the two, calculate the abnormal area of the main mode at each time point, construct a first rendering image, construct a second rendering image based on the first rendering images of multiple continuous time points, and output an adjustment instruction through a neural network model. The second rendering image represents the position of the abnormal area in a period of time. On the one hand, it can effectively avoid moving obstacles. On the other hand, when the obstacles completely block the road, the driving path can also be re-planned to ensure the accurate control of the robot.

[0015] In some embodiments of the present application, in the step of constructing the driving path, the way of constructing the driving path is A* algorithm or shortest path algorithm.

[0016] In some embodiments of the present application, in the steps of calculating the qualified rate of the radar image based on the proportion of missing point clouds in the radar image and calculating the qualified rate of the video image based on the proportion of invalid pixel points in the video image, for the radar image, a set number of point clouds is obtained according to the hardware parameters of the radar, the number of point clouds of the collected radar image is then determined, the difference between the set number of point clouds and the number of point clouds of the collected radar image is calculated as the number of missing point clouds, and the ratio of the number of missing point clouds to the set number of point clouds is calculated to obtain the qualified rate of the radar image; for the video image, the invalid pixel points in the video image are screened based on the pixel values of the pixel points in the video image, the proportion of the invalid pixel points to all pixel points is calculated to obtain the qualified rate of the video image.

[0017] In some embodiments of the present application, in the step of determining the main mode based on the qualified rate of the radar image and the video image, the qualified rates of the radar image and the video image are compared, and the image acquisition mode corresponding to the higher qualified rate is taken as the main mode.

[0018] With the above scheme, since in a dark environment, the camera is difficult to use due to lack of light; in a rainy or foggy environment, the radar is easily affected; the present scheme determines the more applicable mode in the actual image according to the qualified rate, and takes the mode as the applicable mode, thereby ensuring the applicability of the robot in a complex environment.

[0019] In some embodiments of the present application, in the step of obtaining a standard image of the current main mode based on the position of the robot, and determining an abnormal area in the radar image or the video image based on the standard image and the radar image or the video image of the current main mode:

[0020] If the main mode is a video, the video image is compared with the standard image, the positions of abnormal pixel points are marked, the video image with the marked abnormal pixel points is scanned, and an abnormal area in the video image is determined.

[0021] If the main mode is a radar, the radar image is compared with the standard image, and an abnormal area in the radar image is determined.

[0022] With the above scheme, the standard image is an image without obstacles at the position pre-acquired, and the determination of the abnormal area often indicates that there is an obstacle in the area, which may affect the normal driving of the robot. The present scheme compares the actually acquired image with the standard image to determine the position where the obstacle may exist, and takes the position as the abnormal area, thereby providing support for the final determination of the robot.

[0023] In some embodiments of the present application, in the step of comparing the video image with the standard image, marking the positions of abnormal pixel points, scanning the video image with the marked abnormal pixel points, and determining an abnormal area in the video image, the video image with the marked abnormal pixel points is scanned based on a preset step length and a scanning window, and whether the current area of the scanning window is an abnormal area is determined based on the number of abnormal pixel points in the area of the scanning window; in the step of comparing the radar image with the standard image and determining an abnormal area in the radar image, a difference value of the reflection intensity parameters of the standard image and the radar image corresponding to the position is calculated, and if the difference value is greater than a preset parameter threshold value, the area corresponding to the position is taken as an abnormal area.

[0024] In some embodiments of the present application, in the step of marking the preset template image based on the abnormal area to obtain the first rendering image, the video image or the radar image after the abnormal area is marked is scaled to the same size as the template image, and the abnormal area in the video image or the radar image is marked in the template image to obtain the first rendering image.

[0025] In some embodiments of the present application, in the step of constructing the second rendering image based on the first rendering images of multiple continuous time points, the first rendering images of multiple continuous time points are overlapped, and different pixel values are assigned to each pixel point in the abnormal area after overlapping based on the number of overlaps of the pixel point to obtain the second rendering image.

[0026] By the above scheme, the second rendering image overlaps the abnormal area in the first rendering image, and the more the number of overlaps is, the higher the pixel value is, which can intuitively represent the position of the obstacle and the relative position change of the obstacle and the robot, and the instructions output by the neural network model are more accurate.

[0027] Another aspect of the present application relates to a storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the control method of the robot chassis.

[0028] Another aspect of the present application also relates to an electronic device, which includes a computer device including a processor and a memory, the memory having computer instructions stored therein, and the processor is configured to execute the computer instructions stored in the memory, and when the computer instructions are executed by the processor, the system implements the steps implemented by the method.

[0029] In summary, the present application has the following advantages:

[0030] 1. The robot of the present application is equipped with a radar sensor and a camera, and in the process of driving, the accuracy of the two may be reduced due to environmental factors. The present application can determine the main mode by comparing the data collected by the two, calculate the abnormal area of the main mode at each time point, construct the first rendering image, construct the second rendering image through the first rendering images of multiple continuous time points, and output the adjustment instructions through the neural network model. The second rendering image represents the position of the abnormal area in a period of time, which can effectively avoid moving obstacles on one hand, and on the other hand, when the obstacle completely blocks the road, the driving path can be re-planned to ensure the accurate control of the robot.

[0031] 2、Due to the lack of light in the dark environment, the camera is difficult to use; the radar is easily affected in the environment of rain or heavy fog; the scheme determines the current more applicable mode through the qualified rate in the actual image, and takes the mode as the applicable mode, so as to ensure the applicability of the robot in the complex environment;

[0032] 3、The standard image of the scheme is an image collected in advance without an obstacle in the position, and the determination of the abnormal area often indicates that there is an obstacle in the area, which may affect the normal driving of the robot, the scheme compares the actually collected image with the standard image, determines the position where the obstacle may exist, and takes the position as the abnormal area, so as to provide support for the final determination of the robot;

[0033] 4、The second rendering image of the scheme overlaps the abnormal area in the first rendering image, the more the overlapping times are, the higher the pixel value is, which can directly indicate the position of the obstacle, can also indicate the relative position change of the obstacle and the robot, and can make the instruction output by the neural network model more accurate. BRIEF DESCRIPTION OF DRAWINGS

[0034] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the following will briefly introduce the drawings needed to be used in the embodiments or the prior art description. Obviously, the drawings in the following description only represent some embodiments of the present application, and for those skilled in the art, other drawings can also be obtained without creative labor based on these drawings.

[0035] Figure 1 It is a schematic diagram of an embodiment of the control method of the robot chassis of the present application;

[0036] Figure 2 It is a schematic diagram of another embodiment of the control method of the robot chassis of the present application;

[0037] Figure 3 It is a structural schematic diagram of an electronic device provided by an embodiment of the present application. DETAILED DESCRIPTION

[0038] Hereinafter, exemplary embodiments will be described in detail with reference to the accompanying drawings. In the following description, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all the embodiments consistent with the present application. Rather, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0039] The terminology used in the present application is for the purpose of describing particular embodiments only and is not intended to be limiting of the present application. As used in the present application and the following claims, the singular forms "a," "an" and "the" are intended to include the plural forms as well, unless the context clearly indicates otherwise. It will be further understood that the terms "and / or," as used herein, refers to and encompasses any and all possible combinations of one or more of the associated listed items.

[0040] In the prior art, robots are intelligent devices integrated with multiple advanced technologies, widely used in industries, power, security and other fields. They can perform inspection tasks autonomously or remotely, and realize comprehensive and detailed monitoring of target areas through high-precision sensors and intelligent algorithms.

[0041] Robots usually have strong mobility and can freely shuttle in complex environments. Whether it is a narrow passage or rugged terrain, they can flexibly cope with it to ensure the smooth progress of the inspection work. In power inspection, inspection robots can climb along power lines or towers to conduct comprehensive inspection of power equipment and timely detect potential safety hazards.

[0042] In addition to mobility, robots are equipped with various sensors such as infrared thermal imagers, visible light cameras, sound collectors, etc. These sensors can collect various data of the target area in real time, including temperature, humidity, sound, image, etc. Through intelligent algorithms to analyze and process the collected data, the robot can accurately judge the running state of the equipment and predict possible faults to provide timely and accurate warning information for maintenance personnel.

[0043] As shown in Figure 1 The present application provides a control method for a robot chassis, the robot chassis is installed with a radar sensor and a camera, the radar sensor and the camera collect in real time, the steps of the method include:

[0044] In the specific implementation process, the radar sensor and the camera can be installed in front of the robot chassis, the radar sensor can be a laser radar (Lidar) or a millimeter wave radar (RADAR), and the camera can be a camera (Camera).

[0045] Step S100, based on the received destination address and the current position of the robot, a driving path is constructed, and the robot drives along the driving path;

[0046] In the specific implementation process, the robot has pre-stored a map of the current area, and the robot can use the shortest path algorithm or A* algorithm to construct the driving path.

[0047] In the specific implementation process, the destination address can be sent to the robot by the mobile terminal of the staff, and the robot determines its current position based on its positioning device.

[0048] In step S200, radar images collected by the radar sensor and video images collected by the camera at the same time are obtained.

[0049] In the specific implementation process, the radar sensor and the camera are both real-time collection, and at each time, the radar images collected by the radar sensor and the video images collected by the camera are stored.

[0050] In step S300, the qualified rate of the radar image is calculated based on the proportion of the missing point cloud in the radar image, and the qualified rate of the video image is calculated based on the proportion of the invalid pixel points in the video image.

[0051] In the specific implementation process, the number of point clouds is fixed in the working parameters of the radar, and the actual number of point clouds is compared with the preset working parameters to calculate the qualified rate of the radar image, wherein the missing point cloud is the error of the radar; when the light is dark, the pixel points of the camera are dark, and the pixel value is low, the pixel value lower than the pixel value threshold is regarded as an invalid pixel point, and the proportion of the invalid pixel points is calculated, that is, the qualified rate of the video image.

[0052] Specifically, the information missing point is different in the video image and the radar point cloud, because for the video image, whether the data is missing or not, the image data will fill the entire area, and each pixel of the image will have a gray value, RGB value, etc. ; and for the radar point cloud, if the data is missing due to some reasons, it will directly reflect as the missing point in the radar point cloud. Therefore, for the video image, the missing point of the radar point cloud is directly regarded as the information missing part.

[0053] In step S400, the main mode is determined based on the qualified rates of the radar image and the video image, the standard image of the current main mode is obtained based on the position of the robot, and the abnormal area in the radar image or the video image is determined based on the standard image and the radar image or the video image of the current main mode.

[0054] In step S500, the preset template image is marked based on the abnormal area to obtain a first rendering image.

[0055] In step S600, a second rendering image is constructed based on the first rendering images of multiple consecutive time points, the second rendering image is input into a preset neural network model, the neural network model outputs an adjustment instruction, the adjustment instruction includes a turning avoidance instruction and a path re-planning instruction, and if the adjustment instruction is a path re-planning instruction, a driving path is re-constructed, and the robot drives along the re-constructed driving path.

[0056] In specific implementation, the neural network model can be a convolutional neural network model, the adjustment instruction includes a straight instruction, a left turn instruction, a right turn instruction or a path re-planning instruction, and the adjustment instruction further includes an angle of left turn or right turn if the adjustment instruction is the left turn instruction or the right turn instruction.

[0057] In specific implementation, if the adjustment instruction is the path re-planning instruction, the A* algorithm or the shortest path algorithm is used to reconstruct the driving path.

[0058] In specific implementation, the main mode at the previous moment and the main mode at the next moment can be different modes, and in the construction of the second rendering image, the first rendering images at multiple moments are combined, that is, the acquisition data of multiple modes are combined, so as to ensure the determination accuracy of the final neural network model.

[0059] With the above scheme, the robot of the present scheme is equipped with a radar sensor and a camera at the same time, and in the driving process of the vehicle, the accuracy of the two may be reduced due to environmental factors. The present scheme can determine the main mode by comparing the data collected by the two, calculate the abnormal area of the main mode at each moment, construct the first rendering image, construct the second rendering image through the first rendering images at multiple continuous moments, and output the adjustment instruction through the neural network model. The second rendering image represents the position of the abnormal area in a period of time. On the one hand, it can effectively avoid moving obstacles, and on the other hand, when the obstacle completely blocks the road, the driving path can also be re-planned to ensure the accurate control of the robot.

[0060] In some embodiments of the present application, in the step of constructing the driving path, the driving path is constructed by the A* algorithm or the shortest path algorithm.

[0061] In some embodiments of the present application, in the steps of calculating the qualified rate of the radar image based on the proportion of the missing point cloud in the radar image and calculating the qualified rate of the video image based on the proportion of the invalid pixel points in the video image, for the radar image, the set point cloud quantity is obtained according to the hardware parameters of the radar, the point cloud quantity of the collected radar image is determined, the difference between the set point cloud quantity and the point cloud quantity of the collected radar image is calculated as the missing point cloud quantity, the ratio of the missing point cloud quantity to the set point cloud quantity is calculated, and the qualified rate of the radar image is obtained; for the video image, the invalid pixel points in the video image are screened based on the pixel value of the pixel points in the video image, the proportion of the invalid pixel points to all pixel points is calculated, and the qualified rate of the video image is obtained.

[0062] As Figure 2As shown, in some embodiments of the present application, in the step of determining the main mode based on the qualified rate of the radar image and the video image, the step S410 of comparing the qualified rate of the radar image and the video image is included, and the acquisition mode corresponding to the higher qualified rate is taken as the main mode.

[0063] With the above scheme, since in a dark environment, the camera is difficult to use due to lack of light; in a rainy or foggy environment, the radar is easily affected; the present scheme determines the more applicable mode in the actual image according to the qualified rate, takes the mode as the applicable mode, and ensures the applicability of the robot in a complex environment.

[0064] In some embodiments of the present application, in the step of determining the abnormal area in the radar image or the video image based on the standard image of the current main mode and the radar image or the video image of the current main mode, the step includes:

[0065] Step S420, if the main mode is a video, the video image is compared with the standard image, the position of the abnormal pixel point is marked, the video image with the marked abnormal pixel point is scanned, and the abnormal area in the video image is determined.

[0066] Step S430, if the main mode is a radar, the radar image is compared with the standard image, and the abnormal area in the radar image is determined.

[0067] With the above scheme, the standard image is an image acquired in advance without obstacles at the position, and the determination of the abnormal area often indicates that there is an obstacle in the area, which may affect the normal driving of the robot. The present scheme compares the actually acquired image with the standard image to determine the position where the obstacle may exist, and takes the position as the abnormal area to provide support for the final determination of the robot.

[0068] In some embodiments of the present application, in the step of comparing the video image with the standard image, marking the position of the abnormal pixel point, scanning the video image with the marked abnormal pixel point, and determining the abnormal area in the video image, the video image with the marked abnormal pixel point is scanned based on a preset step length and a scanning window, and whether the current area of the scanning window is an abnormal area is determined based on the number of abnormal pixel points in the area of the scanning window; in the step of comparing the radar image with the standard image and determining the abnormal area in the radar image, the difference value of the reflection intensity parameters of the standard image and the radar image corresponding to the position is calculated, and if the difference value is greater than a preset parameter threshold value, the area corresponding to the position is taken as the abnormal area.

[0069] In the implementation process, in the step of scanning the video image with the marked abnormal pixel points based on the preset step length and the scanning window, determining whether the current region of the scanning window is an abnormal region based on the number of abnormal pixel points in the region of the scanning window, the scanning window is scanned row by row from the top left corner of the video image to the bottom right corner of the video image, and specifically, the scanning window is moved based on the preset step length. At each position where the scanning window stops, the ratio of the number of abnormal pixel points in the region of the scanning window to the total number of pixel points in the scanning window is calculated. If the ratio is greater than a preset abnormal threshold, the region at the position of the current scanning window is regarded as an abnormal region.

[0070] In some embodiments of the present application, in the step of marking the preset template image based on the abnormal region to obtain a first rendering image, the video image or the radar image after the abnormal region is marked is scaled to the same size as the template image, and the abnormal region in the video image or the radar image is marked in the template image to obtain the first rendering image.

[0071] In the implementation process, the template image is an image in which all pixels are the same value, and specifically, the pixel value at each position of the template image can be 0.

[0072] In some embodiments of the present application, in the step of constructing a second rendering image based on the first rendering images of multiple consecutive time points, the first rendering images of multiple consecutive time points are overlapped, each pixel point in the abnormal region after overlapping is given a different pixel value based on the number of times of overlapping of the pixel point, and the second rendering image is obtained.

[0073] In the implementation process, in the step of giving each pixel point in the abnormal region after overlapping a different pixel value based on the number of times of overlapping of the pixel point to obtain the second rendering image, the more the number of times of overlapping of the pixel point, the higher the pixel value given to the pixel point.

[0074] By using the above scheme, the second rendering image overlaps the abnormal region in the first rendering image, the more the number of times of overlapping, the higher the pixel value, which can intuitively represent the position of the obstacle and the relative position change of the obstacle and the robot, and can make the instructions output by the neural network model more accurate.

[0075] Another aspect of the present application relates to a storage medium having a computer program stored thereon, which is executed by a processor to implement the steps of the control method of the robot chassis.

[0076] Another aspect of the present application also relates to an electronic device, which comprises a computer device, the computer device comprising a processor and a memory, the memory having stored therein computer instructions, the processor being configured to execute the computer instructions stored in the memory, and the computer instructions, when executed by the processor, causing the system to implement the steps implemented by the method.

[0077] As shown in Figure 3 The embodiments of the present application provide an electronic device, which comprises a processor and a memory storing computer program instructions; the processor implements the 3D visual weather data monitoring method when executing the computer program instructions.

[0078] The electronic device can comprise a processor 1201 and a memory 1202 storing computer program instructions.

[0079] Specifically, the processor 1201 can comprise a central processing unit (CPU), or an application specific integrated circuit (ASIC), or one or more integrated circuits configured to implement one or more embodiments of the present application.

[0080] The memory 1202 can comprise a mass storage that stores data or instructions. By way of example, and without limitation, the memory 1202 can comprise a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a Universal Serial Bus (USB) drive or a combination of two or more of these. The memory 1202 can include removable or non-removable (or fixed) media, where appropriate. The memory 1202 can be internal or external to the integrated gateway disaster recovery device, where appropriate. In certain embodiments, the memory 1202 is non-volatile, solid-state memory.

[0081] The memory can include read-only memory (ROM), random-access memory (RAM), magnetic disk storage mediums, optical storage mediums, flash memory devices, electrical, optical, or other physically tangible storage device. Thus, in general, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., a memory device) encoded with software that, when executed (by one or more processors), is operable to perform operations described with reference to the methods according to the aspects of the present disclosure.

[0082] The processor 1201 implements any one of the battery thermal runaway parameter determination methods in the above embodiments by reading and executing the computer program instructions stored in the memory 1202.

[0083] In one example, the electronic device can further include a communication interface 1203 and a bus 1210. Wherein, as shown, the processor 1201, the memory 1202, the communication interface 1203 are connected through the bus 1210 and complete the communication between each other. Figure 3

[0084] The communication interface 1203 is mainly used to realize the communication between each module, device, unit and / or equipment in the embodiments of the present application.

[0085] The bus 1210 includes hardware, software or both to couple components of the electronic device to each other. By way of example, and not limitation, the bus can include an Accelerated Graphics Port (AGP) or other graphics bus, an Enhanced Industry Standard Architecture (EISA) bus, a Front Side Bus (FSB), a HyperTransport (HT) interconnect, an Industry Standard Architecture (ISA) bus, an InfiniBand (IB) interconnect, a Low Pin Count (LPC) bus, a memory bus, a Micro Channel Architecture (MCA) bus, a Peripheral Component Interconnect (PCI) bus, a PCI-Express (PCI-X) bus, a Serial Advanced Technology Attachment (SATA) bus, a Video Electronics Standards Association local (VLB) bus, or another suitable bus or combination of two or more of these. Where appropriate, the bus 1210 can include one or more buses. Although the present application describes and illustrates a particular bus, the present application contemplates any suitable bus or interconnect.

[0086] It needs to be clear that the present application is not limited to the specific configurations and processes described above and shown in the drawings. For the sake of brevity, detailed descriptions of well-known methods are omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between steps, after understanding the spirit of the present application.

[0087] ​The functions noted in the description of the structural block diagrams above can be implemented as hardware, software, firmware, or a combination thereof. When implemented in hardware, they can be, for example, electronic circuits, application specific integrated circuits (ASICs), appropriate firmware, plug-ins, functional cards, and the like. When implemented in software, the elements of the present application are program or code segments that are used to perform the required tasks. The program or code segments can be stored in a machine-readable medium or transmitted through a data signal carried in a carrier wave over a transmission medium or communication link. A "machine-readable medium" includes any medium that can store or transport information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, fiber optic media, radio frequency (RF) links, and the like. The code segments can be downloaded via computer networks such as the Internet, intranets, and the like.

[0088] It is also important to note that the examples described herein can be implemented in a variety of systems, including and / or incorporating software, firmware, hardware, and / or circuitry. Also, the example need not necessarily be implemented as the steps shown or the order shown. One or more steps can be combined, modified, or deleted where appropriate, or additional steps can be added in accordance with the teachings of this application.

[0089] The computer program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable apparatus or other devices to produce a computer implemented process such that the instructions which execute on the computer or other programmable apparatus provide processes for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. Alternatively, computer program implemented steps can be implemented by special purpose logic circuitry, e.g., an FPGA or an ASIC, or by a combination of special purpose logic circuitry and computer hardware. Those skilled in the art will recognize how best to implement the present disclosure for the applicable software, firmware, hardware, and / or circuitry.

[0090] The above merely describes a specific implementation of the present application. Those skilled in the art can clearly understand the specific working processes of the system, modules and units described above for the convenience and brevity of description, and can refer to the corresponding processes in the foregoing method embodiments, which will not be described herein again. It should be understood that the protection scope of the present application is not limited to this, and any person skilled in the art can easily think of various equivalent modifications or replacements within the technical range disclosed by the present application, and these modifications or replacements should be covered within the protection scope of the present application.

Claims

1. A control method for a robot chassis, characterized in that, The robot chassis is equipped with radar sensors and cameras, which collect data in real time. The method includes the following steps: Based on the received destination address and the robot's current position, a travel path is constructed, and the robot travels along the travel path; Acquire radar images captured by the radar sensor and video images captured by the camera at the same time; The pass rate of the radar image is calculated based on the proportion of missing point clouds in the radar image, and the pass rate of the video image is calculated based on the proportion of invalid pixels in the video image. The dominant mode is determined based on the pass rate of radar images and video images. A standard image of the current dominant mode is obtained based on the robot's position. Abnormal areas in the radar images or video images are determined based on the standard image and the radar image or video image of the current dominant mode. The preset template image is marked based on the abnormal region to obtain the first rendered image; A second rendered image is constructed based on the first rendered image at multiple consecutive time points. The second rendered image is input into a preset neural network model. The neural network model outputs adjustment instructions, including steering and avoidance instructions and path replanning instructions. If the adjustment instruction is a path replanning instruction, the driving path is reconstructed, and the robot drives along the reconstructed driving path.

2. The control method for the robot chassis according to claim 1, characterized in that: In the step of constructing the driving path, the driving path is constructed using either the A* algorithm or the shortest path algorithm.

3. The control method for the robot chassis according to claim 1, characterized in that: In the steps of calculating the pass rate of radar images based on the proportion of missing point clouds in radar images and the pass rate of video images based on the proportion of invalid pixels in video images, for radar images, a set number of point clouds is obtained according to the radar's hardware parameters, and then the number of point clouds in the acquired radar images is determined. The difference between the set number of point clouds and the number of point clouds in the acquired radar images is calculated as the number of missing point clouds. The ratio of the number of missing point clouds to the set number of point clouds is calculated to obtain the pass rate of the radar images. For video images, invalid pixels are filtered based on the pixel values ​​of the pixels in the video images, and the proportion of invalid pixels to all pixels is calculated to obtain the pass rate of the video images.

4. The control method for the robot chassis according to claim 1, characterized in that: In the step of determining the dominant mode based on the pass rates of radar images and video images, the pass rates of radar images and video images are compared, and the acquisition mode of the image with the higher pass rate is taken as the dominant mode.

5. The control method for the robot chassis according to claim 1, characterized in that: In the step of acquiring a standard image of the current dominant mode based on the robot's position, and determining abnormal regions in the radar image or video image based on the standard image and the radar image or video image of the current dominant mode: If the main mode is video, the video image is compared with the standard image, the positions of abnormal pixels are marked, the video image marked with abnormal pixels is scanned, and the abnormal areas in the video image are determined. If the primary mode is radar, the radar image is compared with the standard image to determine the abnormal areas in the radar image.

6. The control method for the robot chassis according to claim 5, characterized in that: In the steps of comparing a video image with the standard image, marking the positions of abnormal pixels, scanning the video image marked with abnormal pixels, and determining abnormal regions in the video image, the video image marked with abnormal pixels is scanned based on a preset step size and scanning window. The number of abnormal pixels in the area of ​​the scanning window determines whether the current area of ​​the scanning window is an abnormal region. In the step of comparing a radar image with the standard image and determining abnormal regions in the radar image, the difference in reflection intensity parameters between the standard image and the radar image at the corresponding position is calculated. If the difference is greater than a preset parameter threshold, the area corresponding to that position is considered an abnormal region.

7. The control method for a robot chassis according to any one of claims 1-6, characterized in that: In the step of marking a preset template image based on the abnormal region to obtain a first rendered image, the video image or radar image after marking the abnormal region is scaled to the same size as the template image, and the abnormal region in the video image or radar image is marked in the template image to obtain the first rendered image.

8. The control method for the robot chassis according to claim 7, characterized in that: In the step of constructing a second rendered image based on a first rendered image at multiple consecutive time points, the second rendered image is obtained by overlapping the first rendered images at multiple consecutive time points and assigning different pixel values ​​to each pixel in the abnormal region after overlapping based on the number of overlaps.

9. A storage medium, characterized in that: It stores a computer program that, when executed by a processor, implements the steps of the control method for the robot chassis as described in any one of claims 1-8.

10. An electronic device, characterized in that: The electronic device includes a computer device, which includes a processor and a memory. The memory stores computer instructions, and the processor executes the computer instructions stored in the memory. When the computer instructions are executed by the processor, the steps implemented by the control method of the robot chassis according to any one of claims 1-8 are achieved.

Images