Robot chassis control method, storage medium and electronic equipment

Abstract

The invention provides a robot chassis control method, a storage medium and electronic equipment. The method comprises the steps that a robot runs along a running path; acquiring a radar image acquired by the radar sensor and a video image acquired by the camera at the same moment; calculating the qualified rate of the radar image based on the proportion of missing point clouds in the radar image, and calculating the qualified rate of the video image based on the proportion of invalid pixel points in the video image; determining a main mode based on the qualified rate, obtaining a standard image of the current main mode based on the position of the robot, and determining an abnormal region in a radar image or a video image based on the standard image and the radar image or the video image of the current main mode; marking a preset template image to obtain a first rendered image; and constructing a second rendered image based on the first rendered image at the plurality of continuous moments, inputting the second rendered image into a preset neural network model, and outputting an adjustment instruction by the neural network model.

Description

Technical Field

[0001] The present invention relates to the field of robot technology, and in particular to a control method, storage medium and electronic equipment of a robot chassis. Background Art

[0002] The application of sensors in robotics is crucial. They are like the "sensory organs" of robots, providing them with the ability to perceive the external environment. Through different types of sensors, robots can obtain a lot of information about their surroundings, including distance, temperature, humidity, light, sound, pressure, magnetic field and other physical quantities.

[0003] In terms of autonomous navigation of existing technologies, robots use laser radar, ultrasonic sensors, infrared sensors, etc. to accurately measure the distance to surrounding obstacles, thereby planning the optimal movement path. On industrial production lines, pressure sensors and temperature sensors can monitor the operating status of equipment in real time to ensure the stability and safety of the production process. In addition, visual sensors and image recognition technology enable robots to recognize and process complex visual information, such as identifying object shapes, colors, textures, etc., which is crucial for achieving tasks such as precise grasping, sorting, and assembly.

[0004] Although the robot chassis of the prior art are all equipped with radar or camera equipment, when the robot is actually navigating and avoiding, due to environmental changes, the radar and camera equipment may be affected by the environment and cause insufficient accuracy, making it difficult to achieve precise control of the robot.

[0005] In view of this, the present invention is proposed. Summary of the invention

[0006] The purpose of the present invention is to provide a control method, storage medium and electronic device for a robot chassis. This solution combines devices of multiple modes. On the one hand, it can achieve effective avoidance of moving obstacles. On the other hand, when the obstacle completely blocks the road, the driving path can also be replanned to ensure precise control of the robot.

[0007] The present invention provides a control method for a robot chassis, wherein the robot chassis is equipped with a radar sensor and a camera, and the radar sensor and the camera collect data in real time. The method comprises the following steps:

[0008] A driving path is constructed based on the received destination address and the current position of the robot, and the robot drives along the driving path;

[0009] Acquire a radar image collected by the radar sensor and a video image collected by the camera at the same time;

[0010] The qualified rate of the radar image is calculated based on the proportion of the missing point cloud in the radar image, and the qualified rate of the video image is calculated based on the proportion of the invalid pixel points in the video image;

[0011] Determine the main mode based on the qualified rate of the radar image and the video image, obtain the standard image of the current main mode based on the position of the robot, and determine the abnormal area in the radar image or the video image based on the standard image and the radar image or the video image of the current main mode;

[0012] Marking a preset template image based on the abnormal area to obtain a first rendered image;

[0013] A second rendered image is constructed based on first rendered images at multiple consecutive moments, and the second rendered image is input into a preset neural network model. The neural network model outputs adjustment instructions, and the adjustment instructions include steering avoidance instructions and path replanning instructions. If the adjustment instructions are path replanning instructions, the driving path is reconstructed, and the robot drives along the reconstructed driving path.

[0014] By adopting the above scheme, the robot of this scheme is equipped with a radar sensor and a camera at the same time. During the driving process of the vehicle, the accuracy of the two may decrease due to environmental factors. This scheme can determine the main mode by comparing the data collected by the two, and calculate the abnormal area of ​​the main mode at each moment, construct a first rendering image, construct a second rendering image through multiple consecutive first rendering images, and output adjustment instructions through a neural network model. The second rendering image represents the position of the abnormal area within a period of time. On the one hand, it can achieve effective avoidance of moving obstacles. On the other hand, when the obstacle completely blocks the road, the driving path can also be replanned to ensure precise control of the robot.

[0015] In some embodiments of the present invention, in the step of constructing the driving path, the driving path is constructed by an A* algorithm or a shortest path algorithm.

[0016] In some embodiments of the present invention, in the steps of calculating the qualified rate of radar images based on the proportion of missing point clouds in the radar images, and calculating the qualified rate of video images based on the proportion of invalid pixels in the video images, for radar images, a set number of point clouds is obtained according to the hardware parameters of the radar, and then the number of point clouds of the collected radar images is determined, and the difference between the set number of point clouds and the number of point clouds of the collected radar images is calculated as the number of missing point clouds, and the ratio of the number of missing point clouds to the set number of point clouds is calculated to obtain the qualified rate of the radar image; for video images, invalid pixels are screened out based on the pixel values ​​of the pixels in the video images, and the ratio of invalid pixels to all pixels is calculated to obtain the qualified rate of the video images.

[0017] In some embodiments of the present invention, in the step of determining the main modality based on the qualified rates of radar images and video images, the qualified rates of radar images and video images are compared, and the acquisition modality of the image corresponding to the higher qualified rate is taken as the main modality.

[0018] The above scheme is adopted. In dark environments, the camera is difficult to use due to lack of light; the radar is easily affected in rainy or foggy environments. This scheme determines the current more applicable mode through the pass rate in the actual image, and uses this mode as the applicable mode to ensure the robot's applicability in complex environments.

[0019] In some embodiments of the present invention, in the step of acquiring a standard image of the current main modality based on the position of the robot, and determining an abnormal area in the radar image or video image based on the standard image and a radar image or video image of the current main modality:

[0020] If the main mode is a video, the video image is compared with the standard image, the position of the abnormal pixel is marked, and the video image with the marked abnormal pixel is scanned to determine the abnormal area in the video image;

[0021] If the main mode is radar, the radar image is compared with the standard image to determine the abnormal area in the radar image.

[0022] Using the above scheme, the standard image is a pre-collected image at a location where there is no obstacle. The judgment of the abnormal area often indicates that there is an obstacle in the area, and the obstacle may affect the normal driving of the robot. This scheme compares the actual collected image with the standard image to determine the location where the obstacle may exist, and regards the location as an abnormal area to provide support for the final judgment of the robot.

[0023] In some embodiments of the present invention, in the step of comparing the video image with the standard image, marking the positions of abnormal pixels, scanning the video image with marked abnormal pixels, and determining the abnormal area in the video image, the video image with marked abnormal pixels is scanned based on a preset step size and scanning window, and based on the number of abnormal pixels in the area of ​​the scanning window, it is determined whether the current area of ​​the scanning window is an abnormal area; in the step of comparing the radar image with the standard image and determining the abnormal area in the radar image, the difference in reflection intensity parameters between the standard image and the radar image at the corresponding position is calculated, and if the difference is greater than a preset parameter threshold, the area corresponding to the position is regarded as an abnormal area.

[0024] In some embodiments of the present invention, in the step of marking a preset template image based on the abnormal area to obtain a first rendered image, the video image or radar image after the abnormal area is marked is scaled to the same size as the template image, and the abnormal area in the video image or radar image is marked in the template image to obtain the first rendered image.

[0025] In some embodiments of the present invention, in the step of constructing a second rendered image based on first rendered images at multiple consecutive moments, the first rendered images at multiple consecutive moments are overlapped, and different pixel values ​​are assigned to each pixel point based on the number of overlaps of the pixel points in the abnormal area after the overlap, so as to obtain the second rendered image.

[0026] Using the above scheme, the second rendered image overlaps the abnormal area in the first rendered image. The more times the overlap is performed, the higher the pixel value is. It can intuitively indicate the position of the obstacle, and can also indicate the relative position change between the obstacle and the robot, so that the instructions output by the neural network model can be more accurate.

[0027] Another aspect of the present invention relates to a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned robot chassis control method.

[0028] Another aspect of the present invention also relates to an electronic device, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method.

[0029] In summary, the present invention has the following beneficial effects:

[0030] 1. The robot of this solution is equipped with a radar sensor and a camera at the same time. During the driving process of the vehicle, the accuracy of the two may decrease due to environmental factors. This solution can determine the main mode by comparing the data collected by the two, and calculate the abnormal area of ​​the main mode at each moment, construct a first rendering image, construct a second rendering image through multiple consecutive first rendering images, and output adjustment instructions through a neural network model. The second rendering image represents the position of the abnormal area within a period of time. On the one hand, it can achieve effective avoidance of moving obstacles. On the other hand, when the obstacle completely blocks the road, the driving path can also be replanned to ensure accurate control of the robot;

[0031] 2. In a dark environment, the camera is difficult to use due to lack of light; the radar is easily affected in rainy or foggy environments; this solution determines the currently more applicable mode through the pass rate in the actual image, and uses this mode as the applicable mode to ensure the robot's applicability in complex environments;

[0032] 3. The standard image of this solution is a pre-collected image at a location where there is no obstacle. The determination of an abnormal area often indicates that there is an obstacle in the area, which may affect the normal driving of the robot. This solution compares the actual collected image with the standard image to determine the location where the obstacle may exist, and regards the location as an abnormal area to provide support for the final determination of the robot;

[0033] 4. The second rendered image of this solution overlaps the abnormal area in the first rendered image. The more times the overlaps, the higher the pixel value. It can intuitively indicate the position of the obstacle, and also indicate the relative position change between the obstacle and the robot, so that the instructions output by the neural network model can be more accurate. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0035] Figure 1 A schematic diagram of an implementation method of a robot chassis control method of the present application;

[0036] Figure 2 A schematic diagram of another implementation of the control method for the robot chassis of the present application;

[0037] Figure 3 A schematic diagram of the structure of an electronic device provided in one embodiment of the present application. DETAILED DESCRIPTION

[0038] Exemplary embodiments will be described in detail herein, examples of which are shown in the accompanying drawings. When the following description refers to the drawings, the same numbers in different drawings represent the same or similar elements unless otherwise indicated. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0039] The terms used in the present invention are only for the purpose of describing specific embodiments and are not intended to limit the present invention. The singular forms "a", "the" and "the" used in the present invention and the appended claims are also intended to include plural forms unless the context clearly indicates other meanings. It should also be understood that the term "and / or" used herein refers to and includes any or all possible combinations of one or more associated listed items.

[0040] In the existing technology, robots are intelligent devices that integrate multiple advanced technologies and are widely used in the fields of industry, electricity, security, etc. They can perform inspection tasks autonomously or remotely, and achieve comprehensive and detailed monitoring of the target area through high-precision sensors and intelligent algorithms.

[0041] Robots usually have strong mobility and can move freely in complex environments. Whether it is a narrow passage or a rugged terrain, they can respond flexibly to ensure the smooth progress of inspection work. During power inspections, inspection robots can climb along wires or towers to conduct comprehensive inspections of power equipment and promptly discover potential safety hazards.

[0042] In addition to mobility, the robot is also equipped with a variety of sensors, such as infrared thermal imagers, visible light cameras, sound collectors, etc. These sensors can collect various data of the target area in real time, including temperature, humidity, sound, images, etc. By analyzing and processing the collected data through intelligent algorithms, the robot can accurately judge the operating status of the equipment, predict possible failures, and provide timely and accurate warning information for operation and maintenance personnel.

[0043] like Figure 1 As shown, the present invention provides a control method for a robot chassis, wherein the robot chassis is equipped with a radar sensor and a camera, and the radar sensor and the camera collect data in real time. The steps of the method include:

[0044] In a specific implementation process, the radar sensor and the camera can be installed in front of the robot chassis, the radar sensor can be a laser radar (Lidar) or a millimeter wave radar (RADAR), and the camera can be a camera (Camera).

[0045] Step S100, constructing a driving path based on the received destination address and the current position of the robot, and the robot drives along the driving path;

[0046] In a specific implementation process, a map of the current area is pre-stored in the robot, and the robot can use the shortest path algorithm or the A* algorithm to construct a driving path.

[0047] In the specific implementation process, the destination address can be sent by the staff's mobile terminal to the robot, and the robot determines its current position based on its own positioning device.

[0048] Step S200, acquiring a radar image captured by the radar sensor and a video image captured by the camera at the same time;

[0049] In a specific implementation process, the radar sensor and the camera both perform real-time acquisition, and the radar image acquired by the radar sensor and the video image acquired by the camera are stored at each moment.

[0050] Step S300, calculating the qualified rate of the radar image based on the proportion of missing point clouds in the radar image, and calculating the qualified rate of the video image based on the proportion of invalid pixel points in the video image;

[0051] During the specific implementation process, a parameter for the number of point clouds is fixed in the working parameters of the radar. The actual number of point clouds is compared with the preset working parameters, and the qualified rate of the radar image is calculated, wherein the missing point cloud is the error of the radar; when the light is dim, the pixel points of the camera are darker, and the pixel value is lower. The pixel value below the pixel value threshold is regarded as an invalid pixel point, and the proportion of invalid pixels is calculated, which is the qualified rate of the video image.

[0052] Specifically, the representation of missing points in video images and radar point clouds is different, because for video images, regardless of whether the data is missing, the image data will fill the entire area, and each pixel of the image will have grayscale values, RGB values, etc.; while for radar point clouds, if the data is missing for some reason, it will be reflected as a direct missing point in the radar point cloud. Therefore, for video images, this solution directly regards the missing points as the part with missing information for radar point clouds.

[0053] Step S400, determining the main mode based on the qualified rates of the radar image and the video image, acquiring a standard image of the current main mode based on the position of the robot, and determining an abnormal area in the radar image or the video image based on the standard image and the radar image or the video image of the current main mode;

[0054] Step S500, marking a preset template image based on the abnormal area to obtain a first rendered image;

[0055] Step S600, constructing a second rendering image based on the first rendering images at multiple consecutive moments, inputting the second rendering image into a preset neural network model, and the neural network model outputting adjustment instructions, the adjustment instructions including steering avoidance instructions and path replanning instructions. If the adjustment instructions are path replanning instructions, the driving path is reconstructed, and the robot drives along the reconstructed driving path.

[0056] During the specific implementation process, the neural network model can be a convolutional neural network model, and the adjustment instruction includes a straight-ahead instruction, a left-turn instruction, a right-turn instruction, or a path replanning instruction. If the adjustment instruction is a left-turn instruction or a right-turn instruction, the adjustment instruction also includes the angle of the left turn or the right turn.

[0057] In a specific implementation process, if the adjustment instruction is a path replanning instruction, the A* algorithm or the shortest path algorithm is used to reconstruct the driving path.

[0058] In the specific implementation process, the main mode at the previous moment and the main mode at the next moment of this scheme can be different modes, and in the construction of the second rendered image, the first rendered images at multiple moments are combined, that is, the collected data of multiple modes are combined to ensure the judgment accuracy of the final neural network model.

[0059] By adopting the above scheme, the robot of this scheme is equipped with a radar sensor and a camera at the same time. During the driving process of the vehicle, the accuracy of the two may decrease due to environmental factors. This scheme can determine the main mode by comparing the data collected by the two, and calculate the abnormal area of ​​the main mode at each moment, construct a first rendering image, construct a second rendering image through multiple consecutive first rendering images, and output adjustment instructions through a neural network model. The second rendering image represents the position of the abnormal area within a period of time. On the one hand, it can achieve effective avoidance of moving obstacles. On the other hand, when the obstacle completely blocks the road, the driving path can also be replanned to ensure precise control of the robot.

[0060] In some embodiments of the present invention, in the step of constructing the driving path, the driving path is constructed by an A* algorithm or a shortest path algorithm.

[0061] In some embodiments of the present invention, in the steps of calculating the qualified rate of radar images based on the proportion of missing point clouds in the radar images, and calculating the qualified rate of video images based on the proportion of invalid pixels in the video images, for radar images, a set number of point clouds is obtained according to the hardware parameters of the radar, and then the number of point clouds of the collected radar images is determined, and the difference between the set number of point clouds and the number of point clouds of the collected radar images is calculated as the number of missing point clouds, and the ratio of the number of missing point clouds to the set number of point clouds is calculated to obtain the qualified rate of the radar image; for video images, invalid pixels are screened out based on the pixel values ​​of the pixels in the video images, and the ratio of invalid pixels to all pixels is calculated to obtain the qualified rate of the video images.

[0062] like Figure 2As shown, in some embodiments of the present invention, the step of determining the main mode based on the qualified rates of radar images and video images includes step S410, comparing the qualified rates of radar images and video images, and taking the acquisition mode of the image corresponding to the higher qualified rate as the main mode.

[0063] The above scheme is adopted. In dark environments, the camera is difficult to use due to lack of light; the radar is easily affected in rainy or foggy environments. This scheme determines the current more applicable mode through the pass rate in the actual image, and uses this mode as the applicable mode to ensure the robot's applicability in complex environments.

[0064] In some embodiments of the present invention, the step of acquiring a standard image of the current main modality based on the position of the robot, and determining an abnormal area in the radar image or video image based on the standard image and the radar image or video image of the current main modality includes:

[0065] Step S420: if the main mode is video, the video image is compared with the standard image, the position of the abnormal pixel is marked, the video image with the marked abnormal pixel is scanned, and the abnormal area in the video image is determined;

[0066] Step S430: If the main mode is radar, the radar image is compared with the standard image to determine the abnormal area in the radar image.

[0067] Using the above scheme, the standard image is a pre-collected image at a location where there is no obstacle. The judgment of the abnormal area often indicates that there is an obstacle in the area, and the obstacle may affect the normal driving of the robot. This scheme compares the actual collected image with the standard image to determine the location where the obstacle may exist, and regards the location as an abnormal area to provide support for the final judgment of the robot.

[0068] In some embodiments of the present invention, in the step of comparing the video image with the standard image, marking the positions of abnormal pixels, scanning the video image with marked abnormal pixels, and determining the abnormal area in the video image, the video image with marked abnormal pixels is scanned based on a preset step size and scanning window, and based on the number of abnormal pixels in the area of ​​the scanning window, it is determined whether the current area of ​​the scanning window is an abnormal area; in the step of comparing the radar image with the standard image and determining the abnormal area in the radar image, the difference in reflection intensity parameters between the standard image and the radar image at the corresponding position is calculated, and if the difference is greater than a preset parameter threshold, the area corresponding to the position is regarded as an abnormal area.

[0069] In a specific implementation process, in the step of scanning a video image marked with abnormal pixels based on a preset step size and a scanning window, and determining whether the current area of ​​the scanning window is an abnormal area based on the number of abnormal pixels in the area of ​​the scanning window, the scanning window scans line by line from the upper left corner of the video image to the lower right corner of the video image, and moves specifically based on a preset step size. At each stop of the scanning window, the ratio of the number of abnormal pixels in the area of ​​the scanning window to the total number of pixels in the scanning window is calculated, and if the ratio is greater than a preset abnormal threshold, the area where the current scanning window is located is regarded as an abnormal area.

[0070] In some embodiments of the present invention, in the step of marking a preset template image based on the abnormal area to obtain a first rendered image, the video image or radar image after the abnormal area is marked is scaled to the same size as the template image, and the abnormal area in the video image or radar image is marked in the template image to obtain the first rendered image.

[0071] In a specific implementation process, the template image is an image in which all pixels are preset to have the same value. Specifically, the pixel value at each position of the template image may be 0.

[0072] In some embodiments of the present invention, in the step of constructing a second rendered image based on first rendered images at multiple consecutive moments, the first rendered images at multiple consecutive moments are overlapped, and different pixel values ​​are assigned to each pixel point based on the number of overlaps of the pixel points in the abnormal area after the overlap, so as to obtain the second rendered image.

[0073] In a specific implementation process, in the step of assigning different pixel values ​​to the pixel points based on the number of overlaps of each pixel point in the abnormal area after overlapping to obtain the second rendered image, the more overlaps the pixel points have, the higher the assigned pixel value is.

[0074] Using the above scheme, the second rendered image overlaps the abnormal area in the first rendered image. The more times the overlap is performed, the higher the pixel value is. It can intuitively indicate the position of the obstacle, and can also indicate the relative position change between the obstacle and the robot, so that the instructions output by the neural network model can be more accurate.

[0075] Another aspect of the present invention relates to a storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the aforementioned robot chassis control method.

[0076] Another aspect of the present invention also relates to an electronic device, which includes a computer device, wherein the computer device includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the method.

[0077] like Figure 3 As shown, an embodiment of the present application provides an electronic device, the device comprising: a processor and a memory storing computer program instructions; when the processor executes the computer program instructions, the above-mentioned 3D visualization meteorological data monitoring method is implemented.

[0078] The electronic device may include a processor 1201 and a memory 1202 storing computer program instructions.

[0079] Specifically, the processor 1201 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0080] The memory 1202 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1202 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1202 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1202 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1202 is a non-volatile solid-state memory.

[0081] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0082] The processor 1201 implements any one of the battery thermal runaway parameter determination methods in the above embodiments by reading and executing computer program instructions stored in the memory 1202 .

[0083] In one example, the electronic device may further include a communication interface 1203 and a bus 1210. Figure 3 As shown, the processor 1201, the memory 1202, and the communication interface 1203 are connected via a bus 1210 and communicate with each other.

[0084] The communication interface 1203 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0085] Bus 1210 includes hardware, software or both, and the components of electronic equipment are coupled to each other. For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front-side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1210 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the present application considers any suitable bus or interconnection.

[0086] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0087] The functional blocks shown in the above-described block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application-specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROMs, flash memories, erasable ROMs (EROMs), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0088] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0089] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device (system) and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0090] The above is only a specific implementation of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A control method for a robot chassis, characterized in that: The robot chassis is equipped with a radar sensor and a camera, which collect data in real time. The method comprises the following steps: A driving path is constructed based on the received destination address and the current position of the robot, and the robot drives along the driving path; Acquire a radar image collected by the radar sensor and a video image collected by the camera at the same time; The qualified rate of the radar image is calculated based on the proportion of the missing point cloud in the radar image, and the qualified rate of the video image is calculated based on the proportion of the invalid pixel points in the video image; Determine the main mode based on the qualified rate of the radar image and the video image, obtain the standard image of the current main mode based on the position of the robot, and determine the abnormal area in the radar image or the video image based on the standard image and the radar image or the video image of the current main mode; Marking a preset template image based on the abnormal area to obtain a first rendered image; A second rendered image is constructed based on first rendered images at multiple consecutive moments, and the second rendered image is input into a preset neural network model. The neural network model outputs adjustment instructions, and the adjustment instructions include steering avoidance instructions and path replanning instructions. If the adjustment instructions are path replanning instructions, the driving path is reconstructed, and the robot drives along the reconstructed driving path.

2. The control method of the robot chassis according to claim 1, characterized in that: In the step of constructing the driving path, the driving path is constructed by using an A* algorithm or a shortest path algorithm.

3. The control method of the robot chassis according to claim 1, characterized in that: In the steps of calculating the qualified rate of radar images based on the proportion of missing point clouds in the radar images, and calculating the qualified rate of video images based on the proportion of invalid pixel points in the video images, for radar images, a set number of point clouds is obtained according to the hardware parameters of the radar, and then the number of point clouds of the collected radar images is determined, the difference between the set number of point clouds and the number of point clouds of the collected radar images is calculated as the number of missing point clouds, and the ratio of the number of missing point clouds to the set number of point clouds is calculated to obtain the qualified rate of radar images; for video images, invalid pixel points are screened out based on the pixel values ​​of the pixel points in the video images, and the ratio of invalid pixel points to all pixel points is calculated to obtain the qualified rate of the video images.

4. The control method of the robot chassis according to claim 1, characterized in that: In the step of determining the main mode based on the qualified rates of the radar image and the video image, the qualified rates of the radar image and the video image are compared, and the acquisition mode of the image corresponding to the higher qualified rate is taken as the main mode.

5. The control method of the robot chassis according to claim 1, characterized in that: In the step of acquiring a standard image of the current main modality based on the position of the robot, and determining an abnormal area in the radar image or video image based on the standard image and the radar image or video image of the current main modality: If the main mode is a video, the video image is compared with the standard image, the position of the abnormal pixel is marked, and the video image with the marked abnormal pixel is scanned to determine the abnormal area in the video image; If the main mode is radar, the radar image is compared with the standard image to determine the abnormal area in the radar image.

6. The control method of the robot chassis according to claim 5, characterized in that: In the step of comparing the video image with the standard image, marking the position of the abnormal pixel points, scanning the video image with the marked abnormal pixel points, and determining the abnormal area in the video image, the video image with the marked abnormal pixel points is scanned based on a preset step size and scanning window, and based on the number of abnormal pixel points in the area of ​​the scanning window, it is determined whether the current area of ​​the scanning window is an abnormal area; in the step of comparing the radar image with the standard image and determining the abnormal area in the radar image, the difference in reflection intensity parameters between the standard image and the radar image at the corresponding position is calculated, and if the difference is greater than a preset parameter threshold, the area corresponding to the position is regarded as an abnormal area.

7. The control method of the robot chassis according to any one of claims 1 to 6, characterized in that: In the step of marking a preset template image based on the abnormal area to obtain a first rendered image, the video image or radar image after the abnormal area is marked is scaled to the same size as the template image, and the abnormal area in the video image or radar image is marked in the template image to obtain the first rendered image.

8. The control method of the robot chassis according to claim 7, characterized in that: In the step of constructing a second rendered image based on the first rendered images at multiple consecutive moments, the first rendered images at multiple consecutive moments are overlapped, and different pixel values ​​are assigned to each pixel point in the abnormal area after overlapping, so as to obtain the second rendered image.

9. A storage medium, characterized in that: A computer program is stored thereon, and when the computer program is executed by a processor, the steps of the control method of the robot chassis as described in any one of claims 1 to 8 are implemented.

10. An electronic device, characterized in that: The electronic device includes a computer device, which includes a processor and a memory, wherein the memory stores computer instructions, and the processor is used to execute the computer instructions stored in the memory. When the computer instructions are executed by the processor, the system implements the steps implemented by the control method of the robot chassis described in any one of claims 1-8.

Images